W0703 18:10:40.682829 1 cmd.go:257] Using insecure, self-signed certificates I0703 18:10:41.132402 1 start.go:138] Unable to read service ca bundle: open /var/run/configmaps/service-ca-bundle/service-ca.crt: no such file or directory I0703 18:10:41.133602 1 observer_polling.go:159] Starting file observer I0703 18:10:42.518831 1 operator.go:60] Starting insights-operator v0.0.0-master+$Format:%H$ I0703 18:10:42.519141 1 legacy_config.go:327] Current config: {"report":false,"storagePath":"/var/lib/insights-operator","interval":"2h","endpoint":"https://console.redhat.com/api/ingress/v1/upload","conditionalGathererEndpoint":"https://console.redhat.com/api/gathering/v2/%s/gathering_rules","pull_report":{"endpoint":"https://console.redhat.com/api/insights-results-aggregator/v2/cluster/%s/reports","delay":"60s","timeout":"3000s","min_retry":"30s"},"impersonate":"system:serviceaccount:openshift-insights:gather","enableGlobalObfuscation":false,"ocm":{"scaEndpoint":"https://api.openshift.com/api/accounts_mgmt/v1/entitlement_certificates","scaInterval":"8h","scaDisabled":false,"clusterTransferEndpoint":"https://api.openshift.com/api/accounts_mgmt/v1/cluster_transfers/","clusterTransferInterval":"12h"},"disableInsightsAlerts":false,"processingStatusEndpoint":"https://console.redhat.com/api/insights-results-aggregator/v2/cluster/%s/request/%s/status","reportEndpointTechPreview":"https://console.redhat.com/api/insights-results-aggregator/v2/cluster/%s/request/%s/report"} I0703 18:10:42.519676 1 secure_serving.go:57] Forcing use of http/1.1 only W0703 18:10:42.519700 1 secure_serving.go:69] Use of insecure cipher 'TLS_ECDHE_ECDSA_WITH_AES_128_CBC_SHA256' detected. W0703 18:10:42.519704 1 secure_serving.go:69] Use of insecure cipher 'TLS_ECDHE_RSA_WITH_AES_128_CBC_SHA256' detected. W0703 18:10:42.519708 1 secure_serving.go:69] Use of insecure cipher 'TLS_RSA_WITH_AES_128_GCM_SHA256' detected. W0703 18:10:42.519712 1 secure_serving.go:69] Use of insecure cipher 'TLS_RSA_WITH_AES_256_GCM_SHA384' detected. W0703 18:10:42.519716 1 secure_serving.go:69] Use of insecure cipher 'TLS_RSA_WITH_AES_128_CBC_SHA' detected. W0703 18:10:42.519719 1 secure_serving.go:69] Use of insecure cipher 'TLS_RSA_WITH_AES_256_CBC_SHA' detected. I0703 18:10:42.520078 1 simple_featuregate_reader.go:171] Starting feature-gate-detector I0703 18:10:42.523732 1 operator.go:125] FeatureGates initialized: knownFeatureGates=[AdditionalRoutingCapabilities AdminNetworkPolicy AlibabaPlatform AzureWorkloadIdentity BuildCSIVolumes CPMSMachineNamePrefix ConsolePluginContentSecurityPolicy ExternalOIDC ExternalOIDCWithUIDAndExtraClaimMappings GatewayAPI GatewayAPIController HighlyAvailableArbiter ImageVolume IngressControllerLBSubnetsAWS KMSv1 MachineConfigNodes ManagedBootImages ManagedBootImagesAWS MetricsCollectionProfiles NetworkDiagnosticsConfig NetworkLiveMigration NetworkSegmentation PinnedImages ProcMountType RouteAdvertisements RouteExternalCertificate ServiceAccountTokenNodeBinding SetEIPForNLBIngressController SigstoreImageVerification StoragePerformantSecurityPolicy UpgradeStatus UserNamespacesPodSecurityStandards UserNamespacesSupport VSphereMultiDisk VSphereMultiNetworks AWSClusterHostedDNS AWSClusterHostedDNSInstall AWSDedicatedHosts AWSServiceLBNetworkSecurityGroup AutomatedEtcdBackup AzureClusterHostedDNSInstall AzureDedicatedHosts AzureMultiDisk BootImageSkewEnforcement BootcNodeManagement ClusterAPIInstall ClusterAPIInstallIBMCloud ClusterMonitoringConfig ClusterVersionOperatorConfiguration DNSNameResolver DualReplica DyanmicServiceEndpointIBMCloud DynamicResourceAllocation EtcdBackendQuota EventedPLEG Example Example2 ExternalSnapshotMetadata GCPClusterHostedDNS GCPClusterHostedDNSInstall GCPCustomAPIEndpoints GCPCustomAPIEndpointsInstall ImageModeStatusReporting ImageStreamImportMode IngressControllerDynamicConfigurationManager InsightsConfig InsightsConfigAPI InsightsOnDemandDataGather IrreconcilableMachineConfig KMSEncryptionProvider MachineAPIMigration MachineAPIOperatorDisableMachineHealthCheckController ManagedBootImagesAzure ManagedBootImagesvSphere MaxUnavailableStatefulSet MinimumKubeletVersion MixedCPUsAllocation MultiArchInstallAzure MultiDiskSetup MutatingAdmissionPolicy NewOLM NewOLMCatalogdAPIV1Metas NewOLMOwnSingleNamespace NewOLMPreflightPermissionChecks NewOLMWebhookProviderOpenshiftServiceCA NoRegistryClusterOperations NodeSwap NutanixMultiSubnets OVNObservability OpenShiftPodSecurityAdmission PreconfiguredUDNAddresses SELinuxMount ShortCertRotation SignatureStores SigstoreImageVerificationPKI TranslateStreamCloseWebsocketRequests VSphereConfigurableMaxAllowedBlockVolumesPerNode VSphereHostVMGroupZonal VSphereMixedNodeEnv VolumeAttributesClass VolumeGroupSnapshot] I0703 18:10:42.523784 1 event.go:377] Event(v1.ObjectReference{Kind:"Deployment", Namespace:"openshift-insights", Name:"insights-operator", UID:"2e6c8367-fed0-46a2-b41f-3ea318786def", APIVersion:"apps/v1", ResourceVersion:"", FieldPath:""}): type: 'Normal' reason: 'FeatureGatesInitialized' FeatureGates updated to featuregates.Features{Enabled:[]v1.FeatureGateName{"AdditionalRoutingCapabilities", "AdminNetworkPolicy", "AlibabaPlatform", "AzureWorkloadIdentity", "BuildCSIVolumes", "CPMSMachineNamePrefix", "ConsolePluginContentSecurityPolicy", "ExternalOIDC", "ExternalOIDCWithUIDAndExtraClaimMappings", "GatewayAPI", "GatewayAPIController", "HighlyAvailableArbiter", "ImageVolume", "IngressControllerLBSubnetsAWS", "KMSv1", "MachineConfigNodes", "ManagedBootImages", "ManagedBootImagesAWS", "MetricsCollectionProfiles", "NetworkDiagnosticsConfig", "NetworkLiveMigration", "NetworkSegmentation", "PinnedImages", "ProcMountType", "RouteAdvertisements", "RouteExternalCertificate", "ServiceAccountTokenNodeBinding", "SetEIPForNLBIngressController", "SigstoreImageVerification", "StoragePerformantSecurityPolicy", "UpgradeStatus", "UserNamespacesPodSecurityStandards", "UserNamespacesSupport", "VSphereMultiDisk", "VSphereMultiNetworks"}, Disabled:[]v1.FeatureGateName{"AWSClusterHostedDNS", "AWSClusterHostedDNSInstall", "AWSDedicatedHosts", "AWSServiceLBNetworkSecurityGroup", "AutomatedEtcdBackup", "AzureClusterHostedDNSInstall", "AzureDedicatedHosts", "AzureMultiDisk", "BootImageSkewEnforcement", "BootcNodeManagement", "ClusterAPIInstall", "ClusterAPIInstallIBMCloud", "ClusterMonitoringConfig", "ClusterVersionOperatorConfiguration", "DNSNameResolver", "DualReplica", "DyanmicServiceEndpointIBMCloud", "DynamicResourceAllocation", "EtcdBackendQuota", "EventedPLEG", "Example", "Example2", "ExternalSnapshotMetadata", "GCPClusterHostedDNS", "GCPClusterHostedDNSInstall", "GCPCustomAPIEndpoints", "GCPCustomAPIEndpointsInstall", "ImageModeStatusReporting", "ImageStreamImportMode", "IngressControllerDynamicConfigurationManager", "InsightsConfig", "InsightsConfigAPI", "InsightsOnDemandDataGather", "IrreconcilableMachineConfig", "KMSEncryptionProvider", "MachineAPIMigration", "MachineAPIOperatorDisableMachineHealthCheckController", "ManagedBootImagesAzure", "ManagedBootImagesvSphere", "MaxUnavailableStatefulSet", "MinimumKubeletVersion", "MixedCPUsAllocation", "MultiArchInstallAzure", "MultiDiskSetup", "MutatingAdmissionPolicy", "NewOLM", "NewOLMCatalogdAPIV1Metas", "NewOLMOwnSingleNamespace", "NewOLMPreflightPermissionChecks", "NewOLMWebhookProviderOpenshiftServiceCA", "NoRegistryClusterOperations", "NodeSwap", "NutanixMultiSubnets", "OVNObservability", "OpenShiftPodSecurityAdmission", "PreconfiguredUDNAddresses", "SELinuxMount", "ShortCertRotation", "SignatureStores", "SigstoreImageVerificationPKI", "TranslateStreamCloseWebsocketRequests", "VSphereConfigurableMaxAllowedBlockVolumesPerNode", "VSphereHostVMGroupZonal", "VSphereMixedNodeEnv", "VolumeAttributesClass", "VolumeGroupSnapshot"}} I0703 18:10:42.524519 1 requestheader_controller.go:180] Starting RequestHeaderAuthRequestController I0703 18:10:42.524531 1 configmap_cafile_content.go:205] "Starting controller" name="client-ca::kube-system::extension-apiserver-authentication::client-ca-file" I0703 18:10:42.524541 1 shared_informer.go:313] Waiting for caches to sync for RequestHeaderAuthRequestController I0703 18:10:42.524538 1 configmap_cafile_content.go:205] "Starting controller" name="client-ca::kube-system::extension-apiserver-authentication::requestheader-client-ca-file" I0703 18:10:42.524547 1 shared_informer.go:313] Waiting for caches to sync for client-ca::kube-system::extension-apiserver-authentication::client-ca-file I0703 18:10:42.524551 1 shared_informer.go:313] Waiting for caches to sync for client-ca::kube-system::extension-apiserver-authentication::requestheader-client-ca-file I0703 18:10:42.524936 1 dynamic_serving_content.go:135] "Starting controller" name="serving-cert::/tmp/serving-cert-1315631682/tls.crt::/tmp/serving-cert-1315631682/tls.key" I0703 18:10:42.525358 1 secure_serving.go:213] Serving securely on [::]:8443 I0703 18:10:42.525394 1 tlsconfig.go:243] "Starting DynamicServingCertificateController" W0703 18:10:42.530373 1 configmapobserver.go:64] Cannot get the configuration config map: configmaps "insights-config" not found. Default configuration is used. I0703 18:10:42.530405 1 secretconfigobserver.go:216] Legacy configuration set: enabled=false endpoint=https://console.redhat.com/api/ingress/v1/upload conditional_gatherer_endpoint=https://console.redhat.com/api/gathering/v2/%s/gathering_rules interval=2h0m0s token=false reportEndpoint=https://console.redhat.com/api/insights-results-aggregator/v2/cluster/%s/reports initialPollingDelay=1m0s minRetryTime=30s pollingTimeout=50m0s processingStatusEndpoint=https://console.redhat.com/api/insights-results-aggregator/v2/cluster/%s/request/%s/status I0703 18:10:42.530525 1 base_controller.go:76] Waiting for caches to sync for ConfigController I0703 18:10:42.535893 1 secretconfigobserver.go:249] Found cloud.openshift.com token I0703 18:10:42.535914 1 secretconfigobserver.go:204] Legacy configuration updated: enabled=true endpoint=https://console.redhat.com/api/ingress/v1/upload conditional_gatherer_endpoint=https://console.redhat.com/api/gathering/v2/%s/gathering_rules interval=2h0m0s token=true reportEndpoint=https://console.redhat.com/api/insights-results-aggregator/v2/cluster/%s/reports initialPollingDelay=1m0s minRetryTime=30s pollingTimeout=50m0s processingStatusEndpoint=https://console.redhat.com/api/insights-results-aggregator/v2/cluster/%s/request/%s/status I0703 18:10:42.540543 1 secretconfigobserver.go:119] support secret does not exist I0703 18:10:42.544728 1 secretconfigobserver.go:249] Found cloud.openshift.com token I0703 18:10:42.548480 1 secretconfigobserver.go:119] support secret does not exist I0703 18:10:42.552401 1 recorder.go:161] Pruning old reports every 7h55m56s, max age is 288h0m0s I0703 18:10:42.556965 1 controllerstatus.go:80] name=insightsuploader healthy=true reason= message= I0703 18:10:42.556985 1 insightsuploader.go:86] Reporting status periodically to https://console.redhat.com/api/ingress/v1/upload every 2h0m0s, starting in 1m30s I0703 18:10:42.557003 1 controllerstatus.go:80] name=insightsreport healthy=true reason= message= I0703 18:10:42.557012 1 insightsreport.go:296] Starting report retriever I0703 18:10:42.557020 1 insightsreport.go:298] Insights analysis reports will be downloaded from the https://console.redhat.com/api/insights-results-aggregator/v2/cluster/%s/reports endpoint with a delay of 1m0s I0703 18:10:42.557022 1 periodic.go:209] Running clusterconfig gatherer I0703 18:10:42.557069 1 tasks_processing.go:45] number of workers: 32 I0703 18:10:42.557098 1 tasks_processing.go:69] worker 1 listening for tasks. I0703 18:10:42.557107 1 tasks_processing.go:69] worker 6 listening for tasks. I0703 18:10:42.557111 1 tasks_processing.go:69] worker 0 listening for tasks. I0703 18:10:42.557119 1 tasks_processing.go:69] worker 4 listening for tasks. I0703 18:10:42.557122 1 tasks_processing.go:69] worker 3 listening for tasks. I0703 18:10:42.557127 1 tasks_processing.go:69] worker 20 listening for tasks. I0703 18:10:42.557132 1 tasks_processing.go:69] worker 7 listening for tasks. I0703 18:10:42.557134 1 tasks_processing.go:71] worker 3 working on operators task. I0703 18:10:42.557119 1 tasks_processing.go:71] worker 0 working on nodenetworkstates task. I0703 18:10:42.557133 1 tasks_processing.go:69] worker 19 listening for tasks. I0703 18:10:42.557144 1 tasks_processing.go:69] worker 22 listening for tasks. I0703 18:10:42.557141 1 tasks_processing.go:69] worker 8 listening for tasks. I0703 18:10:42.557150 1 tasks_processing.go:69] worker 9 listening for tasks. I0703 18:10:42.557154 1 tasks_processing.go:69] worker 23 listening for tasks. I0703 18:10:42.557138 1 tasks_processing.go:69] worker 21 listening for tasks. I0703 18:10:42.557160 1 tasks_processing.go:69] worker 13 listening for tasks. I0703 18:10:42.557161 1 tasks_processing.go:69] worker 10 listening for tasks. I0703 18:10:42.557165 1 tasks_processing.go:69] worker 24 listening for tasks. I0703 18:10:42.557167 1 tasks_processing.go:69] worker 11 listening for tasks. I0703 18:10:42.557174 1 tasks_processing.go:69] worker 25 listening for tasks. I0703 18:10:42.557178 1 tasks_processing.go:69] worker 16 listening for tasks. I0703 18:10:42.557182 1 tasks_processing.go:69] worker 26 listening for tasks. I0703 18:10:42.557182 1 tasks_processing.go:69] worker 14 listening for tasks. I0703 18:10:42.557189 1 tasks_processing.go:69] worker 15 listening for tasks. I0703 18:10:42.557189 1 tasks_processing.go:69] worker 17 listening for tasks. I0703 18:10:42.557193 1 tasks_processing.go:69] worker 27 listening for tasks. I0703 18:10:42.557125 1 tasks_processing.go:71] worker 4 working on image_registries task. I0703 18:10:42.557199 1 tasks_processing.go:69] worker 28 listening for tasks. I0703 18:10:42.557200 1 tasks_processing.go:69] worker 18 listening for tasks. I0703 18:10:42.557207 1 tasks_processing.go:69] worker 29 listening for tasks. I0703 18:10:42.557211 1 tasks_processing.go:69] worker 31 listening for tasks. I0703 18:10:42.557212 1 tasks_processing.go:71] worker 1 working on aggregated_monitoring_cr_names task. I0703 18:10:42.557215 1 tasks_processing.go:71] worker 6 working on olm_operators task. I0703 18:10:42.557282 1 tasks_processing.go:69] worker 30 listening for tasks. I0703 18:10:42.557284 1 tasks_processing.go:69] worker 2 listening for tasks. I0703 18:10:42.557134 1 tasks_processing.go:69] worker 5 listening for tasks. I0703 18:10:42.557416 1 tasks_processing.go:71] worker 9 working on machine_config_pools task. I0703 18:10:42.557430 1 tasks_processing.go:71] worker 14 working on tsdb_status task. W0703 18:10:42.557451 1 gather_prometheus_tsdb_status.go:38] Unable to load metrics client, tsdb status cannot be collected: open /var/run/configmaps/service-ca-bundle/service-ca.crt: no such file or directory I0703 18:10:42.557500 1 tasks_processing.go:71] worker 23 working on ingress_certificates task. I0703 18:10:42.557542 1 gather.go:177] gatherer "clusterconfig" function "tsdb_status" took 66.505µs to process 0 records I0703 18:10:42.557553 1 tasks_processing.go:71] worker 25 working on machine_autoscalers task. I0703 18:10:42.557589 1 tasks_processing.go:71] worker 13 working on number_of_pods_and_netnamespaces_with_sdn_annotations task. I0703 18:10:42.557638 1 tasks_processing.go:71] worker 30 working on node_logs task. I0703 18:10:42.557366 1 tasks_processing.go:71] worker 2 working on mutating_webhook_configurations task. I0703 18:10:42.557421 1 tasks_processing.go:71] worker 5 working on crds task. I0703 18:10:42.557598 1 tasks_processing.go:71] worker 29 working on authentication task. I0703 18:10:42.557505 1 tasks_processing.go:71] worker 14 working on nodes task. I0703 18:10:42.558224 1 tasks_processing.go:71] worker 8 working on ceph_cluster task. I0703 18:10:42.557375 1 tasks_processing.go:71] worker 20 working on cost_management_metrics_configs task. I0703 18:10:42.557379 1 tasks_processing.go:71] worker 7 working on version task. I0703 18:10:42.557385 1 tasks_processing.go:71] worker 19 working on install_plans task. I0703 18:10:42.557125 1 tasks_processing.go:69] worker 12 listening for tasks. I0703 18:10:42.558883 1 tasks_processing.go:71] worker 12 working on service_accounts task. I0703 18:10:42.557411 1 tasks_processing.go:71] worker 26 working on machine_configs task. I0703 18:10:42.557421 1 tasks_processing.go:71] worker 28 working on infrastructures task. I0703 18:10:42.557468 1 tasks_processing.go:71] worker 17 working on openshift_logging task. I0703 18:10:42.557476 1 tasks_processing.go:71] worker 15 working on schedulers task. I0703 18:10:42.557486 1 tasks_processing.go:71] worker 27 working on machines task. I0703 18:10:42.557496 1 tasks_processing.go:71] worker 10 working on silenced_alerts task. W0703 18:10:42.559579 1 gather_silenced_alerts.go:38] Unable to load alerts client, no alerts will be collected: open /var/run/configmaps/service-ca-bundle/service-ca.crt: no such file or directory I0703 18:10:42.559595 1 tasks_processing.go:71] worker 10 working on image task. I0703 18:10:42.557569 1 tasks_processing.go:71] worker 21 working on container_runtime_configs task. I0703 18:10:42.557580 1 tasks_processing.go:71] worker 16 working on overlapping_namespace_uids task. I0703 18:10:42.557509 1 tasks_processing.go:71] worker 11 working on pod_network_connectivity_checks task. I0703 18:10:42.557593 1 tasks_processing.go:71] worker 24 working on networks task. I0703 18:10:42.557606 1 tasks_processing.go:71] worker 31 working on sap_config task. I0703 18:10:42.557604 1 tasks_processing.go:71] worker 18 working on jaegers task. I0703 18:10:42.557390 1 tasks_processing.go:71] worker 22 working on sap_datahubs task. I0703 18:10:42.559672 1 gather.go:177] gatherer "clusterconfig" function "silenced_alerts" took 38.023µs to process 0 records I0703 18:10:42.563033 1 tasks_processing.go:71] worker 0 working on machine_sets task. I0703 18:10:42.563041 1 gather.go:177] gatherer "clusterconfig" function "nodenetworkstates" took 5.88596ms to process 0 records I0703 18:10:42.563286 1 controller.go:128] Initializing last reported time to 0001-01-01T00:00:00Z I0703 18:10:42.563306 1 controller.go:212] Source periodic-clusterconfig *controllerstatus.Simple is not ready I0703 18:10:42.563315 1 controller.go:212] Source periodic-conditional *controllerstatus.Simple is not ready I0703 18:10:42.563319 1 controller.go:212] Source periodic-workloads *controllerstatus.Simple is not ready I0703 18:10:42.563338 1 controller.go:489] The operator is still being initialized I0703 18:10:42.563349 1 controller.go:512] The operator is healthy I0703 18:10:42.567858 1 tasks_processing.go:71] worker 25 working on openstack_controlplanes task. I0703 18:10:42.567909 1 gather.go:177] gatherer "clusterconfig" function "machine_autoscalers" took 10.262949ms to process 0 records I0703 18:10:42.567931 1 gather.go:177] gatherer "clusterconfig" function "ceph_cluster" took 9.574215ms to process 0 records I0703 18:10:42.567939 1 tasks_processing.go:71] worker 8 working on openstack_dataplanenodesets task. I0703 18:10:42.574059 1 tasks_processing.go:71] worker 21 working on active_alerts task. I0703 18:10:42.574090 1 gather.go:177] gatherer "clusterconfig" function "container_runtime_configs" took 14.355771ms to process 0 records I0703 18:10:42.574174 1 gather.go:177] gatherer "clusterconfig" function "openshift_logging" took 14.737331ms to process 0 records I0703 18:10:42.574207 1 tasks_processing.go:71] worker 17 working on openshift_machine_api_events task. W0703 18:10:42.574141 1 gather_active_alerts.go:54] Unable to load alerts client, no alerts will be collected: open /var/run/configmaps/service-ca-bundle/service-ca.crt: no such file or directory I0703 18:10:42.574417 1 tasks_processing.go:71] worker 21 working on support_secret task. I0703 18:10:42.574823 1 gather.go:177] gatherer "clusterconfig" function "active_alerts" took 293.878µs to process 0 records I0703 18:10:42.577999 1 tasks_processing.go:71] worker 11 working on dvo_metrics task. E0703 18:10:42.578008 1 gather.go:140] gatherer "clusterconfig" function "pod_network_connectivity_checks" failed with the error: the server could not find the requested resource (get podnetworkconnectivitychecks.controlplane.operator.openshift.io) I0703 18:10:42.578307 1 gather.go:177] gatherer "clusterconfig" function "pod_network_connectivity_checks" took 18.180264ms to process 0 records E0703 18:10:42.578551 1 gather.go:140] gatherer "clusterconfig" function "machines" failed with the error: machines.machine.openshift.io is forbidden: User "system:serviceaccount:openshift-insights:gather" cannot list resource "machines" in API group "machine.openshift.io" at the cluster scope I0703 18:10:42.578643 1 gather.go:177] gatherer "clusterconfig" function "machines" took 18.634654ms to process 0 records I0703 18:10:42.578898 1 gather.go:177] gatherer "clusterconfig" function "cost_management_metrics_configs" took 19.763329ms to process 0 records I0703 18:10:42.579005 1 gather.go:177] gatherer "clusterconfig" function "jaegers" took 18.138804ms to process 0 records I0703 18:10:42.579065 1 tasks_processing.go:71] worker 18 working on monitoring_persistent_volumes task. I0703 18:10:42.578608 1 tasks_processing.go:71] worker 27 working on clusterroles task. I0703 18:10:42.578982 1 tasks_processing.go:71] worker 20 working on cluster_apiserver task. I0703 18:10:42.580450 1 tasks_processing.go:71] worker 28 working on operators_pods_and_events task. I0703 18:10:42.581104 1 recorder.go:75] Recording config/infrastructure with fingerprint=4174c9c01b8052fc5fc0c1abcf4ee506e7b7388a8a5844752ab9d7416fb648db I0703 18:10:42.581126 1 gather.go:177] gatherer "clusterconfig" function "infrastructures" took 19.390928ms to process 1 records I0703 18:10:42.581135 1 gather.go:177] gatherer "clusterconfig" function "machine_sets" took 15.673749ms to process 0 records I0703 18:10:42.581325 1 recorder.go:75] Recording config/network with fingerprint=9163cb77e7e11a3119f1cf276caf414bb23457a80105f3c8f41f402e1ab49f3f I0703 18:10:42.581337 1 tasks_processing.go:71] worker 24 working on container_images task. I0703 18:10:42.581339 1 gather.go:177] gatherer "clusterconfig" function "networks" took 19.616342ms to process 1 records I0703 18:10:42.581347 1 gather.go:177] gatherer "clusterconfig" function "sap_datahubs" took 19.381729ms to process 0 records I0703 18:10:42.581437 1 recorder.go:75] Recording config/image with fingerprint=45166357488b29323292c6061bc0953123a25ce657f53813e940bd9d1ebbdc24 I0703 18:10:42.581448 1 gather.go:177] gatherer "clusterconfig" function "image" took 20.226212ms to process 1 records I0703 18:10:42.581454 1 gather.go:177] gatherer "clusterconfig" function "node_logs" took 22.265472ms to process 0 records I0703 18:10:42.581328 1 tasks_processing.go:71] worker 0 working on validating_webhook_configurations task. I0703 18:10:42.581750 1 tasks_processing.go:71] worker 22 working on qemu_kubevirt_launcher_logs task. I0703 18:10:42.582290 1 tasks_processing.go:71] worker 10 working on machine_healthchecks task. I0703 18:10:42.582306 1 recorder.go:75] Recording config/clusteroperator/imageregistry.operator.openshift.io/config/cluster with fingerprint=4a730093875003c0e5427c84d2af579feac434f7427981d06ac60894a71f1f02 I0703 18:10:42.582321 1 gather.go:177] gatherer "clusterconfig" function "image_registries" took 22.751888ms to process 1 records I0703 18:10:42.582334 1 gather.go:177] gatherer "clusterconfig" function "sap_config" took 20.194567ms to process 0 records I0703 18:10:42.582341 1 gather.go:177] gatherer "clusterconfig" function "olm_operators" took 22.952818ms to process 0 records I0703 18:10:42.582358 1 tasks_processing.go:71] worker 31 working on metrics task. I0703 18:10:42.582379 1 tasks_processing.go:71] worker 30 working on certificate_signing_requests task. W0703 18:10:42.582401 1 gather_most_recent_metrics.go:64] Unable to load metrics client, no metrics will be collected: open /var/run/configmaps/service-ca-bundle/service-ca.crt: no such file or directory I0703 18:10:42.582413 1 tasks_processing.go:71] worker 31 working on storage_classes task. I0703 18:10:42.582418 1 tasks_processing.go:71] worker 6 working on feature_gates task. I0703 18:10:42.582749 1 gather.go:177] gatherer "clusterconfig" function "metrics" took 34.273µs to process 0 records I0703 18:10:42.582816 1 tasks_processing.go:71] worker 4 working on openstack_dataplanedeployments task. I0703 18:10:42.583316 1 tasks_processing.go:71] worker 8 working on ingress task. I0703 18:10:42.583519 1 gather.go:177] gatherer "clusterconfig" function "openstack_dataplanenodesets" took 15.368868ms to process 0 records I0703 18:10:42.583731 1 tasks_processing.go:71] worker 2 working on nodenetworkconfigurationpolicies task. I0703 18:10:42.584760 1 recorder.go:75] Recording config/mutatingwebhookconfigurations/aws-pod-identity with fingerprint=277ea400c869e71d20f4f860fc0761027093e54d857c79345592a7aef478b588 I0703 18:10:42.585014 1 recorder.go:75] Recording config/mutatingwebhookconfigurations/sre-podimagespec-mutation with fingerprint=7a9497ca6135cfea5b82984ac52aaa709c002d12156865d73c264e0d3c4dac32 I0703 18:10:42.585109 1 recorder.go:75] Recording config/mutatingwebhookconfigurations/sre-service-mutation with fingerprint=06ec4c980735df5ebb0416cb61b905e1bb4b271d3410e34a5dd2154c7907d064 I0703 18:10:42.585158 1 gather.go:177] gatherer "clusterconfig" function "mutating_webhook_configurations" took 25.798946ms to process 3 records I0703 18:10:42.585381 1 recorder.go:75] Recording config/schedulers/cluster with fingerprint=bf4557c770072f6e8693fe629139a0b8a8f4cd77a817b55361f1b58529cdf9cd I0703 18:10:42.585417 1 gather.go:177] gatherer "clusterconfig" function "schedulers" took 25.229757ms to process 1 records I0703 18:10:42.585434 1 tasks_processing.go:71] worker 15 working on openstack_version task. I0703 18:10:42.586159 1 tasks_processing.go:71] worker 29 working on image_pruners task. I0703 18:10:42.586780 1 recorder.go:75] Recording config/authentication with fingerprint=f0760eb1cfb61d365876912f71ba97190b9beedfef02fb4273b5669118a7d071 I0703 18:10:42.586812 1 gather.go:177] gatherer "clusterconfig" function "authentication" took 28.261665ms to process 1 records I0703 18:10:42.586917 1 tasks_processing.go:71] worker 14 working on oauths task. I0703 18:10:42.587915 1 recorder.go:75] Recording config/node/ip-10-0-0-112.ec2.internal with fingerprint=8bf3b35e11fbf4471072dfde2ba07f39b6ed1fd2489ebe13cb23858eaaeeb096 I0703 18:10:42.588374 1 recorder.go:75] Recording config/node/ip-10-0-1-176.ec2.internal with fingerprint=055d08b1ac751b3c095e0df93905d96b2660e389ae863b503e38a20cf2af9385 I0703 18:10:42.588473 1 recorder.go:75] Recording config/node/ip-10-0-2-132.ec2.internal with fingerprint=f9bc83b0175674c304c9228077c5ce31b69c3cb392552b3dbf157e3af4a72625 I0703 18:10:42.588485 1 gather.go:177] gatherer "clusterconfig" function "nodes" took 28.394884ms to process 3 records I0703 18:10:42.588512 1 recorder.go:75] Recording config/namespaces_with_overlapping_uids with fingerprint=4f53cda18c2baa0c0354bb5f9a3ecbe5ed12ab4d8e11ba873c2f11161202b945 I0703 18:10:42.588520 1 gather.go:177] gatherer "clusterconfig" function "overlapping_namespace_uids" took 28.583762ms to process 1 records I0703 18:10:42.588529 1 tasks_processing.go:71] worker 16 working on lokistack task. I0703 18:10:42.594402 1 tasks_processing.go:71] worker 25 working on pdbs task. I0703 18:10:42.594737 1 gather.go:177] gatherer "clusterconfig" function "openstack_controlplanes" took 26.529699ms to process 0 records I0703 18:10:42.601518 1 sca.go:136] Pulling SCA certificates from https://api.openshift.com/api/accounts_mgmt/v1/entitlement_certificates. Next check is in 8h0m0s W0703 18:10:42.601741 1 operator.go:288] started I0703 18:10:42.601520 1 cluster_transfer.go:83] checking the availability of cluster transfer. Next check is in 12h0m0s I0703 18:10:42.602009 1 base_controller.go:76] Waiting for caches to sync for LoggingSyncer I0703 18:10:42.616675 1 tasks_processing.go:71] worker 13 working on proxies task. I0703 18:10:42.616691 1 gather.go:177] gatherer "clusterconfig" function "number_of_pods_and_netnamespaces_with_sdn_annotations" took 58.971964ms to process 0 records E0703 18:10:42.616715 1 gather.go:140] gatherer "clusterconfig" function "machine_healthchecks" failed with the error: machinehealthchecks.machine.openshift.io is forbidden: User "system:serviceaccount:openshift-insights:gather" cannot list resource "machinehealthchecks" in API group "machine.openshift.io" at the cluster scope I0703 18:10:42.616722 1 gather.go:177] gatherer "clusterconfig" function "machine_healthchecks" took 34.395232ms to process 0 records I0703 18:10:42.616732 1 tasks_processing.go:71] worker 10 working on sap_pods task. I0703 18:10:42.617009 1 tasks_processing.go:71] worker 20 working on storage_cluster task. I0703 18:10:42.617285 1 recorder.go:75] Recording config/apiserver with fingerprint=a85179e8761788fc712929b2d9fc005faf73b461a3a0dedfe7826c68a90d472f I0703 18:10:42.617306 1 gather.go:177] gatherer "clusterconfig" function "cluster_apiserver" took 36.916895ms to process 1 records I0703 18:10:42.620511 1 tasks_processing.go:71] worker 2 working on config_maps task. I0703 18:10:42.620525 1 gather.go:177] gatherer "clusterconfig" function "nodenetworkconfigurationpolicies" took 36.599445ms to process 0 records I0703 18:10:42.621078 1 gather.go:177] gatherer "clusterconfig" function "openstack_version" took 35.06256ms to process 0 records I0703 18:10:42.621088 1 gather.go:177] gatherer "clusterconfig" function "lokistack" took 32.171083ms to process 0 records I0703 18:10:42.621095 1 gather.go:177] gatherer "clusterconfig" function "openshift_machine_api_events" took 46.511399ms to process 0 records I0703 18:10:42.621101 1 tasks_processing.go:74] worker 16 stopped. I0703 18:10:42.621100 1 tasks_processing.go:74] worker 15 stopped. I0703 18:10:42.621103 1 gather.go:177] gatherer "clusterconfig" function "openstack_dataplanedeployments" took 37.991893ms to process 0 records I0703 18:10:42.621115 1 tasks_processing.go:74] worker 17 stopped. I0703 18:10:42.621109 1 tasks_processing.go:74] worker 4 stopped. E0703 18:10:42.621120 1 gather.go:140] gatherer "clusterconfig" function "support_secret" failed with the error: secrets "support" not found I0703 18:10:42.621125 1 tasks_processing.go:74] worker 21 stopped. I0703 18:10:42.621159 1 gather.go:177] gatherer "clusterconfig" function "support_secret" took 46.530982ms to process 0 records I0703 18:10:42.621952 1 tasks_processing.go:74] worker 25 stopped. I0703 18:10:42.622082 1 recorder.go:75] Recording config/pdbs/openshift-image-registry/image-registry with fingerprint=edca1e4d4d73cdbe538f2548a88d38ce29ce9f75330068919b7caee8ff12d927 I0703 18:10:42.622118 1 recorder.go:75] Recording config/pdbs/openshift-ingress/router-default with fingerprint=40bf52edc37cf8f134b39a620e17ef4e4fdd4549177d2e82cd451f7526d1abb8 I0703 18:10:42.622143 1 recorder.go:75] Recording config/pdbs/openshift-operator-lifecycle-manager/packageserver-pdb with fingerprint=7607f9ed0bcd1bef44ec32d83bbb0f44de0ce58288f33aa52ee10eea29923ec2 I0703 18:10:42.622157 1 gather.go:177] gatherer "clusterconfig" function "pdbs" took 27.526531ms to process 3 records I0703 18:10:42.624968 1 shared_informer.go:320] Caches are synced for RequestHeaderAuthRequestController I0703 18:10:42.624984 1 shared_informer.go:320] Caches are synced for client-ca::kube-system::extension-apiserver-authentication::requestheader-client-ca-file I0703 18:10:42.625035 1 shared_informer.go:320] Caches are synced for client-ca::kube-system::extension-apiserver-authentication::client-ca-file I0703 18:10:42.625352 1 tasks_processing.go:74] worker 20 stopped. I0703 18:10:42.625370 1 gather.go:177] gatherer "clusterconfig" function "storage_cluster" took 8.324578ms to process 0 records I0703 18:10:42.625854 1 controller.go:212] Source scaController *sca.Controller is not ready I0703 18:10:42.625909 1 controller.go:212] Source clusterTransferController *clustertransfer.Controller is not ready I0703 18:10:42.625940 1 controller.go:212] Source periodic-clusterconfig *controllerstatus.Simple is not ready I0703 18:10:42.625963 1 controller.go:212] Source periodic-conditional *controllerstatus.Simple is not ready I0703 18:10:42.625987 1 controller.go:212] Source periodic-workloads *controllerstatus.Simple is not ready I0703 18:10:42.626031 1 controller.go:489] The operator is still being initialized I0703 18:10:42.626044 1 controller.go:512] The operator is healthy I0703 18:10:42.626074 1 recorder.go:75] Recording config/ingress with fingerprint=d20eba112d353ef71c02ae96b5de6702c66e8e744a1dafa583eb505c4f9a786a I0703 18:10:42.626145 1 gather.go:177] gatherer "clusterconfig" function "ingress" took 42.545603ms to process 1 records I0703 18:10:42.625916 1 tasks_processing.go:74] worker 8 stopped. I0703 18:10:42.626232 1 tasks_processing.go:74] worker 5 stopped. I0703 18:10:42.626731 1 recorder.go:75] Recording config/crd/volumesnapshots.snapshot.storage.k8s.io with fingerprint=de3282caa1b7f4fe3422eaedfd3b494ea36d2ad9a5fc68e9c35b964813dfe156 I0703 18:10:42.626936 1 recorder.go:75] Recording config/crd/volumesnapshotcontents.snapshot.storage.k8s.io with fingerprint=8e578ca8c8e45dc0260be06d4b668b1ddaeb9c18e9dc6adc82d639a5b9258a7e I0703 18:10:42.626944 1 gather.go:177] gatherer "clusterconfig" function "crds" took 68.057058ms to process 2 records I0703 18:10:42.629428 1 tasks_processing.go:74] worker 10 stopped. I0703 18:10:42.629447 1 gather.go:177] gatherer "clusterconfig" function "sap_pods" took 12.687592ms to process 0 records I0703 18:10:42.629773 1 prometheus_rules.go:88] Prometheus rules successfully created I0703 18:10:42.629810 1 tasks_processing.go:74] worker 29 stopped. I0703 18:10:42.629922 1 recorder.go:75] Recording config/clusteroperator/imageregistry.operator.openshift.io/imagepruner/cluster with fingerprint=157218de3044fecdda52e816d948426a2c1d911b15732b24e8a6e772d80fc45d I0703 18:10:42.629937 1 gather.go:177] gatherer "clusterconfig" function "image_pruners" took 43.624327ms to process 1 records I0703 18:10:42.629987 1 tasks_processing.go:74] worker 14 stopped. I0703 18:10:42.630309 1 recorder.go:75] Recording config/oauth with fingerprint=99204e4c97a0f01a3250d510497bce1ae1bee7f339e18c0baaca20e7cadf4613 I0703 18:10:42.630326 1 gather.go:177] gatherer "clusterconfig" function "oauths" took 43.024819ms to process 1 records I0703 18:10:42.630721 1 base_controller.go:82] Caches are synced for ConfigController I0703 18:10:42.630760 1 base_controller.go:119] Starting #1 worker of ConfigController controller ... I0703 18:10:42.631247 1 tasks_processing.go:74] worker 7 stopped. I0703 18:10:42.631641 1 recorder.go:75] Recording config/version with fingerprint=ce110d7b11a0a5f1c54f8c3e4fe58254855dc8b2ebc072f9e0e63bb77bf0dcce I0703 18:10:42.631655 1 recorder.go:75] Recording config/id with fingerprint=82d3871b46e1da89323e1b073fb6375d3e0ccb237bed5436d6330c7808da253f I0703 18:10:42.631662 1 gather.go:177] gatherer "clusterconfig" function "version" took 72.494437ms to process 2 records I0703 18:10:42.631751 1 tasks_processing.go:74] worker 0 stopped. I0703 18:10:42.631794 1 recorder.go:75] Recording config/validatingwebhookconfigurations/multus.openshift.io with fingerprint=6b8ea2196409926b703861085a8a1154974bca784227547aebd78c4c9dd090ab I0703 18:10:42.631888 1 recorder.go:75] Recording config/validatingwebhookconfigurations/network-node-identity.openshift.io with fingerprint=7c42edc9cea719097670021cd47c8fcacce9b7ec8f7b25c5ab22471580ccf5a6 I0703 18:10:42.631908 1 recorder.go:75] Recording config/validatingwebhookconfigurations/performance-addon-operator with fingerprint=a0134c3b546f2c76a8f9219d4a05919296511f1161b3516ec7dd15a1a67b56f5 I0703 18:10:42.631938 1 recorder.go:75] Recording config/validatingwebhookconfigurations/sre-clusterrolebindings-validation with fingerprint=71ff6cbb3a07fbc80078f0b37aa3574aeb29e0bc10b7a365dd262eb93fa90002 I0703 18:10:42.631967 1 recorder.go:75] Recording config/validatingwebhookconfigurations/sre-clusterroles-validation with fingerprint=305111cfece0a60d9cb02c440f60ede1b60d555f60951bf610375911a7d1ae72 I0703 18:10:42.632005 1 recorder.go:75] Recording config/validatingwebhookconfigurations/sre-ingress-config-validation with fingerprint=a73269e34ab9da4e6ce524faa6d47ba317a04e5f6609a7cefd579747a5d9cfd0 I0703 18:10:42.632035 1 recorder.go:75] Recording config/validatingwebhookconfigurations/sre-network-operator-validation with fingerprint=3997dfaee942806303708826e779e38a0c4880d0ccf5f48839cd5a1ceeb5b57b I0703 18:10:42.632074 1 recorder.go:75] Recording config/validatingwebhookconfigurations/sre-regular-user-validation with fingerprint=fc25637347daf02d7012ada786f73d154eb4875754536849bdcb8fd175469730 I0703 18:10:42.632103 1 recorder.go:75] Recording config/validatingwebhookconfigurations/sre-scc-validation with fingerprint=fc78544a7cdf38a68c82254eab9f1807928fd2114cad88bf6e5379100c65f325 I0703 18:10:42.632134 1 recorder.go:75] Recording config/validatingwebhookconfigurations/sre-serviceaccount-validation with fingerprint=a9e5beb06b999a6468163a088016981c42aa52141dc04536ad6ec66bf20c4591 I0703 18:10:42.632163 1 recorder.go:75] Recording config/validatingwebhookconfigurations/sre-techpreviewnoupgrade-validation with fingerprint=30968c7d7ff90f22e3f785513ca1362062a6daa632208fb55c24f07a870083f9 I0703 18:10:42.632170 1 gather.go:177] gatherer "clusterconfig" function "validating_webhook_configurations" took 49.788539ms to process 11 records I0703 18:10:42.633776 1 tasks_processing.go:74] worker 13 stopped. I0703 18:10:42.633866 1 recorder.go:75] Recording config/proxy with fingerprint=6b16c595fcbe774fe0f4143b9ab2698c1f2d245cf50bc2691c4fbfc42fb2330d I0703 18:10:42.633880 1 gather.go:177] gatherer "clusterconfig" function "proxies" took 17.076898ms to process 1 records I0703 18:10:42.637969 1 tasks_processing.go:74] worker 18 stopped. I0703 18:10:42.637990 1 gather.go:177] gatherer "clusterconfig" function "monitoring_persistent_volumes" took 58.889268ms to process 0 records I0703 18:10:42.638134 1 recorder.go:75] Recording config/storage/storageclasses/gp2-csi with fingerprint=a9a8f23edfecef5aeb8c9758c8068648d72867b4b0fb3d8052a6211f16110561 I0703 18:10:42.638193 1 recorder.go:75] Recording config/storage/storageclasses/gp3-csi with fingerprint=b6eebce6a238e00387472f5478e1c27c596bdcd4b304c4cbff5402a569c7d193 I0703 18:10:42.638234 1 gather.go:177] gatherer "clusterconfig" function "storage_classes" took 55.610144ms to process 2 records I0703 18:10:42.638206 1 tasks_processing.go:74] worker 31 stopped. I0703 18:10:42.638376 1 tasks_processing.go:74] worker 6 stopped. I0703 18:10:42.638494 1 recorder.go:75] Recording config/featuregate with fingerprint=50a55d405dc1b11e4e9bbbd55fad8221d3724ddb7640f4b720c5ce359a68b418 I0703 18:10:42.638508 1 gather.go:177] gatherer "clusterconfig" function "feature_gates" took 55.945774ms to process 1 records I0703 18:10:42.638522 1 gather.go:177] gatherer "clusterconfig" function "aggregated_monitoring_cr_names" took 81.198795ms to process 0 records I0703 18:10:42.638530 1 tasks_processing.go:74] worker 1 stopped. I0703 18:10:42.643248 1 tasks_processing.go:74] worker 9 stopped. I0703 18:10:42.643311 1 gather.go:177] gatherer "clusterconfig" function "machine_config_pools" took 85.813634ms to process 0 records I0703 18:10:42.645510 1 configmapobserver.go:84] configmaps "insights-config" not found I0703 18:10:42.645833 1 tasks_processing.go:74] worker 27 stopped. I0703 18:10:42.647109 1 recorder.go:75] Recording cluster-scoped-resources/rbac.authorization.k8s.io/clusterroles/admin with fingerprint=080b92e83aed6a4f50df74a651ff072a1ca3a194a431f814d40ba4a683049f67 I0703 18:10:42.647311 1 recorder.go:75] Recording cluster-scoped-resources/rbac.authorization.k8s.io/clusterroles/edit with fingerprint=0d3d22bc2112b80f5eedbabc80d7cb80d707744fa6286018a823ddd19a2502b4 I0703 18:10:42.647343 1 gather.go:177] gatherer "clusterconfig" function "clusterroles" took 66.222125ms to process 2 records I0703 18:10:42.648332 1 tasks_processing.go:74] worker 30 stopped. I0703 18:10:42.648350 1 gather.go:177] gatherer "clusterconfig" function "certificate_signing_requests" took 65.92704ms to process 0 records W0703 18:10:42.649659 1 gather_dvo_metrics.go:210] Failed to read the DVO metrics. Trying again. I0703 18:10:42.652675 1 gather_logs.go:145] no pods in namespace were found I0703 18:10:42.652697 1 tasks_processing.go:74] worker 22 stopped. I0703 18:10:42.652708 1 gather.go:177] gatherer "clusterconfig" function "qemu_kubevirt_launcher_logs" took 70.925939ms to process 0 records I0703 18:10:42.653814 1 tasks_processing.go:74] worker 24 stopped. E0703 18:10:42.655312 1 cluster_transfer.go:95] failed to pull cluster transfer: unable to retrieve cluster transfer data from https://api.openshift.com/api/accounts_mgmt/v1/cluster_transfers/: Get "https://api.openshift.com/api/accounts_mgmt/v1/cluster_transfers/?search=cluster_uuid+is+%279b3afd3f-07b8-42ee-a07c-8e343b86112f%27+and+status+is+%27accepted%27": dial tcp: lookup api.openshift.com on 172.30.0.10:53: read udp 10.129.0.12:49735->172.30.0.10:53: read: connection refused I0703 18:10:42.655329 1 controllerstatus.go:80] name=clusterTransferController healthy=true reason=Disconnected message=failed to pull cluster transfer: unable to retrieve cluster transfer data from https://api.openshift.com/api/accounts_mgmt/v1/cluster_transfers/: Get "https://api.openshift.com/api/accounts_mgmt/v1/cluster_transfers/?search=cluster_uuid+is+%279b3afd3f-07b8-42ee-a07c-8e343b86112f%27+and+status+is+%27accepted%27": dial tcp: lookup api.openshift.com on 172.30.0.10:53: read udp 10.129.0.12:49735->172.30.0.10:53: read: connection refused I0703 18:10:42.655803 1 recorder.go:75] Recording config/pod/openshift-multus/multus-gn7rl with fingerprint=7df04402d7b0af7567ea0fc30d6561393509c855a52b154989dad5e18262976a I0703 18:10:42.656235 1 recorder.go:75] Recording config/pod/openshift-ovn-kubernetes/ovnkube-node-s2hml with fingerprint=d4e7cd79a41913ac676ed4bb3c5c8d6f48ebf36e0e62e67e912ac40adcbd7abf I0703 18:10:42.656309 1 recorder.go:75] Recording config/running_containers with fingerprint=5e0bffcc1b84d9d9bbc32d770d497fca6352ccffa04ae29fc7246d56cbcdd210 I0703 18:10:42.656323 1 gather.go:177] gatherer "clusterconfig" function "container_images" took 72.459871ms to process 3 records I0703 18:10:42.658286 1 requests.go:205] Asking for SCA certificate with "{"arch": ["x86_64"]}" payload W0703 18:10:42.662100 1 sca.go:161] Failed to pull SCA certs from https://api.openshift.com/api/accounts_mgmt/v1/entitlement_certificates: unable to retrieve SCA certs data from https://api.openshift.com/api/accounts_mgmt/v1/entitlement_certificates: Post "https://api.openshift.com/api/accounts_mgmt/v1/entitlement_certificates": dial tcp: lookup api.openshift.com on 172.30.0.10:53: read udp 10.129.0.12:58317->172.30.0.10:53: read: connection refused I0703 18:10:42.662112 1 controllerstatus.go:80] name=scaController healthy=true reason=NonHTTPError message=Failed to pull SCA certs from https://api.openshift.com/api/accounts_mgmt/v1/entitlement_certificates: unable to retrieve SCA certs data from https://api.openshift.com/api/accounts_mgmt/v1/entitlement_certificates: Post "https://api.openshift.com/api/accounts_mgmt/v1/entitlement_certificates": dial tcp: lookup api.openshift.com on 172.30.0.10:53: read udp 10.129.0.12:58317->172.30.0.10:53: read: connection refused I0703 18:10:42.690355 1 tasks_processing.go:74] worker 2 stopped. E0703 18:10:42.690376 1 gather.go:140] gatherer "clusterconfig" function "config_maps" failed with the error: configmaps "cluster-monitoring-config" not found E0703 18:10:42.690384 1 gather.go:140] gatherer "clusterconfig" function "config_maps" failed with the error: configmaps "gateway-mode-config" not found E0703 18:10:42.690389 1 gather.go:140] gatherer "clusterconfig" function "config_maps" failed with the error: configmaps "insights-config" not found I0703 18:10:42.690405 1 recorder.go:75] Recording config/configmaps/openshift-config/installer-images/images.json with fingerprint=26b6661162b099a0f5a279859b4f46c867929a79d9a4a41fde4be4e6fe138018 I0703 18:10:42.690457 1 recorder.go:75] Recording config/configmaps/openshift-config/kube-root-ca.crt/ca.crt with fingerprint=f73fcd48d6f9ebbc83253a1b5f23ef59d0683465ec46e474389103c717467dfb I0703 18:10:42.690468 1 recorder.go:75] Recording config/configmaps/openshift-config/openshift-install/invoker with fingerprint=76b482f683cd3ef9da02debac5b26080a5aeb06ff768ee5c21117514dff29d8a I0703 18:10:42.690476 1 recorder.go:75] Recording config/configmaps/openshift-config/openshift-install/version with fingerprint=c93090eb0d2a4736885abeb79c91680cfd01fda46464f83456b085d4dc8239f0 I0703 18:10:42.690482 1 recorder.go:75] Recording config/configmaps/openshift-config/openshift-service-ca.crt/service-ca.crt with fingerprint=e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855 I0703 18:10:42.690545 1 recorder.go:75] Recording config/configmaps/openshift-config/rosa-brand-logo/rosa-brand-logo.svg with fingerprint=6ed8ca4dd7a8eee7249182bc006e9649ce84d76c551ddfaaa33e55d8c4cc1ed0 I0703 18:10:42.690557 1 recorder.go:75] Recording config/configmaps/kube-system/cluster-config-v1/install-config with fingerprint=ab3811c6b83fd7b8e920094cfa3080d1b4ee3c35ec4c8379437b21d27bd6608d I0703 18:10:42.690565 1 gather.go:177] gatherer "clusterconfig" function "config_maps" took 69.775898ms to process 7 records I0703 18:10:42.702714 1 base_controller.go:82] Caches are synced for LoggingSyncer I0703 18:10:42.702731 1 base_controller.go:119] Starting #1 worker of LoggingSyncer controller ... I0703 18:10:42.712726 1 gather_cluster_operators.go:184] Unable to get dnsrecords.ingress.operator.openshift.io resource due to: dnsrecords.ingress.operator.openshift.io "default" not found I0703 18:10:42.720232 1 gather_cluster_operators.go:184] Unable to get dnsrecords.ingress.operator.openshift.io resource due to: dnsrecords.ingress.operator.openshift.io "default" not found I0703 18:10:42.736309 1 tasks_processing.go:74] worker 26 stopped. I0703 18:10:42.736355 1 recorder.go:75] Recording aggregated/unused_machine_configs_count with fingerprint=4bfc9fa984e5dfcd45848faaf05269de7619bf42edf9f781751af5ee05c1a499 I0703 18:10:42.736372 1 gather.go:177] gatherer "clusterconfig" function "machine_configs" took 177.39073ms to process 1 records I0703 18:10:42.746467 1 tasks_processing.go:74] worker 23 stopped. E0703 18:10:42.746489 1 gather.go:140] gatherer "clusterconfig" function "ingress_certificates" failed with the error: failed to get secret 'router-certs-default' in namespace 'openshift-ingress': secrets "router-certs-default" not found E0703 18:10:42.746497 1 gather.go:140] gatherer "clusterconfig" function "ingress_certificates" failed with the error: failed to get secret '2raqhlbjn7f0b0l6ufd0fqt5c04iv3to-primary-cert-bundle-secret' in namespace 'openshift-ingress-operator': secrets "2raqhlbjn7f0b0l6ufd0fqt5c04iv3to-primary-cert-bundle-secret" not found I0703 18:10:42.746574 1 recorder.go:75] Recording aggregated/ingress_controllers_certs with fingerprint=4457be90bc3cb92287c48855e3343ea73c68d4c220387cbe733af4d5a4b9194a I0703 18:10:42.746594 1 gather.go:177] gatherer "clusterconfig" function "ingress_certificates" took 188.507945ms to process 1 records I0703 18:10:43.039815 1 gather_cluster_operator_pods_and_events.go:121] Found 18 pods with 21 containers I0703 18:10:43.039833 1 gather_cluster_operator_pods_and_events.go:235] Maximum buffer size: 1198372 bytes I0703 18:10:43.040129 1 gather_cluster_operator_pods_and_events.go:365] Fetching logs for dns container dns-default-9zvth pod in namespace openshift-dns (previous: false). I0703 18:10:43.280155 1 gather_cluster_operator_pods_and_events.go:408] Failed to fetch log for dns-default-9zvth pod in namespace openshift-dns for failing operator dns (previous: false): "container \"dns\" in pod \"dns-default-9zvth\" is waiting to start: ContainerCreating" I0703 18:10:43.280176 1 gather_cluster_operator_pods_and_events.go:280] Error: "container \"dns\" in pod \"dns-default-9zvth\" is waiting to start: ContainerCreating" I0703 18:10:43.280190 1 gather_cluster_operator_pods_and_events.go:365] Fetching logs for kube-rbac-proxy container dns-default-9zvth pod in namespace openshift-dns (previous: false). I0703 18:10:43.446878 1 gather_cluster_operator_pods_and_events.go:408] Failed to fetch log for dns-default-9zvth pod in namespace openshift-dns for failing operator kube-rbac-proxy (previous: false): "container \"kube-rbac-proxy\" in pod \"dns-default-9zvth\" is waiting to start: ContainerCreating" I0703 18:10:43.446899 1 gather_cluster_operator_pods_and_events.go:280] Error: "container \"kube-rbac-proxy\" in pod \"dns-default-9zvth\" is waiting to start: ContainerCreating" I0703 18:10:43.446910 1 gather_cluster_operator_pods_and_events.go:365] Fetching logs for dns container dns-default-cgvs6 pod in namespace openshift-dns (previous: false). W0703 18:10:43.642409 1 gather_dvo_metrics.go:210] Failed to read the DVO metrics. Trying again. I0703 18:10:43.681401 1 gather_cluster_operator_pods_and_events.go:408] Failed to fetch log for dns-default-cgvs6 pod in namespace openshift-dns for failing operator dns (previous: false): "container \"dns\" in pod \"dns-default-cgvs6\" is waiting to start: ContainerCreating" I0703 18:10:43.681421 1 gather_cluster_operator_pods_and_events.go:280] Error: "container \"dns\" in pod \"dns-default-cgvs6\" is waiting to start: ContainerCreating" I0703 18:10:43.681430 1 gather_cluster_operator_pods_and_events.go:365] Fetching logs for kube-rbac-proxy container dns-default-cgvs6 pod in namespace openshift-dns (previous: false). I0703 18:10:43.863307 1 gather_cluster_operator_pods_and_events.go:408] Failed to fetch log for dns-default-cgvs6 pod in namespace openshift-dns for failing operator kube-rbac-proxy (previous: false): "container \"kube-rbac-proxy\" in pod \"dns-default-cgvs6\" is waiting to start: ContainerCreating" I0703 18:10:43.863328 1 gather_cluster_operator_pods_and_events.go:280] Error: "container \"kube-rbac-proxy\" in pod \"dns-default-cgvs6\" is waiting to start: ContainerCreating" I0703 18:10:43.863338 1 gather_cluster_operator_pods_and_events.go:365] Fetching logs for dns container dns-default-dxfbn pod in namespace openshift-dns (previous: false). I0703 18:10:44.055347 1 gather_cluster_operators.go:184] Unable to get configs.samples.operator.openshift.io resource due to: configs.samples.operator.openshift.io "cluster" not found I0703 18:10:44.083784 1 gather_cluster_operator_pods_and_events.go:408] Failed to fetch log for dns-default-dxfbn pod in namespace openshift-dns for failing operator dns (previous: false): "container \"dns\" in pod \"dns-default-dxfbn\" is waiting to start: ContainerCreating" I0703 18:10:44.083801 1 gather_cluster_operator_pods_and_events.go:280] Error: "container \"dns\" in pod \"dns-default-dxfbn\" is waiting to start: ContainerCreating" I0703 18:10:44.083809 1 gather_cluster_operator_pods_and_events.go:365] Fetching logs for kube-rbac-proxy container dns-default-dxfbn pod in namespace openshift-dns (previous: false). I0703 18:10:44.249380 1 gather_cluster_operator_pods_and_events.go:408] Failed to fetch log for dns-default-dxfbn pod in namespace openshift-dns for failing operator kube-rbac-proxy (previous: false): "container \"kube-rbac-proxy\" in pod \"dns-default-dxfbn\" is waiting to start: ContainerCreating" I0703 18:10:44.249398 1 gather_cluster_operator_pods_and_events.go:280] Error: "container \"kube-rbac-proxy\" in pod \"dns-default-dxfbn\" is waiting to start: ContainerCreating" I0703 18:10:44.249408 1 gather_cluster_operator_pods_and_events.go:365] Fetching logs for dns-node-resolver container node-resolver-cgckm pod in namespace openshift-dns (previous: false). I0703 18:10:44.449755 1 gather_cluster_operator_pods_and_events.go:280] Error: "log buffer is empty" I0703 18:10:44.449775 1 gather_cluster_operator_pods_and_events.go:365] Fetching logs for dns-node-resolver container node-resolver-fp6fj pod in namespace openshift-dns (previous: false). I0703 18:10:44.454045 1 tasks_processing.go:74] worker 3 stopped. I0703 18:10:44.454090 1 recorder.go:75] Recording config/clusteroperator/console with fingerprint=b63ddb178d5ab43babb1528259c8d4b6f63d1f3b0fba3d043954ce019ae61cda I0703 18:10:44.454122 1 recorder.go:75] Recording config/clusteroperator/csi-snapshot-controller with fingerprint=de1f41f985efbd6147b92fc588b48809f5ab69f76713655bf85bad709704ea90 I0703 18:10:44.454149 1 recorder.go:75] Recording config/clusteroperator/operator.openshift.io/csisnapshotcontroller/cluster with fingerprint=5adc514f4b63e2f1ecc68bf6f9c0af70c5eea04522a49524e102721b1c41f80e I0703 18:10:44.454174 1 recorder.go:75] Recording config/clusteroperator/dns with fingerprint=28c7149f96d85cdf3731cbcafe20ca27bb04f758e6c7a49bf33616078931f489 I0703 18:10:44.454209 1 recorder.go:75] Recording config/clusteroperator/operator.openshift.io/dns/default with fingerprint=9e7b4ce029030d3d8c3b49af92c556acdcc415000b40d3f969dbdc42c432b47f I0703 18:10:44.454234 1 recorder.go:75] Recording config/clusteroperator/image-registry with fingerprint=015ff4f287b89658f29d45ea353ba8f890d572ff2f88860f545694209459eb06 I0703 18:10:44.454265 1 recorder.go:75] Recording config/clusteroperator/ingress with fingerprint=633648a11c1a4ed8a6457304ad3c1dc60afeb4304b04df83dfa23917d28b190c I0703 18:10:44.454289 1 recorder.go:75] Recording config/clusteroperator/operator.openshift.io/ingresscontroller/openshift-ingress-operator/default with fingerprint=602b9adf8007cf5613847e9f3711b99ba958d226daab1dc33fd846c8541ff64c I0703 18:10:44.454305 1 recorder.go:75] Recording config/clusteroperator/insights with fingerprint=4fb40966432bff1a7bc99426ffc0b1237f79dde1d22283e70889cbd15a58d8ff I0703 18:10:44.454323 1 recorder.go:75] Recording config/clusteroperator/kube-apiserver with fingerprint=cb93d408336cbc294bf833c4cbc18b25db9bc8de159bdbc48f832c2da88655bb I0703 18:10:44.454334 1 recorder.go:75] Recording config/clusteroperator/operator.openshift.io/kubeapiserver/cluster with fingerprint=51503bf0b784fcf65ea46bcaf1f72ac1a5c4d5dc211934f18f27871efed05762 I0703 18:10:44.454350 1 recorder.go:75] Recording config/clusteroperator/kube-controller-manager with fingerprint=e9a73307090f0790d9927d307a02d42813cecc7913d50b69ba5e685d44394d6a I0703 18:10:44.454360 1 recorder.go:75] Recording config/clusteroperator/operator.openshift.io/kubecontrollermanager/cluster with fingerprint=ce90c0d4f367d7da085074268031798382ae7c54fdcb0a21f15a4818fe308c11 I0703 18:10:44.454377 1 recorder.go:75] Recording config/clusteroperator/kube-scheduler with fingerprint=f44b1c000a50a713a17799f4c3fd2282848727ff7ff58e5db3a3ab6a9e21ff1f I0703 18:10:44.454387 1 recorder.go:75] Recording config/clusteroperator/operator.openshift.io/kubescheduler/cluster with fingerprint=f2940fb9fd20c19951dfc295eb363b7fba0c505f5ae61f01967a063099e6b60a I0703 18:10:44.454400 1 recorder.go:75] Recording config/clusteroperator/kube-storage-version-migrator with fingerprint=78783d755f8dfa5bee9a6b1748b909c5005c8b365ad7870fda092b5d676a5bd5 I0703 18:10:44.454409 1 recorder.go:75] Recording config/clusteroperator/operator.openshift.io/kubestorageversionmigrator/cluster with fingerprint=9351181aa7e6ada41ef581ab31e13516c6b934cc95710154bafb2eb222cb58db I0703 18:10:44.454424 1 recorder.go:75] Recording config/clusteroperator/monitoring with fingerprint=b21ecaefd3b9ac5f971001ccbd8d81b6d9581fe7933ee477ba37ef44ace8af38 I0703 18:10:44.454530 1 recorder.go:75] Recording config/clusteroperator/network with fingerprint=27f787d4208c3a6d6072bf0c738b46b584df2b5fe5a57a1158da331ff59c9777 I0703 18:10:44.454540 1 recorder.go:75] Recording config/clusteroperator/network.operator.openshift.io/operatorpki/openshift-ovn-kubernetes/ovn with fingerprint=626a89d20e0deaed5b6dfb533acfe65f4bb1618bd200a703b62e60c5d16d94ab I0703 18:10:44.454547 1 recorder.go:75] Recording config/clusteroperator/network.operator.openshift.io/operatorpki/openshift-ovn-kubernetes/signer with fingerprint=90410b16914712b85b3c4578716ad8c0ae072e688f4cd1e022bf76f20da3506d I0703 18:10:44.454567 1 recorder.go:75] Recording config/clusteroperator/operator.openshift.io/network/cluster with fingerprint=a93d15eaecb455a0e40ecb2826eeecc1533899204ddd3c3921d15ab70af7ae75 I0703 18:10:44.454588 1 recorder.go:75] Recording config/clusteroperator/node-tuning with fingerprint=943ff82ba904181c4ef1a40fc46c792273d662d9d2e9f6e1cb5881f893188f2f I0703 18:10:44.454625 1 recorder.go:75] Recording config/clusteroperator/openshift-apiserver with fingerprint=009bb5ca00542e6e41ce7c9c80acc95fcccb9efcdba467cf176584199d3cc217 I0703 18:10:44.454635 1 recorder.go:75] Recording config/clusteroperator/operator.openshift.io/openshiftapiserver/cluster with fingerprint=e712e6cf27339b441e4ed1f4cde91dbde7e952698ba93407e4457db63a4a4c76 I0703 18:10:44.454652 1 recorder.go:75] Recording config/clusteroperator/openshift-controller-manager with fingerprint=220412d81d15d30d40121b44ef48443925ae4c8853eb0bd72900892f7c79bed5 I0703 18:10:44.454662 1 recorder.go:75] Recording config/clusteroperator/operator.openshift.io/openshiftcontrollermanager/cluster with fingerprint=d71a0f4672f9b45d9fc8293bf1687afc650fd28d32e2e30de27523fe7b4eadf7 I0703 18:10:44.454674 1 recorder.go:75] Recording config/clusteroperator/openshift-samples with fingerprint=107434246c34e4388766f15dd2281c4326fe0f17aefc5aa563ac8a00fe151166 I0703 18:10:44.454690 1 recorder.go:75] Recording config/clusteroperator/operator-lifecycle-manager with fingerprint=7a1551b34cda5ad8378ed51bcd4fc2ddc8b7c4f1785f3bb7f89c99f961114ce2 I0703 18:10:44.454706 1 recorder.go:75] Recording config/clusteroperator/operator-lifecycle-manager-catalog with fingerprint=50c9eed99d681d2948b42e9fac064a0fe6db2e785cfb8c369a0fb0ca9cdecde3 I0703 18:10:44.454722 1 recorder.go:75] Recording config/clusteroperator/operator-lifecycle-manager-packageserver with fingerprint=daffb92725acf8c3ee070d3dc87eb6f62dbc972e55d6ce31aefb00acf5554ddf I0703 18:10:44.454735 1 recorder.go:75] Recording config/clusteroperator/service-ca with fingerprint=3646e28ce3a9e8b6ea84f7353331c7bfb5b5ee2dd4c778bb0e965e12e2a45b90 I0703 18:10:44.454760 1 recorder.go:75] Recording config/clusteroperator/storage with fingerprint=7a63ab68b21a4ea3de5ae111c877460f128afd1554c5b94cba7704e63adbdcd5 I0703 18:10:44.454776 1 recorder.go:75] Recording config/clusteroperator/operator.openshift.io/clustercsidriver/ebs.csi.aws.com with fingerprint=510064d6f6bcced87ab5bd2ddaff3d0edd7f93f4a4f7af2641f29fc53ffab21e I0703 18:10:44.454782 1 recorder.go:75] Recording config/clusteroperator/operator.openshift.io/storage/cluster with fingerprint=8e480f8c1ce1b39baac42d8ec780c57c2592929ae0c801b61ffad49ba13f33ad I0703 18:10:44.454789 1 gather.go:177] gatherer "clusterconfig" function "operators" took 1.896893957s to process 35 records W0703 18:10:44.642588 1 gather_dvo_metrics.go:210] Failed to read the DVO metrics. Trying again. I0703 18:10:44.647482 1 gather_cluster_operator_pods_and_events.go:280] Error: "log buffer is empty" I0703 18:10:44.647500 1 gather_cluster_operator_pods_and_events.go:365] Fetching logs for dns-node-resolver container node-resolver-n92bg pod in namespace openshift-dns (previous: false). I0703 18:10:44.848911 1 gather_cluster_operator_pods_and_events.go:280] Error: "log buffer is empty" I0703 18:10:44.848930 1 gather_cluster_operator_pods_and_events.go:365] Fetching logs for registry container image-registry-599dbd5dfd-5nkh2 pod in namespace openshift-image-registry (previous: false). I0703 18:10:45.048180 1 gather_cluster_operator_pods_and_events.go:408] Failed to fetch log for image-registry-599dbd5dfd-5nkh2 pod in namespace openshift-image-registry for failing operator registry (previous: false): "container \"registry\" in pod \"image-registry-599dbd5dfd-5nkh2\" is waiting to start: ContainerCreating" I0703 18:10:45.048198 1 gather_cluster_operator_pods_and_events.go:280] Error: "container \"registry\" in pod \"image-registry-599dbd5dfd-5nkh2\" is waiting to start: ContainerCreating" I0703 18:10:45.048208 1 gather_cluster_operator_pods_and_events.go:365] Fetching logs for registry container image-registry-74df9bb549-gnd2m pod in namespace openshift-image-registry (previous: false). I0703 18:10:45.248923 1 gather_cluster_operator_pods_and_events.go:408] Failed to fetch log for image-registry-74df9bb549-gnd2m pod in namespace openshift-image-registry for failing operator registry (previous: false): "container \"registry\" in pod \"image-registry-74df9bb549-gnd2m\" is waiting to start: ContainerCreating" I0703 18:10:45.248942 1 gather_cluster_operator_pods_and_events.go:280] Error: "container \"registry\" in pod \"image-registry-74df9bb549-gnd2m\" is waiting to start: ContainerCreating" I0703 18:10:45.248953 1 gather_cluster_operator_pods_and_events.go:365] Fetching logs for registry container image-registry-74df9bb549-pvrs9 pod in namespace openshift-image-registry (previous: false). I0703 18:10:45.447712 1 gather_cluster_operator_pods_and_events.go:408] Failed to fetch log for image-registry-74df9bb549-pvrs9 pod in namespace openshift-image-registry for failing operator registry (previous: false): "container \"registry\" in pod \"image-registry-74df9bb549-pvrs9\" is waiting to start: ContainerCreating" I0703 18:10:45.447733 1 gather_cluster_operator_pods_and_events.go:280] Error: "container \"registry\" in pod \"image-registry-74df9bb549-pvrs9\" is waiting to start: ContainerCreating" I0703 18:10:45.447743 1 gather_cluster_operator_pods_and_events.go:365] Fetching logs for node-ca container node-ca-94wpn pod in namespace openshift-image-registry (previous: false). W0703 18:10:45.643705 1 gather_dvo_metrics.go:210] Failed to read the DVO metrics. Trying again. I0703 18:10:45.658936 1 gather_cluster_operator_pods_and_events.go:280] Error: "log buffer is empty" I0703 18:10:45.658955 1 gather_cluster_operator_pods_and_events.go:365] Fetching logs for node-ca container node-ca-h8wj9 pod in namespace openshift-image-registry (previous: false). I0703 18:10:45.847877 1 gather_cluster_operator_pods_and_events.go:280] Error: "log buffer is empty" I0703 18:10:45.847896 1 gather_cluster_operator_pods_and_events.go:365] Fetching logs for node-ca container node-ca-qvctj pod in namespace openshift-image-registry (previous: false). I0703 18:10:46.048161 1 gather_cluster_operator_pods_and_events.go:280] Error: "log buffer is empty" I0703 18:10:46.048183 1 gather_cluster_operator_pods_and_events.go:365] Fetching logs for router container router-default-59c5c86b87-98ptl pod in namespace openshift-ingress (previous: false). I0703 18:10:46.248919 1 gather_cluster_operator_pods_and_events.go:408] Failed to fetch log for router-default-59c5c86b87-98ptl pod in namespace openshift-ingress for failing operator router (previous: false): "container \"router\" in pod \"router-default-59c5c86b87-98ptl\" is waiting to start: ContainerCreating" I0703 18:10:46.248937 1 gather_cluster_operator_pods_and_events.go:280] Error: "container \"router\" in pod \"router-default-59c5c86b87-98ptl\" is waiting to start: ContainerCreating" I0703 18:10:46.248948 1 gather_cluster_operator_pods_and_events.go:365] Fetching logs for router container router-default-59c5c86b87-zjmpx pod in namespace openshift-ingress (previous: false). I0703 18:10:46.447599 1 gather_cluster_operator_pods_and_events.go:408] Failed to fetch log for router-default-59c5c86b87-zjmpx pod in namespace openshift-ingress for failing operator router (previous: false): "container \"router\" in pod \"router-default-59c5c86b87-zjmpx\" is waiting to start: ContainerCreating" I0703 18:10:46.447643 1 gather_cluster_operator_pods_and_events.go:280] Error: "container \"router\" in pod \"router-default-59c5c86b87-zjmpx\" is waiting to start: ContainerCreating" I0703 18:10:46.447659 1 gather_cluster_operator_pods_and_events.go:365] Fetching logs for router container router-default-97dcf4df9-w49vd pod in namespace openshift-ingress (previous: false). W0703 18:10:46.641932 1 gather_dvo_metrics.go:210] Failed to read the DVO metrics. Trying again. I0703 18:10:46.647702 1 gather_cluster_operator_pods_and_events.go:408] Failed to fetch log for router-default-97dcf4df9-w49vd pod in namespace openshift-ingress for failing operator router (previous: false): "container \"router\" in pod \"router-default-97dcf4df9-w49vd\" is waiting to start: ContainerCreating" I0703 18:10:46.647723 1 gather_cluster_operator_pods_and_events.go:280] Error: "container \"router\" in pod \"router-default-97dcf4df9-w49vd\" is waiting to start: ContainerCreating" I0703 18:10:46.647737 1 gather_cluster_operator_pods_and_events.go:365] Fetching logs for serve-healthcheck-canary container ingress-canary-5z6lh pod in namespace openshift-ingress-canary (previous: false). I0703 18:10:46.848278 1 gather_cluster_operator_pods_and_events.go:408] Failed to fetch log for ingress-canary-5z6lh pod in namespace openshift-ingress-canary for failing operator serve-healthcheck-canary (previous: false): "container \"serve-healthcheck-canary\" in pod \"ingress-canary-5z6lh\" is waiting to start: ContainerCreating" I0703 18:10:46.848307 1 gather_cluster_operator_pods_and_events.go:280] Error: "container \"serve-healthcheck-canary\" in pod \"ingress-canary-5z6lh\" is waiting to start: ContainerCreating" I0703 18:10:46.848318 1 gather_cluster_operator_pods_and_events.go:365] Fetching logs for serve-healthcheck-canary container ingress-canary-jhmfz pod in namespace openshift-ingress-canary (previous: false). I0703 18:10:47.049131 1 gather_cluster_operator_pods_and_events.go:408] Failed to fetch log for ingress-canary-jhmfz pod in namespace openshift-ingress-canary for failing operator serve-healthcheck-canary (previous: false): "container \"serve-healthcheck-canary\" in pod \"ingress-canary-jhmfz\" is waiting to start: ContainerCreating" I0703 18:10:47.049152 1 gather_cluster_operator_pods_and_events.go:280] Error: "container \"serve-healthcheck-canary\" in pod \"ingress-canary-jhmfz\" is waiting to start: ContainerCreating" I0703 18:10:47.049163 1 gather_cluster_operator_pods_and_events.go:365] Fetching logs for serve-healthcheck-canary container ingress-canary-t8r7r pod in namespace openshift-ingress-canary (previous: false). I0703 18:10:47.247601 1 gather_cluster_operator_pods_and_events.go:408] Failed to fetch log for ingress-canary-t8r7r pod in namespace openshift-ingress-canary for failing operator serve-healthcheck-canary (previous: false): "container \"serve-healthcheck-canary\" in pod \"ingress-canary-t8r7r\" is waiting to start: ContainerCreating" I0703 18:10:47.247641 1 gather_cluster_operator_pods_and_events.go:280] Error: "container \"serve-healthcheck-canary\" in pod \"ingress-canary-t8r7r\" is waiting to start: ContainerCreating" I0703 18:10:47.247660 1 tasks_processing.go:74] worker 28 stopped. I0703 18:10:47.247757 1 recorder.go:75] Recording events/openshift-dns-operator with fingerprint=fcd9869e01a19c66809d95a5958f90937bf4667513bd8f70a65fd1426ed8a255 I0703 18:10:47.247818 1 recorder.go:75] Recording events/openshift-dns with fingerprint=a5dfceac8df9eb2608a54bbc8d7744eabe3f67589f8c87e99b14f4190d18d942 I0703 18:10:47.247958 1 recorder.go:75] Recording events/openshift-image-registry with fingerprint=a2a1717c601e29bf3b842542efde810f02aad32450b1c159b54cca61e12b0318 I0703 18:10:47.248014 1 recorder.go:75] Recording events/openshift-ingress-operator with fingerprint=983b8485b9e5cf5fb1e0207b8e518036aeb6405bc5b36deb3388ff4409c8c537 I0703 18:10:47.248084 1 recorder.go:75] Recording events/openshift-ingress with fingerprint=c6343a31bfaf2875144ca1593bb8dcb7f8653e0229050769f32f2f68742df5b7 I0703 18:10:47.248116 1 recorder.go:75] Recording events/openshift-ingress-canary with fingerprint=d0d90b959e8258088e04e3715b1899a81b7c6f2f3a4f56ad60c20e25185708ef I0703 18:10:47.248129 1 gather.go:177] gatherer "clusterconfig" function "operators_pods_and_events" took 4.667155054s to process 6 records W0703 18:10:47.643227 1 gather_dvo_metrics.go:210] Failed to read the DVO metrics. Trying again. W0703 18:10:47.643260 1 gather_dvo_metrics.go:117] Unable to read metrics from endpoint "http://deployment-validation-operator-metrics.openshift-deployment-validation-operator.svc:8383": DVO metrics service was not available within the 5s timeout: context deadline exceeded I0703 18:10:47.643281 1 tasks_processing.go:74] worker 11 stopped. E0703 18:10:47.643294 1 gather.go:140] gatherer "clusterconfig" function "dvo_metrics" failed with the error: DVO metrics service was not available within the 5s timeout: context deadline exceeded I0703 18:10:47.643308 1 recorder.go:75] Recording config/dvo_metrics with fingerprint=e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855 W0703 18:10:47.643328 1 gather.go:155] issue recording gatherer "clusterconfig" function "dvo_metrics" result "config/dvo_metrics" because of the warning: warning: the record with the same fingerprint "e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855" was already recorded at path "config/configmaps/openshift-config/openshift-service-ca.crt/service-ca.crt", recording another one with a different path "config/dvo_metrics" I0703 18:10:47.643345 1 gather.go:177] gatherer "clusterconfig" function "dvo_metrics" took 5.065259278s to process 1 records I0703 18:10:55.196537 1 tasks_processing.go:74] worker 19 stopped. I0703 18:10:55.196582 1 recorder.go:75] Recording config/installplans with fingerprint=7b887df561a3a9e6ef0dc672845aa5d56e348505006b7496d3a2f83892b0c95b I0703 18:10:55.196597 1 gather.go:177] gatherer "clusterconfig" function "install_plans" took 12.637731477s to process 1 records I0703 18:10:55.791082 1 configmapobserver.go:84] configmaps "insights-config" not found I0703 18:10:55.968753 1 tasks_processing.go:74] worker 12 stopped. I0703 18:10:55.969045 1 recorder.go:75] Recording config/serviceaccounts with fingerprint=6eea9bca9c2d016e4128bf2b374fefdd9610f4d08552d21add7f6bb399a3af40 I0703 18:10:55.969067 1 gather.go:177] gatherer "clusterconfig" function "service_accounts" took 13.40985201s to process 1 records E0703 18:10:55.969148 1 periodic.go:247] "Unhandled Error" err="clusterconfig failed after 13.412s with: function \"pod_network_connectivity_checks\" failed with an error, function \"machines\" failed with an error, function \"machine_healthchecks\" failed with an error, function \"support_secret\" failed with an error, function \"config_maps\" failed with an error, function \"ingress_certificates\" failed with an error, function \"dvo_metrics\" failed with an error" I0703 18:10:55.970261 1 controllerstatus.go:89] name=periodic-clusterconfig healthy=false reason=PeriodicGatherFailed message=Source clusterconfig could not be retrieved: function "pod_network_connectivity_checks" failed with an error, function "machines" failed with an error, function "machine_healthchecks" failed with an error, function "support_secret" failed with an error, function "config_maps" failed with an error, function "ingress_certificates" failed with an error, function "dvo_metrics" failed with an error I0703 18:10:55.970276 1 periodic.go:209] Running workloads gatherer I0703 18:10:55.970291 1 tasks_processing.go:45] number of workers: 2 I0703 18:10:55.970297 1 tasks_processing.go:69] worker 1 listening for tasks. I0703 18:10:55.970301 1 tasks_processing.go:71] worker 1 working on workload_info task. I0703 18:10:55.970316 1 tasks_processing.go:69] worker 0 listening for tasks. I0703 18:10:55.970331 1 tasks_processing.go:71] worker 0 working on helmchart_info task. I0703 18:10:56.002248 1 gather_workloads_info.go:278] Loaded pods in 0s, will wait 22s for image data I0703 18:10:56.010795 1 tasks_processing.go:74] worker 0 stopped. I0703 18:10:56.010812 1 gather.go:177] gatherer "workloads" function "helmchart_info" took 40.450055ms to process 0 records I0703 18:10:56.013296 1 gather_workloads_info.go:387] No image sha256:2904a78e2eb73fd6a9bb94c105c2a056831fb4113fbb7b0607c50adc9d879c9b (12ms) I0703 18:10:56.024807 1 gather_workloads_info.go:387] No image sha256:80748ba08e1c264a8c105e7f607eff386a66378e024443a844993ee9292858c1 (11ms) I0703 18:10:56.035885 1 gather_workloads_info.go:387] No image sha256:04c87c054a3f366a7dfbe0a93ebb0c80a098ee16842c5794b67c1202eec61996 (11ms) I0703 18:10:56.046880 1 gather_workloads_info.go:387] No image sha256:765f0d23b637f685f98a31bd47c131b03cf72a40761a3f9a9d6320faa3c33733 (11ms) I0703 18:10:56.063560 1 gather_workloads_info.go:387] No image sha256:2e564f336c77116053f34d4201d364d8da04e789cfffa0ea422574c95f2d6404 (17ms) I0703 18:10:56.074184 1 gather_workloads_info.go:387] No image sha256:47154813651033d59751fb655a384dbffb64dd26f10bd7f3be0c3128d0486356 (11ms) I0703 18:10:56.085102 1 gather_workloads_info.go:387] No image sha256:56a85660a445eced5c79a595a0eccf590087c5672d50f49d4c25ad52f9a44f04 (11ms) I0703 18:10:56.095729 1 gather_workloads_info.go:387] No image sha256:e84cb128d930bd1ab867cc89b7b7bf2b2c0e41105ab93b5381069945b3ee9c57 (11ms) I0703 18:10:56.106399 1 gather_workloads_info.go:387] No image sha256:a258c226562adb14e3a163a1940938526ee6a0928982a7667d85d9a7334ce639 (11ms) I0703 18:10:56.118992 1 gather_workloads_info.go:387] No image sha256:521712486e2c6e3c020dad6a1cb340db8e55665b69f7c208fab9cd9e965fd588 (13ms) I0703 18:10:56.130268 1 gather_workloads_info.go:387] No image sha256:2e57e192c3c1240fd935dcd55c8fde5e70e78bf81d6176c96edf21fafe59f8ba (11ms) I0703 18:10:56.213139 1 gather_workloads_info.go:387] No image sha256:1a2532940843248c57d52141185dd71fbc393ab28b65d48f682038632c1dbbad (83ms) I0703 18:10:56.315786 1 gather_workloads_info.go:387] No image sha256:695cf2f0cc07683c2a3ce1eaf3e56fe18abc6e2bac716f7d9843f5d173b9df52 (103ms) I0703 18:10:56.413392 1 gather_workloads_info.go:387] No image sha256:5a95c19d82767e0235b4edb4a0536482c816904897aae1dc3eb255cb52b87a9f (98ms) I0703 18:10:56.513365 1 gather_workloads_info.go:387] No image sha256:4556896f77307821531ef91b7b7faccb82b824ea695693b2989f597f0deca038 (100ms) I0703 18:10:56.613447 1 gather_workloads_info.go:387] No image sha256:a0105d1eb62cf6ac9e5e2ef28d3e89bf6dc514bc594fc7090fe5a5ee18a09c87 (100ms) I0703 18:10:56.669642 1 configmapobserver.go:84] configmaps "insights-config" not found I0703 18:10:56.713053 1 gather_workloads_info.go:387] No image sha256:2598489729a4b258e4ecda4a06f6875133f2a10ced5c5241f8a57a8a05418e36 (100ms) I0703 18:10:56.814056 1 gather_workloads_info.go:387] No image sha256:a56211d075aa43cbb491f669a5b2e46ee023dc95b7d51dbac28f463948c5ad61 (101ms) I0703 18:10:56.866407 1 configmapobserver.go:84] configmaps "insights-config" not found I0703 18:10:56.915556 1 gather_workloads_info.go:387] No image sha256:a498046d64605bcccee2440aa4f04a4602baaae263cf01d977ec5208e876b1fd (101ms) I0703 18:10:57.013577 1 gather_workloads_info.go:387] No image sha256:943018739e3db1763c3184b460dbc409e058abbac76d57b9927faad317be85e4 (98ms) I0703 18:10:57.113437 1 gather_workloads_info.go:387] No image sha256:95418d0ba6fc0aa96647d97cf6530967f7d7ae97a82ce7f7e41c8e44310b0b72 (100ms) I0703 18:10:57.213484 1 gather_workloads_info.go:387] No image sha256:b3909bf664c77097f75b3768830863d642eed3815dab2bfb4415c771ca2d5007 (100ms) I0703 18:10:57.313170 1 gather_workloads_info.go:387] No image sha256:7b31223098f08328f5ddea8e5b871dbbd5f5a61ec550e8956f66793c0c6031a9 (100ms) I0703 18:10:57.414367 1 gather_workloads_info.go:387] No image sha256:5f0b67cfbbc381243fb91ccc17345b56d05f4d717c667e8c644e5bf05633ba71 (101ms) I0703 18:10:57.513538 1 gather_workloads_info.go:387] No image sha256:c940ea87e7d133d75ba0002ef00c0806825eed3db8094cdb260d1bac18127733 (99ms) I0703 18:10:57.613847 1 gather_workloads_info.go:387] No image sha256:36b9e89c3cfcf1ab9ae500486e38afb6862cba48cb0b4d84a09508ab8f3d299f (100ms) I0703 18:10:57.714049 1 gather_workloads_info.go:387] No image sha256:0a99240166165eb5718e7516a43282fe32df9c7c5e809b31b58abe44e42ff94d (100ms) I0703 18:10:57.813850 1 gather_workloads_info.go:387] No image sha256:03cf4cd7ef1518610c6c7b3ad27d1622d82e98e3dc6e3f8e5d0fceb5c8d3786e (100ms) I0703 18:10:57.914265 1 gather_workloads_info.go:387] No image sha256:289816958633a763a72dbc44e1dad40466223164e7e253039514f0d974ea5d21 (100ms) I0703 18:10:58.014494 1 gather_workloads_info.go:387] No image sha256:91828234f107c068c8a4966d08370ae7b73e637651dbc6d92c18c4553402c22c (100ms) I0703 18:10:58.114543 1 gather_workloads_info.go:387] No image sha256:653c666f842c13e0baae2e89a9b1efe0e2ef56f621ffb5b32005115d2a26ab8c (100ms) I0703 18:10:58.214396 1 gather_workloads_info.go:387] No image sha256:ca1344cb64140188b7cae7bbc51fb751566c0b0c97d5e39b5850e628032c4a5e (100ms) I0703 18:10:58.316494 1 gather_workloads_info.go:387] No image sha256:7adc1eab05d6724c76ba751f6df816b08d6e70b78dee9eb94fa6fd9690542c98 (102ms) I0703 18:10:58.316536 1 tasks_processing.go:74] worker 1 stopped. E0703 18:10:58.316547 1 gather.go:140] gatherer "workloads" function "workload_info" failed with the error: no running pods found for the insights-runtime-extractor statefulset I0703 18:10:58.316882 1 recorder.go:75] Recording config/workload_info with fingerprint=f89d24a4ea1e4e5422ea0e910606624df3f108d73fbbc57cba713d84b897d594 I0703 18:10:58.316899 1 gather.go:177] gatherer "workloads" function "workload_info" took 2.346225359s to process 1 records E0703 18:10:58.316925 1 periodic.go:247] "Unhandled Error" err="workloads failed after 2.346s with: function \"workload_info\" failed with an error" I0703 18:10:58.318031 1 controllerstatus.go:89] name=periodic-workloads healthy=false reason=PeriodicGatherFailed message=Source workloads could not be retrieved: function "workload_info" failed with an error I0703 18:10:58.318041 1 periodic.go:209] Running conditional gatherer I0703 18:10:58.326647 1 requests.go:294] Making HTTP GET request at: https://console.redhat.com/api/gathering/v2/4.20.8/gathering_rules I0703 18:10:58.333092 1 conditional_gatherer.go:107] Get "https://console.redhat.com/api/gathering/v2/4.20.8/gathering_rules": dial tcp: lookup console.redhat.com on 172.30.0.10:53: read udp 10.129.0.12:35462->172.30.0.10:53: read: connection refused E0703 18:10:58.333352 1 conditional_gatherer.go:322] unable to update alerts cache: open /var/run/configmaps/service-ca-bundle/service-ca.crt: no such file or directory I0703 18:10:58.333412 1 conditional_gatherer.go:384] updating version cache for conditional gatherer I0703 18:10:58.344435 1 conditional_gatherer.go:392] cluster version is '4.20.8' E0703 18:10:58.344449 1 conditional_gatherer.go:209] error checking conditions for a gathering rule: alerts cache is missing E0703 18:10:58.344455 1 conditional_gatherer.go:209] error checking conditions for a gathering rule: alerts cache is missing E0703 18:10:58.344458 1 conditional_gatherer.go:209] error checking conditions for a gathering rule: alerts cache is missing E0703 18:10:58.344461 1 conditional_gatherer.go:209] error checking conditions for a gathering rule: alerts cache is missing E0703 18:10:58.344464 1 conditional_gatherer.go:209] error checking conditions for a gathering rule: alerts cache is missing E0703 18:10:58.344469 1 conditional_gatherer.go:209] error checking conditions for a gathering rule: alerts cache is missing E0703 18:10:58.344472 1 conditional_gatherer.go:209] error checking conditions for a gathering rule: alerts cache is missing E0703 18:10:58.344475 1 conditional_gatherer.go:209] error checking conditions for a gathering rule: alerts cache is missing E0703 18:10:58.344479 1 conditional_gatherer.go:209] error checking conditions for a gathering rule: alerts cache is missing I0703 18:10:58.344494 1 tasks_processing.go:45] number of workers: 3 I0703 18:10:58.344506 1 tasks_processing.go:69] worker 2 listening for tasks. I0703 18:10:58.344510 1 tasks_processing.go:71] worker 2 working on remote_configuration task. I0703 18:10:58.344518 1 tasks_processing.go:69] worker 0 listening for tasks. I0703 18:10:58.344533 1 tasks_processing.go:69] worker 1 listening for tasks. I0703 18:10:58.344538 1 tasks_processing.go:71] worker 2 working on conditional_gatherer_rules task. I0703 18:10:58.344535 1 tasks_processing.go:71] worker 0 working on rapid_container_logs task. I0703 18:10:58.344544 1 tasks_processing.go:74] worker 1 stopped. I0703 18:10:58.344570 1 recorder.go:75] Recording insights-operator/remote-configuration with fingerprint=0394430c431eec4d48bb1811a90918e95161d2282c59af26f2473613cc0959db I0703 18:10:58.344580 1 gather.go:177] gatherer "conditional" function "remote_configuration" took 819ns to process 1 records I0703 18:10:58.344647 1 tasks_processing.go:74] worker 2 stopped. I0703 18:10:58.344673 1 recorder.go:75] Recording insights-operator/conditional-gatherer-rules with fingerprint=7034af97e7e41c22e4b775abdd4b9066c8ebb19da33eb7f69f39bfd2eb5f6406 I0703 18:10:58.344681 1 gather.go:177] gatherer "conditional" function "conditional_gatherer_rules" took 2.221µs to process 1 records I0703 18:10:58.344817 1 tasks_processing.go:74] worker 0 stopped. I0703 18:10:58.344828 1 gather.go:177] gatherer "conditional" function "rapid_container_logs" took 263.522µs to process 0 records I0703 18:10:58.344849 1 controllerstatus.go:89] name=periodic-conditional healthy=false reason=NotAvailable message=Get "https://console.redhat.com/api/gathering/v2/4.20.8/gathering_rules": dial tcp: lookup console.redhat.com on 172.30.0.10:53: read udp 10.129.0.12:35462->172.30.0.10:53: read: connection refused I0703 18:10:58.344866 1 recorder.go:75] Recording insights-operator/remote-configuration.json with fingerprint=359de9c990c741675cec72fda96b5c3682221efdb4799f5eaa6e9805bcd3b5c1 I0703 18:10:58.373388 1 recorder.go:75] Recording insights-operator/gathers with fingerprint=2b67badb32e41e07d570be16aff6923801bd29b3b63d47f782c6279495382e0f I0703 18:10:58.373551 1 diskrecorder.go:70] Writing 101 records to /var/lib/insights-operator/insights-2026-07-03-181058.tar.gz I0703 18:10:58.380554 1 diskrecorder.go:51] Wrote 101 records to disk in 7ms I0703 18:10:58.380581 1 periodic.go:278] Gathering cluster info every 2h0m0s I0703 18:10:58.380594 1 periodic.go:279] Configuration is dataReporting: interval: 2h0m0s, uploadEndpoint: https://console.redhat.com/api/ingress/v1/upload, storagePath: /var/lib/insights-operator, downloadEndpoint: https://console.redhat.com/api/insights-results-aggregator/v2/cluster/%s/reports, conditionalGathererEndpoint: https://console.redhat.com/api/gathering/v2/%s/gathering_rules, obfuscation: [] sca: disabled: false, endpoint: https://api.openshift.com/api/accounts_mgmt/v1/entitlement_certificates, interval: 8h0m0s alerting: disabled: false clusterTransfer: endpoint: https://api.openshift.com/api/accounts_mgmt/v1/cluster_transfers/, interval: 12h0m0s proxy: httpProxy: , httpsProxy: , noProxy: I0703 18:11:08.911092 1 configmapobserver.go:84] configmaps "insights-config" not found I0703 18:11:46.134259 1 observer_polling.go:111] Observed file "/var/run/secrets/serving-cert/tls.crt" has been created (hash="fd3e1bd66819564a7466e99ab25dc50cd43c78bad725e2309f59aa2711ad3f2d") W0703 18:11:46.134313 1 builder.go:160] Restart triggered because of file /var/run/secrets/serving-cert/tls.crt was created I0703 18:11:46.134390 1 observer_polling.go:111] Observed file "/var/run/secrets/serving-cert/tls.key" has been created (hash="443a1c0d4e336ac7694782654e3c82446cbc677407896f3aaa878766c270404c") I0703 18:11:46.134516 1 observer_polling.go:111] Observed file "/var/run/configmaps/service-ca-bundle/service-ca.crt" has been created (hash="32ee368d2a6908eb625b34de875ecac02deb788047dc2d4fcef7f96e2e636ced") I0703 18:11:46.134564 1 genericapiserver.go:548] "[graceful-termination] shutdown event" name="ShutdownInitiated" I0703 18:11:46.134877 1 periodic.go:170] Shutting down I0703 18:11:46.134905 1 base_controller.go:181] Shutting down LoggingSyncer ... I0703 18:11:46.134920 1 base_controller.go:181] Shutting down ConfigController ... E0703 18:11:46.135035 1 controller.go:299] Unable to write cluster operator status: client rate limiter Wait returned an error: context canceled E0703 18:11:46.135072 1 controller.go:299] Unable to write cluster operator status: client rate limiter Wait returned an error: context canceled E0703 18:11:46.135103 1 controller.go:299] Unable to write cluster operator status: client rate limiter Wait returned an error: context canceled E0703 18:11:46.135130 1 controller.go:299] Unable to write cluster operator status: client rate limiter Wait returned an error: context canceled E0703 18:11:46.135156 1 controller.go:299] Unable to write cluster operator status: client rate limiter Wait returned an error: context canceled E0703 18:11:46.135188 1 controller.go:299] Unable to write cluster operator status: client rate limiter Wait returned an error: context canceled E0703 18:11:46.135227 1 controller.go:299] Unable to write cluster operator status: client rate limiter Wait returned an error: context canceled E0703 18:11:46.135250 1 controller.go:299] Unable to write cluster operator status: client rate limiter Wait returned an error: context canceled E0703 18:11:46.135271 1 controller.go:299] Unable to write cluster operator status: client rate limiter Wait returned an error: context canceled E0703 18:11:46.135296 1 controller.go:299] Unable to write cluster operator status: client rate limiter Wait returned an error: context canceled E0703 18:11:46.135320 1 controller.go:299] Unable to write cluster operator status: client rate limiter Wait returned an error: context canceled E0703 18:11:46.135341 1 controller.go:299] Unable to write cluster operator status: client rate limiter Wait returned an error: context canceled E0703 18:11:46.135362 1 controller.go:299] Unable to write cluster operator status: client rate limiter Wait returned an error: context canceled E0703 18:11:46.135387 1 controller.go:299] Unable to write cluster operator status: client rate limiter Wait returned an error: context canceled E0703 18:11:46.135432 1 controller.go:299] Unable to write cluster operator status: client rate limiter Wait returned an error: context canceled E0703 18:11:46.135463 1 controller.go:299] Unable to write cluster operator status: client rate limiter Wait returned an error: context canceled E0703 18:11:46.135491 1 controller.go:299] Unable to write cluster operator status: client rate limiter Wait returned an error: context canceled E0703 18:11:46.135514 1 controller.go:299] Unable to write cluster operator status: client rate limiter Wait returned an error: context canceled E0703 18:11:46.135540 1 controller.go:299] Unable to write cluster operator status: client rate limiter Wait returned an error: context canceled E0703 18:11:46.135560 1 controller.go:299] Unable to write cluster operator status: client rate limiter Wait returned an error: context canceled E0703 18:11:46.135584 1 controller.go:299] Unable to write cluster operator status: client rate limiter Wait returned an error: context canceled E0703 18:11:46.135637 1 controller.go:299] Unable to write cluster operator status: client rate limiter Wait returned an error: context canceled E0703 18:11:46.135665 1 controller.go:299] Unable to write cluster operator status: client rate limiter Wait returned an error: context canceled E0703 18:11:46.135709 1 controller.go:299] Unable to write cluster operator status: client rate limiter Wait returned an error: context canceled E0703 18:11:46.135743 1 controller.go:299] Unable to write cluster operator status: client rate limiter Wait returned an error: context canceled E0703 18:11:46.136045 1 controller.go:299] Unable to write cluster operator status: client rate limiter Wait returned an error: context canceled E0703 18:11:46.136096 1 controller.go:299] Unable to write cluster operator status: client rate limiter Wait returned an error: context canceled E0703 18:11:46.136145 1 controller.go:299] Unable to write cluster operator status: client rate limiter Wait returned an error: context canceled E0703 18:11:46.136180 1 controller.go:299] Unable to write cluster operator status: client rate limiter Wait returned an error: context canceled E0703 18:11:46.136207 1 controller.go:299] Unable to write cluster operator status: client rate limiter Wait returned an error: context canceled E0703 18:11:46.136253 1 controller.go:299] Unable to write cluster operator status: client rate limiter Wait returned an error: context canceled E0703 18:11:46.136491 1 controller.go:299] Unable to write cluster operator status: client rate limiter Wait returned an error: context canceled E0703 18:11:46.136533 1 controller.go:299] Unable to write cluster operator status: client rate limiter Wait returned an error: context canceled E0703 18:11:46.136557 1 controller.go:299] Unable to write cluster operator status: client rate limiter Wait returned an error: context canceled E0703 18:11:46.136586 1 controller.go:299] Unable to write cluster operator status: client rate limiter Wait returned an error: context canceled E0703 18:11:46.136648 1 controller.go:299] Unable to write cluster operator status: client rate limiter Wait returned an error: context canceled E0703 18:11:46.136673 1 controller.go:299] Unable to write cluster operator status: client rate limiter Wait returned an error: context canceled E0703 18:11:46.137161 1 controller.go:299] Unable to write cluster operator status: client rate limiter Wait returned an error: context canceled E0703 18:11:46.137213 1 controller.go:299] Unable to write cluster operator status: client rate limiter Wait returned an error: context canceled E0703 18:11:46.137252 1 controller.go:299] Unable to write cluster operator status: client rate limiter Wait returned an error: context canceled E0703 18:11:46.137281 1 controller.go:299] Unable to write cluster operator status: client rate limiter Wait returned an error: context canceled