W0410 16:38:15.667165 1 cmd.go:257] Using insecure, self-signed certificates I0410 16:38:16.515628 1 start.go:138] Unable to read service ca bundle: open /var/run/configmaps/service-ca-bundle/service-ca.crt: no such file or directory I0410 16:38:16.515935 1 observer_polling.go:159] Starting file observer I0410 16:38:17.099430 1 operator.go:60] Starting insights-operator v0.0.0-master+$Format:%H$ I0410 16:38:17.099627 1 legacy_config.go:327] Current config: {"report":false,"storagePath":"/var/lib/insights-operator","interval":"2h","endpoint":"https://console.redhat.com/api/ingress/v1/upload","conditionalGathererEndpoint":"https://console.redhat.com/api/gathering/v2/%s/gathering_rules","pull_report":{"endpoint":"https://console.redhat.com/api/insights-results-aggregator/v2/cluster/%s/reports","delay":"60s","timeout":"3000s","min_retry":"30s"},"impersonate":"system:serviceaccount:openshift-insights:gather","enableGlobalObfuscation":false,"ocm":{"scaEndpoint":"https://api.openshift.com/api/accounts_mgmt/v1/entitlement_certificates","scaInterval":"8h","scaDisabled":false,"clusterTransferEndpoint":"https://api.openshift.com/api/accounts_mgmt/v1/cluster_transfers/","clusterTransferInterval":"12h"},"disableInsightsAlerts":false,"processingStatusEndpoint":"https://console.redhat.com/api/insights-results-aggregator/v2/cluster/%s/request/%s/status","reportEndpointTechPreview":"https://console.redhat.com/api/insights-results-aggregator/v2/cluster/%s/request/%s/report"} I0410 16:38:17.100236 1 secure_serving.go:57] Forcing use of http/1.1 only I0410 16:38:17.100263 1 simple_featuregate_reader.go:171] Starting feature-gate-detector W0410 16:38:17.100267 1 secure_serving.go:69] Use of insecure cipher 'TLS_ECDHE_ECDSA_WITH_AES_128_CBC_SHA256' detected. W0410 16:38:17.100280 1 secure_serving.go:69] Use of insecure cipher 'TLS_ECDHE_RSA_WITH_AES_128_CBC_SHA256' detected. W0410 16:38:17.100286 1 secure_serving.go:69] Use of insecure cipher 'TLS_RSA_WITH_AES_128_GCM_SHA256' detected. W0410 16:38:17.100289 1 secure_serving.go:69] Use of insecure cipher 'TLS_RSA_WITH_AES_256_GCM_SHA384' detected. W0410 16:38:17.100292 1 secure_serving.go:69] Use of insecure cipher 'TLS_RSA_WITH_AES_128_CBC_SHA' detected. W0410 16:38:17.100295 1 secure_serving.go:69] Use of insecure cipher 'TLS_RSA_WITH_AES_256_CBC_SHA' detected. I0410 16:38:17.106830 1 operator.go:125] FeatureGates initialized: knownFeatureGates=[AdditionalRoutingCapabilities AdminNetworkPolicy AlibabaPlatform AzureWorkloadIdentity BuildCSIVolumes CPMSMachineNamePrefix ConsolePluginContentSecurityPolicy ExternalOIDC ExternalOIDCWithUIDAndExtraClaimMappings GatewayAPI GatewayAPIController HighlyAvailableArbiter ImageVolume IngressControllerLBSubnetsAWS KMSv1 MachineConfigNodes ManagedBootImages ManagedBootImagesAWS MetricsCollectionProfiles NetworkDiagnosticsConfig NetworkLiveMigration NetworkSegmentation PinnedImages ProcMountType RouteAdvertisements RouteExternalCertificate ServiceAccountTokenNodeBinding SetEIPForNLBIngressController SigstoreImageVerification StoragePerformantSecurityPolicy UpgradeStatus UserNamespacesPodSecurityStandards UserNamespacesSupport VSphereMultiDisk VSphereMultiNetworks AWSClusterHostedDNS AWSClusterHostedDNSInstall AWSDedicatedHosts AWSServiceLBNetworkSecurityGroup AutomatedEtcdBackup AzureClusterHostedDNSInstall AzureDedicatedHosts AzureMultiDisk BootImageSkewEnforcement BootcNodeManagement ClusterAPIInstall ClusterAPIInstallIBMCloud ClusterMonitoringConfig ClusterVersionOperatorConfiguration DNSNameResolver DualReplica DyanmicServiceEndpointIBMCloud DynamicResourceAllocation EtcdBackendQuota EventedPLEG Example Example2 ExternalSnapshotMetadata GCPClusterHostedDNS GCPClusterHostedDNSInstall GCPCustomAPIEndpoints GCPCustomAPIEndpointsInstall ImageModeStatusReporting ImageStreamImportMode IngressControllerDynamicConfigurationManager InsightsConfig InsightsConfigAPI InsightsOnDemandDataGather IrreconcilableMachineConfig KMSEncryptionProvider MachineAPIMigration MachineAPIOperatorDisableMachineHealthCheckController ManagedBootImagesAzure ManagedBootImagesvSphere MaxUnavailableStatefulSet MinimumKubeletVersion MixedCPUsAllocation MultiArchInstallAzure MultiDiskSetup MutatingAdmissionPolicy NewOLM NewOLMCatalogdAPIV1Metas NewOLMOwnSingleNamespace NewOLMPreflightPermissionChecks NewOLMWebhookProviderOpenshiftServiceCA NoRegistryClusterOperations NodeSwap NutanixMultiSubnets OVNObservability OpenShiftPodSecurityAdmission PreconfiguredUDNAddresses SELinuxMount ShortCertRotation SignatureStores SigstoreImageVerificationPKI TranslateStreamCloseWebsocketRequests VSphereConfigurableMaxAllowedBlockVolumesPerNode VSphereHostVMGroupZonal VSphereMixedNodeEnv VolumeAttributesClass VolumeGroupSnapshot] I0410 16:38:17.106848 1 event.go:377] Event(v1.ObjectReference{Kind:"Deployment", Namespace:"openshift-insights", Name:"insights-operator", UID:"792284c8-639e-45b2-956c-90b0f13cef77", APIVersion:"apps/v1", ResourceVersion:"", FieldPath:""}): type: 'Normal' reason: 'FeatureGatesInitialized' FeatureGates updated to featuregates.Features{Enabled:[]v1.FeatureGateName{"AdditionalRoutingCapabilities", "AdminNetworkPolicy", "AlibabaPlatform", "AzureWorkloadIdentity", "BuildCSIVolumes", "CPMSMachineNamePrefix", "ConsolePluginContentSecurityPolicy", "ExternalOIDC", "ExternalOIDCWithUIDAndExtraClaimMappings", "GatewayAPI", "GatewayAPIController", "HighlyAvailableArbiter", "ImageVolume", "IngressControllerLBSubnetsAWS", "KMSv1", "MachineConfigNodes", "ManagedBootImages", "ManagedBootImagesAWS", "MetricsCollectionProfiles", "NetworkDiagnosticsConfig", "NetworkLiveMigration", "NetworkSegmentation", "PinnedImages", "ProcMountType", "RouteAdvertisements", "RouteExternalCertificate", "ServiceAccountTokenNodeBinding", "SetEIPForNLBIngressController", "SigstoreImageVerification", "StoragePerformantSecurityPolicy", "UpgradeStatus", "UserNamespacesPodSecurityStandards", "UserNamespacesSupport", "VSphereMultiDisk", "VSphereMultiNetworks"}, Disabled:[]v1.FeatureGateName{"AWSClusterHostedDNS", "AWSClusterHostedDNSInstall", "AWSDedicatedHosts", "AWSServiceLBNetworkSecurityGroup", "AutomatedEtcdBackup", "AzureClusterHostedDNSInstall", "AzureDedicatedHosts", "AzureMultiDisk", "BootImageSkewEnforcement", "BootcNodeManagement", "ClusterAPIInstall", "ClusterAPIInstallIBMCloud", "ClusterMonitoringConfig", "ClusterVersionOperatorConfiguration", "DNSNameResolver", "DualReplica", "DyanmicServiceEndpointIBMCloud", "DynamicResourceAllocation", "EtcdBackendQuota", "EventedPLEG", "Example", "Example2", "ExternalSnapshotMetadata", "GCPClusterHostedDNS", "GCPClusterHostedDNSInstall", "GCPCustomAPIEndpoints", "GCPCustomAPIEndpointsInstall", "ImageModeStatusReporting", "ImageStreamImportMode", "IngressControllerDynamicConfigurationManager", "InsightsConfig", "InsightsConfigAPI", "InsightsOnDemandDataGather", "IrreconcilableMachineConfig", "KMSEncryptionProvider", "MachineAPIMigration", "MachineAPIOperatorDisableMachineHealthCheckController", "ManagedBootImagesAzure", "ManagedBootImagesvSphere", "MaxUnavailableStatefulSet", "MinimumKubeletVersion", "MixedCPUsAllocation", "MultiArchInstallAzure", "MultiDiskSetup", "MutatingAdmissionPolicy", "NewOLM", "NewOLMCatalogdAPIV1Metas", "NewOLMOwnSingleNamespace", "NewOLMPreflightPermissionChecks", "NewOLMWebhookProviderOpenshiftServiceCA", "NoRegistryClusterOperations", "NodeSwap", "NutanixMultiSubnets", "OVNObservability", "OpenShiftPodSecurityAdmission", "PreconfiguredUDNAddresses", "SELinuxMount", "ShortCertRotation", "SignatureStores", "SigstoreImageVerificationPKI", "TranslateStreamCloseWebsocketRequests", "VSphereConfigurableMaxAllowedBlockVolumesPerNode", "VSphereHostVMGroupZonal", "VSphereMixedNodeEnv", "VolumeAttributesClass", "VolumeGroupSnapshot"}} I0410 16:38:17.108923 1 requestheader_controller.go:180] Starting RequestHeaderAuthRequestController I0410 16:38:17.108937 1 shared_informer.go:313] Waiting for caches to sync for RequestHeaderAuthRequestController I0410 16:38:17.108936 1 configmap_cafile_content.go:205] "Starting controller" name="client-ca::kube-system::extension-apiserver-authentication::client-ca-file" I0410 16:38:17.108946 1 configmap_cafile_content.go:205] "Starting controller" name="client-ca::kube-system::extension-apiserver-authentication::requestheader-client-ca-file" I0410 16:38:17.108951 1 shared_informer.go:313] Waiting for caches to sync for client-ca::kube-system::extension-apiserver-authentication::client-ca-file I0410 16:38:17.108957 1 shared_informer.go:313] Waiting for caches to sync for client-ca::kube-system::extension-apiserver-authentication::requestheader-client-ca-file I0410 16:38:17.109227 1 dynamic_serving_content.go:135] "Starting controller" name="serving-cert::/tmp/serving-cert-1992727005/tls.crt::/tmp/serving-cert-1992727005/tls.key" I0410 16:38:17.109518 1 secure_serving.go:213] Serving securely on [::]:8443 I0410 16:38:17.109544 1 tlsconfig.go:243] "Starting DynamicServingCertificateController" W0410 16:38:17.115448 1 configmapobserver.go:64] Cannot get the configuration config map: configmaps "insights-config" not found. Default configuration is used. I0410 16:38:17.115477 1 secretconfigobserver.go:216] Legacy configuration set: enabled=false endpoint=https://console.redhat.com/api/ingress/v1/upload conditional_gatherer_endpoint=https://console.redhat.com/api/gathering/v2/%s/gathering_rules interval=2h0m0s token=false reportEndpoint=https://console.redhat.com/api/insights-results-aggregator/v2/cluster/%s/reports initialPollingDelay=1m0s minRetryTime=30s pollingTimeout=50m0s processingStatusEndpoint=https://console.redhat.com/api/insights-results-aggregator/v2/cluster/%s/request/%s/status I0410 16:38:17.115612 1 base_controller.go:76] Waiting for caches to sync for ConfigController I0410 16:38:17.127704 1 secretconfigobserver.go:249] Found cloud.openshift.com token I0410 16:38:17.127732 1 secretconfigobserver.go:204] Legacy configuration updated: enabled=true endpoint=https://console.redhat.com/api/ingress/v1/upload conditional_gatherer_endpoint=https://console.redhat.com/api/gathering/v2/%s/gathering_rules interval=2h0m0s token=true reportEndpoint=https://console.redhat.com/api/insights-results-aggregator/v2/cluster/%s/reports initialPollingDelay=1m0s minRetryTime=30s pollingTimeout=50m0s processingStatusEndpoint=https://console.redhat.com/api/insights-results-aggregator/v2/cluster/%s/request/%s/status I0410 16:38:17.140775 1 secretconfigobserver.go:119] support secret does not exist I0410 16:38:17.157042 1 secretconfigobserver.go:249] Found cloud.openshift.com token I0410 16:38:17.161349 1 secretconfigobserver.go:119] support secret does not exist I0410 16:38:17.170512 1 recorder.go:161] Pruning old reports every 5h35m6s, max age is 288h0m0s I0410 16:38:17.178839 1 controllerstatus.go:80] name=insightsreport healthy=true reason= message= I0410 16:38:17.178839 1 controllerstatus.go:80] name=insightsuploader healthy=true reason= message= I0410 16:38:17.178851 1 insightsreport.go:296] Starting report retriever I0410 16:38:17.178866 1 insightsreport.go:298] Insights analysis reports will be downloaded from the https://console.redhat.com/api/insights-results-aggregator/v2/cluster/%s/reports endpoint with a delay of 1m0s I0410 16:38:17.178888 1 periodic.go:209] Running clusterconfig gatherer I0410 16:38:17.178944 1 insightsuploader.go:86] Reporting status periodically to https://console.redhat.com/api/ingress/v1/upload every 2h0m0s, starting in 1m30s I0410 16:38:17.178950 1 tasks_processing.go:45] number of workers: 64 I0410 16:38:17.179027 1 tasks_processing.go:69] worker 21 listening for tasks. I0410 16:38:17.179036 1 tasks_processing.go:69] worker 0 listening for tasks. I0410 16:38:17.179036 1 tasks_processing.go:69] worker 34 listening for tasks. I0410 16:38:17.179042 1 tasks_processing.go:69] worker 46 listening for tasks. I0410 16:38:17.179049 1 tasks_processing.go:71] worker 34 working on pod_network_connectivity_checks task. I0410 16:38:17.179050 1 tasks_processing.go:69] worker 1 listening for tasks. I0410 16:38:17.179049 1 tasks_processing.go:69] worker 47 listening for tasks. I0410 16:38:17.179041 1 tasks_processing.go:71] worker 0 working on container_runtime_configs task. I0410 16:38:17.179059 1 tasks_processing.go:69] worker 48 listening for tasks. I0410 16:38:17.179062 1 tasks_processing.go:69] worker 35 listening for tasks. I0410 16:38:17.179068 1 tasks_processing.go:69] worker 49 listening for tasks. I0410 16:38:17.179071 1 tasks_processing.go:69] worker 36 listening for tasks. I0410 16:38:17.179077 1 tasks_processing.go:69] worker 50 listening for tasks. I0410 16:38:17.179082 1 tasks_processing.go:69] worker 37 listening for tasks. I0410 16:38:17.179084 1 tasks_processing.go:69] worker 51 listening for tasks. I0410 16:38:17.179089 1 tasks_processing.go:69] worker 52 listening for tasks. I0410 16:38:17.179093 1 tasks_processing.go:69] worker 38 listening for tasks. I0410 16:38:17.179096 1 tasks_processing.go:69] worker 53 listening for tasks. I0410 16:38:17.179103 1 tasks_processing.go:69] worker 39 listening for tasks. I0410 16:38:17.179104 1 tasks_processing.go:69] worker 54 listening for tasks. I0410 16:38:17.179109 1 tasks_processing.go:69] worker 55 listening for tasks. I0410 16:38:17.179115 1 tasks_processing.go:69] worker 40 listening for tasks. I0410 16:38:17.179117 1 tasks_processing.go:69] worker 56 listening for tasks. I0410 16:38:17.179122 1 tasks_processing.go:69] worker 57 listening for tasks. I0410 16:38:17.179128 1 tasks_processing.go:69] worker 41 listening for tasks. I0410 16:38:17.179131 1 tasks_processing.go:69] worker 58 listening for tasks. I0410 16:38:17.179136 1 tasks_processing.go:69] worker 42 listening for tasks. I0410 16:38:17.179138 1 tasks_processing.go:69] worker 59 listening for tasks. I0410 16:38:17.179144 1 tasks_processing.go:69] worker 43 listening for tasks. I0410 16:38:17.179145 1 tasks_processing.go:69] worker 60 listening for tasks. I0410 16:38:17.179151 1 tasks_processing.go:69] worker 44 listening for tasks. I0410 16:38:17.179153 1 tasks_processing.go:69] worker 61 listening for tasks. I0410 16:38:17.179160 1 tasks_processing.go:69] worker 45 listening for tasks. I0410 16:38:17.179161 1 tasks_processing.go:69] worker 62 listening for tasks. I0410 16:38:17.179168 1 tasks_processing.go:69] worker 63 listening for tasks. I0410 16:38:17.179172 1 tasks_processing.go:69] worker 16 listening for tasks. I0410 16:38:17.179175 1 tasks_processing.go:69] worker 13 listening for tasks. I0410 16:38:17.179179 1 tasks_processing.go:69] worker 14 listening for tasks. I0410 16:38:17.179180 1 tasks_processing.go:69] worker 12 listening for tasks. I0410 16:38:17.179187 1 tasks_processing.go:69] worker 15 listening for tasks. I0410 16:38:17.179190 1 tasks_processing.go:69] worker 17 listening for tasks. I0410 16:38:17.179198 1 tasks_processing.go:69] worker 6 listening for tasks. I0410 16:38:17.179198 1 tasks_processing.go:69] worker 3 listening for tasks. I0410 16:38:17.179204 1 tasks_processing.go:69] worker 2 listening for tasks. I0410 16:38:17.179209 1 tasks_processing.go:69] worker 4 listening for tasks. I0410 16:38:17.179210 1 tasks_processing.go:69] worker 7 listening for tasks. I0410 16:38:17.179217 1 tasks_processing.go:69] worker 5 listening for tasks. I0410 16:38:17.179212 1 tasks_processing.go:69] worker 11 listening for tasks. I0410 16:38:17.179216 1 tasks_processing.go:69] worker 18 listening for tasks. I0410 16:38:17.179230 1 tasks_processing.go:69] worker 10 listening for tasks. I0410 16:38:17.179221 1 tasks_processing.go:69] worker 9 listening for tasks. I0410 16:38:17.179236 1 tasks_processing.go:69] worker 23 listening for tasks. I0410 16:38:17.179226 1 tasks_processing.go:69] worker 24 listening for tasks. I0410 16:38:17.179231 1 tasks_processing.go:69] worker 8 listening for tasks. I0410 16:38:17.179040 1 tasks_processing.go:71] worker 21 working on storage_classes task. I0410 16:38:17.179249 1 tasks_processing.go:69] worker 33 listening for tasks. I0410 16:38:17.179249 1 tasks_processing.go:69] worker 20 listening for tasks. I0410 16:38:17.179249 1 tasks_processing.go:69] worker 31 listening for tasks. I0410 16:38:17.179255 1 tasks_processing.go:69] worker 30 listening for tasks. I0410 16:38:17.179256 1 tasks_processing.go:69] worker 29 listening for tasks. I0410 16:38:17.179235 1 tasks_processing.go:69] worker 19 listening for tasks. I0410 16:38:17.179263 1 tasks_processing.go:71] worker 46 working on ingress task. I0410 16:38:17.179242 1 tasks_processing.go:69] worker 27 listening for tasks. I0410 16:38:17.179269 1 tasks_processing.go:71] worker 49 working on tsdb_status task. I0410 16:38:17.179267 1 tasks_processing.go:69] worker 32 listening for tasks. I0410 16:38:17.179276 1 tasks_processing.go:71] worker 51 working on container_images task. I0410 16:38:17.179277 1 tasks_processing.go:71] worker 36 working on nodenetworkstates task. I0410 16:38:17.179281 1 tasks_processing.go:71] worker 27 working on lokistack task. I0410 16:38:17.179260 1 tasks_processing.go:69] worker 26 listening for tasks. I0410 16:38:17.179280 1 tasks_processing.go:71] worker 50 working on machine_healthchecks task. I0410 16:38:17.179292 1 tasks_processing.go:71] worker 56 working on number_of_pods_and_netnamespaces_with_sdn_annotations task. W0410 16:38:17.179294 1 gather_prometheus_tsdb_status.go:38] Unable to load metrics client, tsdb status cannot be collected: open /var/run/configmaps/service-ca-bundle/service-ca.crt: no such file or directory I0410 16:38:17.179289 1 tasks_processing.go:71] worker 40 working on service_accounts task. I0410 16:38:17.179301 1 tasks_processing.go:71] worker 1 working on install_plans task. I0410 16:38:17.179296 1 tasks_processing.go:71] worker 47 working on openstack_controlplanes task. I0410 16:38:17.179314 1 gather.go:177] gatherer "clusterconfig" function "tsdb_status" took 26.875µs to process 0 records I0410 16:38:17.179325 1 tasks_processing.go:71] worker 57 working on machines task. I0410 16:38:17.179279 1 tasks_processing.go:71] worker 38 working on networks task. I0410 16:38:17.179235 1 tasks_processing.go:69] worker 22 listening for tasks. I0410 16:38:17.179436 1 tasks_processing.go:71] worker 15 working on pdbs task. I0410 16:38:17.179304 1 tasks_processing.go:71] worker 49 working on infrastructures task. I0410 16:38:17.179294 1 tasks_processing.go:71] worker 26 working on operators_pods_and_events task. I0410 16:38:17.179238 1 tasks_processing.go:69] worker 28 listening for tasks. I0410 16:38:17.179259 1 tasks_processing.go:71] worker 35 working on overlapping_namespace_uids task. I0410 16:38:17.179437 1 tasks_processing.go:71] worker 63 working on ceph_cluster task. I0410 16:38:17.179270 1 tasks_processing.go:71] worker 37 working on schedulers task. I0410 16:38:17.179264 1 tasks_processing.go:71] worker 52 working on nodenetworkconfigurationpolicies task. I0410 16:38:17.179272 1 tasks_processing.go:71] worker 41 working on image task. I0410 16:38:17.179281 1 tasks_processing.go:71] worker 32 working on node_logs task. I0410 16:38:17.179288 1 tasks_processing.go:71] worker 39 working on crds task. I0410 16:38:17.179377 1 tasks_processing.go:71] worker 48 working on sap_config task. I0410 16:38:17.179384 1 tasks_processing.go:71] worker 55 working on machine_autoscalers task. I0410 16:38:17.179390 1 tasks_processing.go:71] worker 62 working on active_alerts task. I0410 16:38:17.179394 1 tasks_processing.go:71] worker 58 working on support_secret task. I0410 16:38:17.179398 1 tasks_processing.go:71] worker 42 working on metrics task. I0410 16:38:17.179392 1 tasks_processing.go:71] worker 54 working on openstack_version task. W0410 16:38:17.180487 1 gather_most_recent_metrics.go:64] Unable to load metrics client, no metrics will be collected: open /var/run/configmaps/service-ca-bundle/service-ca.crt: no such file or directory I0410 16:38:17.180499 1 tasks_processing.go:71] worker 42 working on clusterroles task. I0410 16:38:17.180573 1 gather.go:177] gatherer "clusterconfig" function "metrics" took 32.078µs to process 0 records I0410 16:38:17.179402 1 tasks_processing.go:71] worker 59 working on operators task. I0410 16:38:17.179420 1 tasks_processing.go:71] worker 61 working on jaegers task. I0410 16:38:17.179424 1 tasks_processing.go:71] worker 45 working on machine_sets task. I0410 16:38:17.179427 1 tasks_processing.go:71] worker 44 working on monitoring_persistent_volumes task. I0410 16:38:17.179430 1 tasks_processing.go:71] worker 43 working on validating_webhook_configurations task. I0410 16:38:17.179431 1 tasks_processing.go:71] worker 14 working on openstack_dataplanenodesets task. I0410 16:38:17.179432 1 tasks_processing.go:71] worker 60 working on openshift_logging task. I0410 16:38:17.179439 1 tasks_processing.go:71] worker 23 working on sap_datahubs task. I0410 16:38:17.179441 1 tasks_processing.go:71] worker 16 working on openstack_dataplanedeployments task. I0410 16:38:17.179442 1 tasks_processing.go:71] worker 22 working on proxies task. I0410 16:38:17.179444 1 tasks_processing.go:71] worker 2 working on mutating_webhook_configurations task. I0410 16:38:17.179447 1 tasks_processing.go:71] worker 4 working on machine_config_pools task. I0410 16:38:17.179446 1 tasks_processing.go:71] worker 13 working on olm_operators task. I0410 16:38:17.179449 1 tasks_processing.go:71] worker 17 working on config_maps task. I0410 16:38:17.179451 1 tasks_processing.go:71] worker 7 working on aggregated_monitoring_cr_names task. I0410 16:38:17.179452 1 tasks_processing.go:71] worker 20 working on openshift_machine_api_events task. I0410 16:38:17.179455 1 tasks_processing.go:71] worker 5 working on sap_pods task. I0410 16:38:17.179456 1 tasks_processing.go:71] worker 24 working on storage_cluster task. I0410 16:38:17.179459 1 tasks_processing.go:71] worker 11 working on dvo_metrics task. I0410 16:38:17.179459 1 tasks_processing.go:71] worker 8 working on qemu_kubevirt_launcher_logs task. I0410 16:38:17.179458 1 tasks_processing.go:71] worker 12 working on machine_configs task. I0410 16:38:17.179462 1 tasks_processing.go:71] worker 18 working on oauths task. I0410 16:38:17.179464 1 tasks_processing.go:71] worker 29 working on certificate_signing_requests task. I0410 16:38:17.179463 1 tasks_processing.go:71] worker 33 working on cluster_apiserver task. I0410 16:38:17.179466 1 tasks_processing.go:71] worker 10 working on image_pruners task. I0410 16:38:17.179468 1 tasks_processing.go:71] worker 31 working on nodes task. I0410 16:38:17.179470 1 tasks_processing.go:71] worker 9 working on version task. I0410 16:38:17.179470 1 tasks_processing.go:71] worker 6 working on ingress_certificates task. I0410 16:38:17.179474 1 tasks_processing.go:71] worker 19 working on authentication task. I0410 16:38:17.179476 1 tasks_processing.go:71] worker 30 working on silenced_alerts task. I0410 16:38:17.179283 1 tasks_processing.go:71] worker 53 working on cost_management_metrics_configs task. I0410 16:38:17.180035 1 tasks_processing.go:71] worker 3 working on feature_gates task. I0410 16:38:17.180784 1 tasks_processing.go:71] worker 28 working on image_registries task. W0410 16:38:17.180995 1 gather_silenced_alerts.go:38] Unable to load alerts client, no alerts will be collected: open /var/run/configmaps/service-ca-bundle/service-ca.crt: no such file or directory I0410 16:38:17.181013 1 tasks_processing.go:74] worker 30 stopped. I0410 16:38:17.181024 1 gather.go:177] gatherer "clusterconfig" function "silenced_alerts" took 31.566µs to process 0 records I0410 16:38:17.179246 1 tasks_processing.go:69] worker 25 listening for tasks. I0410 16:38:17.181180 1 tasks_processing.go:74] worker 25 stopped. W0410 16:38:17.180991 1 gather_active_alerts.go:54] Unable to load alerts client, no alerts will be collected: open /var/run/configmaps/service-ca-bundle/service-ca.crt: no such file or directory I0410 16:38:17.181189 1 tasks_processing.go:74] worker 62 stopped. I0410 16:38:17.181197 1 gather.go:177] gatherer "clusterconfig" function "active_alerts" took 855.763µs to process 0 records I0410 16:38:17.182088 1 tasks_processing.go:74] worker 0 stopped. I0410 16:38:17.182266 1 gather.go:177] gatherer "clusterconfig" function "container_runtime_configs" took 3.019812ms to process 0 records I0410 16:38:17.183121 1 tasks_processing.go:74] worker 36 stopped. I0410 16:38:17.183186 1 gather.go:177] gatherer "clusterconfig" function "nodenetworkstates" took 3.832552ms to process 0 records I0410 16:38:17.183642 1 tasks_processing.go:74] worker 34 stopped. E0410 16:38:17.183653 1 gather.go:140] gatherer "clusterconfig" function "pod_network_connectivity_checks" failed with the error: the server could not find the requested resource (get podnetworkconnectivitychecks.controlplane.operator.openshift.io) I0410 16:38:17.183660 1 gather.go:177] gatherer "clusterconfig" function "pod_network_connectivity_checks" took 4.581945ms to process 0 records I0410 16:38:17.183701 1 tasks_processing.go:74] worker 50 stopped. E0410 16:38:17.183718 1 gather.go:140] gatherer "clusterconfig" function "machine_healthchecks" failed with the error: machinehealthchecks.machine.openshift.io is forbidden: User "system:serviceaccount:openshift-insights:gather" cannot list resource "machinehealthchecks" in API group "machine.openshift.io" at the cluster scope I0410 16:38:17.183733 1 gather.go:177] gatherer "clusterconfig" function "machine_healthchecks" took 4.400564ms to process 0 records I0410 16:38:17.186884 1 tasks_processing.go:74] worker 48 stopped. I0410 16:38:17.186897 1 gather.go:177] gatherer "clusterconfig" function "sap_config" took 6.644085ms to process 0 records I0410 16:38:17.186905 1 gather.go:177] gatherer "clusterconfig" function "machine_autoscalers" took 6.591598ms to process 0 records I0410 16:38:17.186911 1 gather.go:177] gatherer "clusterconfig" function "openstack_version" took 6.414772ms to process 0 records I0410 16:38:17.186916 1 tasks_processing.go:74] worker 54 stopped. I0410 16:38:17.186919 1 tasks_processing.go:74] worker 55 stopped. I0410 16:38:17.188045 1 tasks_processing.go:74] worker 63 stopped. I0410 16:38:17.188057 1 gather.go:177] gatherer "clusterconfig" function "ceph_cluster" took 8.236197ms to process 0 records I0410 16:38:17.188063 1 gather.go:177] gatherer "clusterconfig" function "openstack_controlplanes" took 8.739755ms to process 0 records I0410 16:38:17.188068 1 tasks_processing.go:74] worker 47 stopped. I0410 16:38:17.188075 1 tasks_processing.go:74] worker 27 stopped. I0410 16:38:17.188085 1 gather.go:177] gatherer "clusterconfig" function "lokistack" took 8.784188ms to process 0 records I0410 16:38:17.188205 1 tasks_processing.go:74] worker 52 stopped. I0410 16:38:17.188217 1 gather.go:177] gatherer "clusterconfig" function "nodenetworkconfigurationpolicies" took 8.199465ms to process 0 records I0410 16:38:17.189447 1 controller.go:128] Initializing last reported time to 0001-01-01T00:00:00Z I0410 16:38:17.189462 1 controller.go:212] Source periodic-clusterconfig *controllerstatus.Simple is not ready I0410 16:38:17.189467 1 controller.go:212] Source periodic-conditional *controllerstatus.Simple is not ready I0410 16:38:17.189471 1 controller.go:212] Source periodic-workloads *controllerstatus.Simple is not ready I0410 16:38:17.189485 1 controller.go:489] The operator is still being initialized I0410 16:38:17.189491 1 controller.go:512] The operator is healthy I0410 16:38:17.190211 1 tasks_processing.go:74] worker 61 stopped. I0410 16:38:17.190221 1 gather.go:177] gatherer "clusterconfig" function "jaegers" took 9.527139ms to process 0 records I0410 16:38:17.190694 1 tasks_processing.go:74] worker 45 stopped. I0410 16:38:17.190707 1 gather.go:177] gatherer "clusterconfig" function "machine_sets" took 9.98627ms to process 0 records I0410 16:38:17.191055 1 tasks_processing.go:74] worker 49 stopped. I0410 16:38:17.191558 1 recorder.go:75] Recording config/infrastructure with fingerprint=9099832542006c4502937d05888d5f72b71583529d7d3f5fce8b9229cb22f2a6 I0410 16:38:17.191571 1 gather.go:177] gatherer "clusterconfig" function "infrastructures" took 11.480532ms to process 1 records E0410 16:38:17.191582 1 gather.go:140] gatherer "clusterconfig" function "machines" failed with the error: machines.machine.openshift.io is forbidden: User "system:serviceaccount:openshift-insights:gather" cannot list resource "machines" in API group "machine.openshift.io" at the cluster scope I0410 16:38:17.191588 1 gather.go:177] gatherer "clusterconfig" function "machines" took 11.929143ms to process 0 records I0410 16:38:17.191596 1 tasks_processing.go:74] worker 57 stopped. I0410 16:38:17.193891 1 tasks_processing.go:74] worker 33 stopped. I0410 16:38:17.194003 1 recorder.go:75] Recording config/apiserver with fingerprint=02baf48dad7125c34686d8ea609f6c862ab1e99de77b169a72a217a473a14655 I0410 16:38:17.194014 1 gather.go:177] gatherer "clusterconfig" function "cluster_apiserver" took 13.100346ms to process 1 records I0410 16:38:17.197385 1 tasks_processing.go:74] worker 19 stopped. I0410 16:38:17.197695 1 recorder.go:75] Recording config/authentication with fingerprint=6a7978d66440e64bdf889774795f0c1335b2b90e35c6c4fb62df58f5ccaad2fa I0410 16:38:17.197711 1 gather.go:177] gatherer "clusterconfig" function "authentication" took 16.498826ms to process 1 records E0410 16:38:17.197723 1 gather.go:140] gatherer "clusterconfig" function "support_secret" failed with the error: secrets "support" not found I0410 16:38:17.197728 1 gather.go:177] gatherer "clusterconfig" function "support_secret" took 17.072354ms to process 0 records I0410 16:38:17.197764 1 tasks_processing.go:74] worker 58 stopped. I0410 16:38:17.197842 1 tasks_processing.go:74] worker 10 stopped. I0410 16:38:17.197919 1 recorder.go:75] Recording config/clusteroperator/imageregistry.operator.openshift.io/imagepruner/cluster with fingerprint=bcef40d1d8e51fcdb7c907d130dc6b095a99f461d89266084e65d75c80474b63 I0410 16:38:17.197928 1 gather.go:177] gatherer "clusterconfig" function "image_pruners" took 16.687376ms to process 1 records I0410 16:38:17.198005 1 recorder.go:75] Recording config/featuregate with fingerprint=cbc1ce301d556e04d4582b4a23a50ff3ab48bf221bf764bd955e6018f0f21fb5 I0410 16:38:17.198007 1 tasks_processing.go:74] worker 3 stopped. I0410 16:38:17.198012 1 gather.go:177] gatherer "clusterconfig" function "feature_gates" took 17.086639ms to process 1 records I0410 16:38:17.198550 1 tasks_processing.go:74] worker 21 stopped. I0410 16:38:17.198653 1 recorder.go:75] Recording config/storage/storageclasses/gp2-csi with fingerprint=1468271acb42d19d23f8281ef71a6e1a3358f26c6852d89575b0d4bd4d8c5969 I0410 16:38:17.198672 1 recorder.go:75] Recording config/storage/storageclasses/gp3-csi with fingerprint=bd73b4edd9f8da561da115600b0979064e18d935d1fda575d89ae4de9e70f752 I0410 16:38:17.198678 1 gather.go:177] gatherer "clusterconfig" function "storage_classes" took 19.29276ms to process 2 records I0410 16:38:17.198748 1 tasks_processing.go:74] worker 13 stopped. I0410 16:38:17.198760 1 gather.go:177] gatherer "clusterconfig" function "olm_operators" took 17.796851ms to process 0 records I0410 16:38:17.207982 1 tasks_processing.go:74] worker 20 stopped. I0410 16:38:17.207999 1 gather.go:177] gatherer "clusterconfig" function "openshift_machine_api_events" took 27.251271ms to process 0 records I0410 16:38:17.208200 1 tasks_processing.go:74] worker 46 stopped. I0410 16:38:17.208354 1 recorder.go:75] Recording config/ingress with fingerprint=d18dbb86bfef8774a1ae66d19267aa4504514340635148392bf302b893b9debe I0410 16:38:17.208375 1 gather.go:177] gatherer "clusterconfig" function "ingress" took 28.928951ms to process 1 records I0410 16:38:17.208385 1 gather.go:177] gatherer "clusterconfig" function "cost_management_metrics_configs" took 27.262818ms to process 0 records I0410 16:38:17.208394 1 tasks_processing.go:74] worker 53 stopped. I0410 16:38:17.208551 1 tasks_processing.go:74] worker 32 stopped. I0410 16:38:17.208568 1 gather.go:177] gatherer "clusterconfig" function "node_logs" took 28.485094ms to process 0 records I0410 16:38:17.208645 1 tasks_processing.go:74] worker 41 stopped. I0410 16:38:17.208660 1 recorder.go:75] Recording config/image with fingerprint=a8f8ea5a2008831c1c5ba05216785bd69f99c302c9aa5385d88cf41326177dee I0410 16:38:17.208671 1 gather.go:177] gatherer "clusterconfig" function "image" took 28.560741ms to process 1 records I0410 16:38:17.208754 1 tasks_processing.go:74] worker 38 stopped. I0410 16:38:17.208794 1 recorder.go:75] Recording config/network with fingerprint=c0c44af09334472a539db75d37497eb8d195819216e853dc890407aea7f18ef0 I0410 16:38:17.208808 1 gather.go:177] gatherer "clusterconfig" function "networks" took 29.317395ms to process 1 records I0410 16:38:17.208968 1 shared_informer.go:320] Caches are synced for RequestHeaderAuthRequestController I0410 16:38:17.208995 1 shared_informer.go:320] Caches are synced for client-ca::kube-system::extension-apiserver-authentication::requestheader-client-ca-file I0410 16:38:17.209003 1 shared_informer.go:320] Caches are synced for client-ca::kube-system::extension-apiserver-authentication::client-ca-file I0410 16:38:17.213373 1 tasks_processing.go:74] worker 15 stopped. I0410 16:38:17.213469 1 gather_logs.go:145] no pods in namespace were found I0410 16:38:17.213520 1 recorder.go:75] Recording config/pdbs/openshift-image-registry/image-registry with fingerprint=0655284af144055054c5d198e56f614c2e48f60d2f732c821ab051c13ad33720 I0410 16:38:17.213546 1 recorder.go:75] Recording config/pdbs/openshift-ingress/router-default with fingerprint=228f5b9f8c953f56f18436d92150d614db66a5205dfeacb90f45b2ff00baa7cc I0410 16:38:17.213564 1 recorder.go:75] Recording config/pdbs/openshift-operator-lifecycle-manager/packageserver-pdb with fingerprint=c646c52dc46250d77688ea0210ef0238511ef451347b9616ab2978810973acb3 I0410 16:38:17.213572 1 gather.go:177] gatherer "clusterconfig" function "pdbs" took 33.880343ms to process 3 records I0410 16:38:17.213578 1 gather.go:177] gatherer "clusterconfig" function "qemu_kubevirt_launcher_logs" took 32.620316ms to process 0 records I0410 16:38:17.213584 1 tasks_processing.go:74] worker 8 stopped. I0410 16:38:17.213895 1 tasks_processing.go:74] worker 37 stopped. I0410 16:38:17.214245 1 recorder.go:75] Recording config/schedulers/cluster with fingerprint=7e3b529336fd2b7a60a8882580a81d7233132ce350982fe3a38760e6b8ae09c8 I0410 16:38:17.214275 1 gather.go:177] gatherer "clusterconfig" function "schedulers" took 33.974429ms to process 1 records I0410 16:38:17.215751 1 base_controller.go:82] Caches are synced for ConfigController I0410 16:38:17.215769 1 base_controller.go:119] Starting #1 worker of ConfigController controller ... I0410 16:38:17.216025 1 tasks_processing.go:74] worker 43 stopped. I0410 16:38:17.216808 1 recorder.go:75] Recording config/validatingwebhookconfigurations/multus.openshift.io with fingerprint=fdee43660a5b942a5f98ebb3e5378a15114d9c266f26ce313a191eb828aeb2c8 I0410 16:38:17.216967 1 recorder.go:75] Recording config/validatingwebhookconfigurations/network-node-identity.openshift.io with fingerprint=d61be9c1861270b63740359018e95bc73affa42a6001fd7390c6af736251f5a4 I0410 16:38:17.217002 1 recorder.go:75] Recording config/validatingwebhookconfigurations/performance-addon-operator with fingerprint=8f0802d069a32d15aa18b2b5f3314c595dc3ffc2fe1288a05cd997ac0167e995 I0410 16:38:17.217040 1 recorder.go:75] Recording config/validatingwebhookconfigurations/sre-clusterrolebindings-validation with fingerprint=3c90dce9692a06c150562a2522191a1af46b41a7a724941a468f2c9043d008c8 I0410 16:38:17.217082 1 recorder.go:75] Recording config/validatingwebhookconfigurations/sre-clusterroles-validation with fingerprint=6e89a97b7137e23d7ea3d5fa3b6717c8dc2ef3516e4f4a27349f3b423403fb7a I0410 16:38:17.217120 1 recorder.go:75] Recording config/validatingwebhookconfigurations/sre-ingress-config-validation with fingerprint=6a6a6324c0592617e79b43bbf8b0045d87584819731142dcfc4c8d47095ebbff I0410 16:38:17.217158 1 recorder.go:75] Recording config/validatingwebhookconfigurations/sre-network-operator-validation with fingerprint=05fc36098e1414b85304b041e0521892e3f59a31a7a6017fe195faaf547735a1 I0410 16:38:17.217215 1 recorder.go:75] Recording config/validatingwebhookconfigurations/sre-regular-user-validation with fingerprint=de8f26569fabaddf733a02b6e9c4e8b8c30f9c4a1e6113726c7e4fa59e04adfd I0410 16:38:17.217252 1 recorder.go:75] Recording config/validatingwebhookconfigurations/sre-scc-validation with fingerprint=80fdebc93159f53ecaee60f10db198eca837db22021ea18b5f44fe2aa2c3dc31 I0410 16:38:17.217293 1 recorder.go:75] Recording config/validatingwebhookconfigurations/sre-serviceaccount-validation with fingerprint=4c835b43231ecea35b3192215755621093581d8a3e053524ba1efb936f430e6a I0410 16:38:17.217331 1 recorder.go:75] Recording config/validatingwebhookconfigurations/sre-techpreviewnoupgrade-validation with fingerprint=bd4e1996aac01d9bfeae04afefef98718b1f3d6e6aac9f1a02cb20eb62a91859 I0410 16:38:17.217343 1 gather.go:177] gatherer "clusterconfig" function "validating_webhook_configurations" took 35.193072ms to process 11 records I0410 16:38:17.217366 1 recorder.go:75] Recording config/namespaces_with_overlapping_uids with fingerprint=4f53cda18c2baa0c0354bb5f9a3ecbe5ed12ab4d8e11ba873c2f11161202b945 I0410 16:38:17.217377 1 gather.go:177] gatherer "clusterconfig" function "overlapping_namespace_uids" took 37.238488ms to process 1 records I0410 16:38:17.217386 1 tasks_processing.go:74] worker 35 stopped. I0410 16:38:17.222874 1 tasks_processing.go:74] worker 44 stopped. I0410 16:38:17.222893 1 gather.go:177] gatherer "clusterconfig" function "monitoring_persistent_volumes" took 42.195709ms to process 0 records I0410 16:38:17.225494 1 tasks_processing.go:74] worker 51 stopped. I0410 16:38:17.228426 1 recorder.go:75] Recording config/pod/openshift-console-operator/console-operator-575cd97545-lcd4l with fingerprint=8147a967a4f3b428b7e5aa3e8a6a450805fddfea6caa62bac6e24e00c72ef39e I0410 16:38:17.228499 1 recorder.go:75] Recording config/running_containers with fingerprint=243bd8178597c2c214396b1a61366f5020b37d3bd46aa448532b52095f905b38 I0410 16:38:17.228512 1 gather.go:177] gatherer "clusterconfig" function "container_images" took 46.203359ms to process 2 records I0410 16:38:17.229941 1 configmapobserver.go:84] configmaps "insights-config" not found I0410 16:38:17.230036 1 tasks_processing.go:74] worker 22 stopped. I0410 16:38:17.230089 1 recorder.go:75] Recording config/proxy with fingerprint=821a67fade3bc539f9f2a82f69506ce9daad24763ed65ce0d0f0739b302cc08a I0410 16:38:17.230107 1 gather.go:177] gatherer "clusterconfig" function "proxies" took 49.275081ms to process 1 records I0410 16:38:17.230200 1 tasks_processing.go:74] worker 14 stopped. I0410 16:38:17.230212 1 gather.go:177] gatherer "clusterconfig" function "openstack_dataplanenodesets" took 48.986071ms to process 0 records I0410 16:38:17.230262 1 tasks_processing.go:74] worker 42 stopped. I0410 16:38:17.230443 1 recorder.go:75] Recording cluster-scoped-resources/rbac.authorization.k8s.io/clusterroles/admin with fingerprint=726b03e996b056ba96ec5410419e2195183f71baf2f6458128db91f99d54466a I0410 16:38:17.230536 1 recorder.go:75] Recording cluster-scoped-resources/rbac.authorization.k8s.io/clusterroles/edit with fingerprint=73d0ec867a13b7e429d8acf7220e2a08af5a9880eab314a3070bed43e28509a2 I0410 16:38:17.230547 1 gather.go:177] gatherer "clusterconfig" function "clusterroles" took 49.744743ms to process 2 records I0410 16:38:17.234454 1 tasks_processing.go:74] worker 24 stopped. I0410 16:38:17.234471 1 gather.go:177] gatherer "clusterconfig" function "storage_cluster" took 53.205441ms to process 0 records I0410 16:38:17.239324 1 tasks_processing.go:74] worker 5 stopped. I0410 16:38:17.239343 1 gather.go:177] gatherer "clusterconfig" function "sap_pods" took 58.197073ms to process 0 records I0410 16:38:17.239429 1 tasks_processing.go:74] worker 9 stopped. I0410 16:38:17.239595 1 recorder.go:75] Recording config/version with fingerprint=299d25e88e0d934afbac4fa9b367e4a0e882315af4ccf5c93e5210e3d6d91028 I0410 16:38:17.239610 1 recorder.go:75] Recording config/id with fingerprint=40dfbef5743e71c41db15878d966ce420a4b7744e5593fc42359a8cb1fadbdca I0410 16:38:17.239615 1 gather.go:177] gatherer "clusterconfig" function "version" took 58.450477ms to process 2 records I0410 16:38:17.239621 1 gather.go:177] gatherer "clusterconfig" function "sap_datahubs" took 57.612711ms to process 0 records I0410 16:38:17.239625 1 gather.go:177] gatherer "clusterconfig" function "openshift_logging" took 57.500081ms to process 0 records I0410 16:38:17.239630 1 tasks_processing.go:74] worker 60 stopped. I0410 16:38:17.239634 1 tasks_processing.go:74] worker 23 stopped. I0410 16:38:17.239669 1 tasks_processing.go:74] worker 16 stopped. I0410 16:38:17.239678 1 gather.go:177] gatherer "clusterconfig" function "openstack_dataplanedeployments" took 57.922863ms to process 0 records I0410 16:38:17.239740 1 tasks_processing.go:74] worker 56 stopped. I0410 16:38:17.239754 1 gather.go:177] gatherer "clusterconfig" function "number_of_pods_and_netnamespaces_with_sdn_annotations" took 60.434616ms to process 0 records I0410 16:38:17.245288 1 cluster_transfer.go:83] checking the availability of cluster transfer. Next check is in 12h0m0s I0410 16:38:17.245353 1 sca.go:136] Pulling SCA certificates from https://api.openshift.com/api/accounts_mgmt/v1/entitlement_certificates. Next check is in 8h0m0s W0410 16:38:17.245432 1 operator.go:288] started I0410 16:38:17.245451 1 base_controller.go:76] Waiting for caches to sync for LoggingSyncer I0410 16:38:17.245808 1 tasks_processing.go:74] worker 29 stopped. I0410 16:38:17.245822 1 gather.go:177] gatherer "clusterconfig" function "certificate_signing_requests" took 64.591909ms to process 0 records I0410 16:38:17.246894 1 tasks_processing.go:74] worker 4 stopped. I0410 16:38:17.246907 1 gather.go:177] gatherer "clusterconfig" function "machine_config_pools" took 65.79501ms to process 0 records I0410 16:38:17.247721 1 tasks_processing.go:74] worker 18 stopped. I0410 16:38:17.247942 1 recorder.go:75] Recording config/oauth with fingerprint=928a67ad08f89123b90b8999fcf6cf9ce18beaa1100673040165095d9e8c4595 I0410 16:38:17.247957 1 gather.go:177] gatherer "clusterconfig" function "oauths" took 66.486606ms to process 1 records I0410 16:38:17.248031 1 tasks_processing.go:74] worker 39 stopped. I0410 16:38:17.248580 1 recorder.go:75] Recording config/crd/volumesnapshots.snapshot.storage.k8s.io with fingerprint=ffdcfff955b881b77b60bcef4d3faecc9cbee95a219472339f6e5ef7c51e2723 I0410 16:38:17.248845 1 recorder.go:75] Recording config/crd/volumesnapshotcontents.snapshot.storage.k8s.io with fingerprint=cb7ac99778a29b2d0ebd51d55ab259f05bc5d02d37c27cb05a1060bd72f54690 I0410 16:38:17.248859 1 gather.go:177] gatherer "clusterconfig" function "crds" took 67.929026ms to process 2 records I0410 16:38:17.248942 1 tasks_processing.go:74] worker 28 stopped. I0410 16:38:17.249155 1 recorder.go:75] Recording config/clusteroperator/imageregistry.operator.openshift.io/config/cluster with fingerprint=3db087f3d84edb5d3123d3613d1bdb4d5eb91ffa70f8312d36bbe75d32c636ea I0410 16:38:17.249166 1 gather.go:177] gatherer "clusterconfig" function "image_registries" took 67.636433ms to process 1 records I0410 16:38:17.250165 1 tasks_processing.go:74] worker 2 stopped. I0410 16:38:17.250297 1 recorder.go:75] Recording config/mutatingwebhookconfigurations/aws-pod-identity with fingerprint=4923ee2595b6e5c9ec2be87c07a69f8a270db5b6b789f801297afe4307c4a17a I0410 16:38:17.250329 1 recorder.go:75] Recording config/mutatingwebhookconfigurations/sre-podimagespec-mutation with fingerprint=1c70a563d99b8a7eaa8a10804fe873eb31288afb6eebcd90d5ca45c9537d5ea3 I0410 16:38:17.250351 1 recorder.go:75] Recording config/mutatingwebhookconfigurations/sre-service-mutation with fingerprint=8892cdb92853ed055e014dba9399da1916dd5a36bd3b1ee3e51ba6f842fa8b6d I0410 16:38:17.250359 1 gather.go:177] gatherer "clusterconfig" function "mutating_webhook_configurations" took 69.153402ms to process 3 records W0410 16:38:17.251654 1 gather_dvo_metrics.go:210] Failed to read the DVO metrics. Trying again. I0410 16:38:17.253681 1 controller.go:212] Source periodic-conditional *controllerstatus.Simple is not ready I0410 16:38:17.253740 1 controller.go:212] Source periodic-workloads *controllerstatus.Simple is not ready I0410 16:38:17.253751 1 controller.go:212] Source scaController *sca.Controller is not ready I0410 16:38:17.253757 1 controller.go:212] Source clusterTransferController *clustertransfer.Controller is not ready I0410 16:38:17.253761 1 controller.go:212] Source periodic-clusterconfig *controllerstatus.Simple is not ready I0410 16:38:17.253800 1 controller.go:489] The operator is still being initialized I0410 16:38:17.253835 1 controller.go:512] The operator is healthy I0410 16:38:17.256337 1 prometheus_rules.go:88] Prometheus rules successfully created I0410 16:38:17.257481 1 tasks_processing.go:74] worker 7 stopped. I0410 16:38:17.257501 1 gather.go:177] gatherer "clusterconfig" function "aggregated_monitoring_cr_names" took 76.212596ms to process 0 records E0410 16:38:17.260606 1 cluster_transfer.go:95] failed to pull cluster transfer: unable to retrieve cluster transfer data from https://api.openshift.com/api/accounts_mgmt/v1/cluster_transfers/: Get "https://api.openshift.com/api/accounts_mgmt/v1/cluster_transfers/?search=cluster_uuid+is+%2759f18ac5-46e1-44b9-b3e4-a5a00e24d247%27+and+status+is+%27accepted%27": dial tcp: lookup api.openshift.com on 172.30.0.10:53: read udp 10.128.0.10:46055->172.30.0.10:53: read: connection refused I0410 16:38:17.260619 1 controllerstatus.go:80] name=clusterTransferController healthy=true reason=Disconnected message=failed to pull cluster transfer: unable to retrieve cluster transfer data from https://api.openshift.com/api/accounts_mgmt/v1/cluster_transfers/: Get "https://api.openshift.com/api/accounts_mgmt/v1/cluster_transfers/?search=cluster_uuid+is+%2759f18ac5-46e1-44b9-b3e4-a5a00e24d247%27+and+status+is+%27accepted%27": dial tcp: lookup api.openshift.com on 172.30.0.10:53: read udp 10.128.0.10:46055->172.30.0.10:53: read: connection refused I0410 16:38:17.266587 1 tasks_processing.go:74] worker 17 stopped. E0410 16:38:17.266613 1 gather.go:140] gatherer "clusterconfig" function "config_maps" failed with the error: configmaps "cluster-monitoring-config" not found E0410 16:38:17.266632 1 gather.go:140] gatherer "clusterconfig" function "config_maps" failed with the error: configmaps "gateway-mode-config" not found E0410 16:38:17.266640 1 gather.go:140] gatherer "clusterconfig" function "config_maps" failed with the error: configmaps "insights-config" not found I0410 16:38:17.266657 1 recorder.go:75] Recording config/configmaps/openshift-config/installer-images/images.json with fingerprint=26b6661162b099a0f5a279859b4f46c867929a79d9a4a41fde4be4e6fe138018 I0410 16:38:17.266713 1 recorder.go:75] Recording config/configmaps/openshift-config/kube-root-ca.crt/ca.crt with fingerprint=d476c7d3f5b104863f08f481b1264dcc68cc272ecefb0ecb709b18a6afab034d I0410 16:38:17.266727 1 recorder.go:75] Recording config/configmaps/openshift-config/openshift-install/version with fingerprint=c93090eb0d2a4736885abeb79c91680cfd01fda46464f83456b085d4dc8239f0 I0410 16:38:17.266737 1 recorder.go:75] Recording config/configmaps/openshift-config/openshift-install/invoker with fingerprint=76b482f683cd3ef9da02debac5b26080a5aeb06ff768ee5c21117514dff29d8a I0410 16:38:17.266745 1 recorder.go:75] Recording config/configmaps/openshift-config/openshift-service-ca.crt/service-ca.crt with fingerprint=e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855 I0410 16:38:17.266803 1 recorder.go:75] Recording config/configmaps/openshift-config/rosa-brand-logo/rosa-brand-logo.svg with fingerprint=6ed8ca4dd7a8eee7249182bc006e9649ce84d76c551ddfaaa33e55d8c4cc1ed0 I0410 16:38:17.266821 1 recorder.go:75] Recording config/configmaps/kube-system/cluster-config-v1/install-config with fingerprint=ab3811c6b83fd7b8e920094cfa3080d1b4ee3c35ec4c8379437b21d27bd6608d I0410 16:38:17.266831 1 gather.go:177] gatherer "clusterconfig" function "config_maps" took 85.422988ms to process 7 records I0410 16:38:17.272578 1 tasks_processing.go:74] worker 31 stopped. I0410 16:38:17.272857 1 recorder.go:75] Recording config/node/ip-10-0-1-37.ec2.internal with fingerprint=7049474d9acbc29edb9e1a5d7a0d592071d80c162e211a039efa66577a5bd113 I0410 16:38:17.272874 1 gather.go:177] gatherer "clusterconfig" function "nodes" took 91.657835ms to process 1 records E0410 16:38:17.272882 1 gather.go:140] gatherer "clusterconfig" function "ingress_certificates" failed with the error: failed to get secret 'router-certs-default' in namespace 'openshift-ingress': secrets "router-certs-default" not found E0410 16:38:17.272886 1 gather.go:140] gatherer "clusterconfig" function "ingress_certificates" failed with the error: failed to get secret '2pjda9copft31brcj754brber09es9ns-primary-cert-bundle-secret' in namespace 'openshift-ingress-operator': secrets "2pjda9copft31brcj754brber09es9ns-primary-cert-bundle-secret" not found I0410 16:38:17.272922 1 recorder.go:75] Recording aggregated/ingress_controllers_certs with fingerprint=3dab80850d98200402df6122b04374efe46756ab7a60fc111f37662b632b58cc I0410 16:38:17.272931 1 gather.go:177] gatherer "clusterconfig" function "ingress_certificates" took 91.661719ms to process 1 records I0410 16:38:17.272938 1 tasks_processing.go:74] worker 6 stopped. I0410 16:38:17.274792 1 tasks_processing.go:74] worker 12 stopped. I0410 16:38:17.274816 1 recorder.go:75] Recording aggregated/unused_machine_configs_count with fingerprint=4bfc9fa984e5dfcd45848faaf05269de7619bf42edf9f781751af5ee05c1a499 I0410 16:38:17.274827 1 gather.go:177] gatherer "clusterconfig" function "machine_configs" took 93.890947ms to process 1 records I0410 16:38:17.280730 1 requests.go:205] Asking for SCA certificate with "{"arch": ["x86_64"]}" payload W0410 16:38:17.284176 1 sca.go:161] Failed to pull SCA certs from https://api.openshift.com/api/accounts_mgmt/v1/entitlement_certificates: unable to retrieve SCA certs data from https://api.openshift.com/api/accounts_mgmt/v1/entitlement_certificates: Post "https://api.openshift.com/api/accounts_mgmt/v1/entitlement_certificates": dial tcp: lookup api.openshift.com on 172.30.0.10:53: read udp 10.128.0.10:60452->172.30.0.10:53: read: connection refused I0410 16:38:17.284189 1 controllerstatus.go:80] name=scaController healthy=true reason=NonHTTPError message=Failed to pull SCA certs from https://api.openshift.com/api/accounts_mgmt/v1/entitlement_certificates: unable to retrieve SCA certs data from https://api.openshift.com/api/accounts_mgmt/v1/entitlement_certificates: Post "https://api.openshift.com/api/accounts_mgmt/v1/entitlement_certificates": dial tcp: lookup api.openshift.com on 172.30.0.10:53: read udp 10.128.0.10:60452->172.30.0.10:53: read: connection refused I0410 16:38:17.345649 1 base_controller.go:82] Caches are synced for LoggingSyncer I0410 16:38:17.345664 1 base_controller.go:119] Starting #1 worker of LoggingSyncer controller ... W0410 16:38:18.251582 1 gather_dvo_metrics.go:210] Failed to read the DVO metrics. Trying again. I0410 16:38:18.423095 1 gather_cluster_operator_pods_and_events.go:121] Found 12 pods with 14 containers I0410 16:38:18.423110 1 gather_cluster_operator_pods_and_events.go:235] Maximum buffer size: 1797558 bytes I0410 16:38:18.424153 1 gather_cluster_operator_pods_and_events.go:365] Fetching logs for dns container dns-default-s28xc pod in namespace openshift-dns (previous: false). I0410 16:38:18.648117 1 gather_cluster_operators.go:184] Unable to get configs.samples.operator.openshift.io resource due to: configs.samples.operator.openshift.io "cluster" not found I0410 16:38:18.649376 1 gather_cluster_operator_pods_and_events.go:408] Failed to fetch log for dns-default-s28xc pod in namespace openshift-dns for failing operator dns (previous: false): "container \"dns\" in pod \"dns-default-s28xc\" is waiting to start: ContainerCreating" I0410 16:38:18.649389 1 gather_cluster_operator_pods_and_events.go:280] Error: "container \"dns\" in pod \"dns-default-s28xc\" is waiting to start: ContainerCreating" I0410 16:38:18.649398 1 gather_cluster_operator_pods_and_events.go:365] Fetching logs for kube-rbac-proxy container dns-default-s28xc pod in namespace openshift-dns (previous: false). I0410 16:38:18.826611 1 gather_cluster_operator_pods_and_events.go:408] Failed to fetch log for dns-default-s28xc pod in namespace openshift-dns for failing operator kube-rbac-proxy (previous: false): "container \"kube-rbac-proxy\" in pod \"dns-default-s28xc\" is waiting to start: ContainerCreating" I0410 16:38:18.826630 1 gather_cluster_operator_pods_and_events.go:280] Error: "container \"kube-rbac-proxy\" in pod \"dns-default-s28xc\" is waiting to start: ContainerCreating" I0410 16:38:18.826642 1 gather_cluster_operator_pods_and_events.go:365] Fetching logs for dns-node-resolver container node-resolver-z56kk pod in namespace openshift-dns (previous: false). I0410 16:38:19.057935 1 gather_cluster_operator_pods_and_events.go:280] Error: "log buffer is empty" I0410 16:38:19.057954 1 gather_cluster_operator_pods_and_events.go:365] Fetching logs for registry container image-registry-6fbc8dfdb-4lwcn pod in namespace openshift-image-registry (previous: false). I0410 16:38:19.222133 1 gather_cluster_operator_pods_and_events.go:280] Error: "log buffer is empty" I0410 16:38:19.222157 1 gather_cluster_operator_pods_and_events.go:365] Fetching logs for registry container image-registry-6fbc8dfdb-w2sls pod in namespace openshift-image-registry (previous: false). I0410 16:38:19.251269 1 tasks_processing.go:74] worker 59 stopped. I0410 16:38:19.251337 1 recorder.go:75] Recording config/clusteroperator/console with fingerprint=c9e5cc2630cb9925d74549454005d3ac57f4a51f933efb77517624a6e5ddfbaf I0410 16:38:19.251383 1 recorder.go:75] Recording config/clusteroperator/csi-snapshot-controller with fingerprint=6022cbf432629eaef5a0fc103788b8cf614777965f31672d42443ff39e2708d6 I0410 16:38:19.251438 1 recorder.go:75] Recording config/clusteroperator/operator.openshift.io/csisnapshotcontroller/cluster with fingerprint=5adc514f4b63e2f1ecc68bf6f9c0af70c5eea04522a49524e102721b1c41f80e I0410 16:38:19.251468 1 recorder.go:75] Recording config/clusteroperator/dns with fingerprint=4dbf3078528f08ee2390533009128751fcfe8407f8132a73055f28fe6b74b877 I0410 16:38:19.251486 1 recorder.go:75] Recording config/clusteroperator/operator.openshift.io/dns/default with fingerprint=9e7b4ce029030d3d8c3b49af92c556acdcc415000b40d3f969dbdc42c432b47f I0410 16:38:19.251510 1 recorder.go:75] Recording config/clusteroperator/image-registry with fingerprint=1ea7945fc36c272b97da6b8e3af287a81090fd78a3d748595a2e4f9bf209a8e1 I0410 16:38:19.251548 1 recorder.go:75] Recording config/clusteroperator/ingress with fingerprint=2d60e5c615623ce7b8f6e66965f5df6b56428942e319bdd1d6d27727e87c4ec7 I0410 16:38:19.251591 1 recorder.go:75] Recording config/clusteroperator/operator.openshift.io/ingresscontroller/openshift-ingress-operator/default with fingerprint=8b1ca5b917efefa23a576ce77e0a329ee153ed43cb1c01d6b1d964b58047c577 I0410 16:38:19.251605 1 recorder.go:75] Recording config/clusteroperator/insights with fingerprint=d4e25547d542908d76108112f4ab556ae54d0d8a1eb8d87c7c22c7a366e09d44 I0410 16:38:19.251621 1 recorder.go:75] Recording config/clusteroperator/kube-apiserver with fingerprint=19db953efaead63eddc7b74a97f21b8f508298a1a8acbf41d63d1a9023ac2c43 I0410 16:38:19.251630 1 recorder.go:75] Recording config/clusteroperator/operator.openshift.io/kubeapiserver/cluster with fingerprint=51503bf0b784fcf65ea46bcaf1f72ac1a5c4d5dc211934f18f27871efed05762 I0410 16:38:19.251657 1 recorder.go:75] Recording config/clusteroperator/kube-controller-manager with fingerprint=159d7d234262987e4329396b393bbb79c8fee3e135b44e036dc944be3a81d06d I0410 16:38:19.251673 1 recorder.go:75] Recording config/clusteroperator/operator.openshift.io/kubecontrollermanager/cluster with fingerprint=ce90c0d4f367d7da085074268031798382ae7c54fdcb0a21f15a4818fe308c11 I0410 16:38:19.251702 1 recorder.go:75] Recording config/clusteroperator/kube-scheduler with fingerprint=8e93d26f41890a5ae6a549ee4eff51d70c18e28e77d5ab264a69d10b2e84d318 I0410 16:38:19.251712 1 recorder.go:75] Recording config/clusteroperator/operator.openshift.io/kubescheduler/cluster with fingerprint=f2940fb9fd20c19951dfc295eb363b7fba0c505f5ae61f01967a063099e6b60a I0410 16:38:19.251733 1 recorder.go:75] Recording config/clusteroperator/kube-storage-version-migrator with fingerprint=afc7ed8dd54a1c39e6cc813398708700d2d916d95814a062523453bd75b279f5 I0410 16:38:19.251742 1 recorder.go:75] Recording config/clusteroperator/operator.openshift.io/kubestorageversionmigrator/cluster with fingerprint=9351181aa7e6ada41ef581ab31e13516c6b934cc95710154bafb2eb222cb58db I0410 16:38:19.251757 1 recorder.go:75] Recording config/clusteroperator/monitoring with fingerprint=faf5b95b9b5faf7c3b16bcc79334caee4459bf22bdc6436c3c972bed3d874704 I0410 16:38:19.251876 1 recorder.go:75] Recording config/clusteroperator/network with fingerprint=882deee8a7e604403579561b906584497dcfda57035d272e3a9af105930a4a2b I0410 16:38:19.251886 1 recorder.go:75] Recording config/clusteroperator/network.operator.openshift.io/operatorpki/openshift-ovn-kubernetes/ovn with fingerprint=626a89d20e0deaed5b6dfb533acfe65f4bb1618bd200a703b62e60c5d16d94ab I0410 16:38:19.251893 1 recorder.go:75] Recording config/clusteroperator/network.operator.openshift.io/operatorpki/openshift-ovn-kubernetes/signer with fingerprint=90410b16914712b85b3c4578716ad8c0ae072e688f4cd1e022bf76f20da3506d I0410 16:38:19.251914 1 recorder.go:75] Recording config/clusteroperator/operator.openshift.io/network/cluster with fingerprint=a93d15eaecb455a0e40ecb2826eeecc1533899204ddd3c3921d15ab70af7ae75 I0410 16:38:19.251934 1 recorder.go:75] Recording config/clusteroperator/node-tuning with fingerprint=35f49642fa9658107ade6e673bd118e79e3a644acfd402a28bf6c8fd89b64f26 I0410 16:38:19.251971 1 recorder.go:75] Recording config/clusteroperator/openshift-apiserver with fingerprint=a7c67d40835eb34b27901bd89d18f0668e81937f8d2bf1ce3235b95a52da5e87 I0410 16:38:19.251988 1 recorder.go:75] Recording config/clusteroperator/operator.openshift.io/openshiftapiserver/cluster with fingerprint=e712e6cf27339b441e4ed1f4cde91dbde7e952698ba93407e4457db63a4a4c76 I0410 16:38:19.252012 1 recorder.go:75] Recording config/clusteroperator/openshift-controller-manager with fingerprint=ce4e65e5b91bb47d1bec93ab19d78f88884b5fb8153b2ac5beca12febc57bc4e I0410 16:38:19.252021 1 recorder.go:75] Recording config/clusteroperator/operator.openshift.io/openshiftcontrollermanager/cluster with fingerprint=d71a0f4672f9b45d9fc8293bf1687afc650fd28d32e2e30de27523fe7b4eadf7 I0410 16:38:19.252034 1 recorder.go:75] Recording config/clusteroperator/openshift-samples with fingerprint=f6e95f83406f84310c351150c42af6b05dba56ea8e260afb01563ff3b9aa57d9 I0410 16:38:19.252051 1 recorder.go:75] Recording config/clusteroperator/operator-lifecycle-manager with fingerprint=56493daa3a42edaa27d6c5a3c7b3bdf1d27549a7ee035a03f6baf4b738f92f94 I0410 16:38:19.252067 1 recorder.go:75] Recording config/clusteroperator/operator-lifecycle-manager-catalog with fingerprint=0c8d19d525373fd20e450aa66622b3f7b1b761d65544fab72ca39d07ceb845c3 I0410 16:38:19.252082 1 recorder.go:75] Recording config/clusteroperator/operator-lifecycle-manager-packageserver with fingerprint=dd0f843958f071dca40c6920fa5eeea88f23430b2e3c76afe10fa6e8a1368ada W0410 16:38:19.252104 1 gather_dvo_metrics.go:210] Failed to read the DVO metrics. Trying again. I0410 16:38:19.252134 1 recorder.go:75] Recording config/clusteroperator/service-ca with fingerprint=ce1a861a48051b55d6ce3d304ea94a320d93244ae3df63067bcc68e0f65c8b33 I0410 16:38:19.252144 1 recorder.go:75] Recording config/clusteroperator/operator.openshift.io/serviceca/cluster with fingerprint=812f7edc2cdb30e61e7f2b29454357a40b1a507a4b0c2b7729193b67f0e3b4aa I0410 16:38:19.252168 1 recorder.go:75] Recording config/clusteroperator/storage with fingerprint=231848c1c97550ff0fb314c34c4c15635120c21c10493150952cc80c42c4ed62 I0410 16:38:19.252185 1 recorder.go:75] Recording config/clusteroperator/operator.openshift.io/clustercsidriver/ebs.csi.aws.com with fingerprint=510064d6f6bcced87ab5bd2ddaff3d0edd7f93f4a4f7af2641f29fc53ffab21e I0410 16:38:19.252193 1 recorder.go:75] Recording config/clusteroperator/operator.openshift.io/storage/cluster with fingerprint=8e480f8c1ce1b39baac42d8ec780c57c2592929ae0c801b61ffad49ba13f33ad I0410 16:38:19.252199 1 gather.go:177] gatherer "clusterconfig" function "operators" took 2.070599816s to process 36 records I0410 16:38:19.427283 1 gather_cluster_operator_pods_and_events.go:408] Failed to fetch log for image-registry-6fbc8dfdb-w2sls pod in namespace openshift-image-registry for failing operator registry (previous: false): "container \"registry\" in pod \"image-registry-6fbc8dfdb-w2sls\" is waiting to start: ContainerCreating" I0410 16:38:19.427300 1 gather_cluster_operator_pods_and_events.go:280] Error: "container \"registry\" in pod \"image-registry-6fbc8dfdb-w2sls\" is waiting to start: ContainerCreating" I0410 16:38:19.427312 1 gather_cluster_operator_pods_and_events.go:365] Fetching logs for registry container image-registry-79bc77bf97-gbgbh pod in namespace openshift-image-registry (previous: false). I0410 16:38:19.622967 1 gather_cluster_operator_pods_and_events.go:280] Error: "log buffer is empty" I0410 16:38:19.622986 1 gather_cluster_operator_pods_and_events.go:365] Fetching logs for node-ca container node-ca-qt4ct pod in namespace openshift-image-registry (previous: false). I0410 16:38:19.828119 1 gather_cluster_operator_pods_and_events.go:280] Error: "log buffer is empty" I0410 16:38:19.828139 1 gather_cluster_operator_pods_and_events.go:365] Fetching logs for router container router-default-86b8698cdb-jn4jq pod in namespace openshift-ingress (previous: false). I0410 16:38:20.040976 1 gather_cluster_operator_pods_and_events.go:408] Failed to fetch log for router-default-86b8698cdb-jn4jq pod in namespace openshift-ingress for failing operator router (previous: false): "container \"router\" in pod \"router-default-86b8698cdb-jn4jq\" is waiting to start: ContainerCreating" I0410 16:38:20.040996 1 gather_cluster_operator_pods_and_events.go:280] Error: "container \"router\" in pod \"router-default-86b8698cdb-jn4jq\" is waiting to start: ContainerCreating" I0410 16:38:20.041008 1 gather_cluster_operator_pods_and_events.go:365] Fetching logs for router container router-default-86b8698cdb-scmd8 pod in namespace openshift-ingress (previous: false). I0410 16:38:20.226516 1 gather_cluster_operator_pods_and_events.go:280] Error: "log buffer is empty" I0410 16:38:20.226535 1 gather_cluster_operator_pods_and_events.go:365] Fetching logs for router container router-default-b7c798989-r74cd pod in namespace openshift-ingress (previous: false). W0410 16:38:20.251334 1 gather_dvo_metrics.go:210] Failed to read the DVO metrics. Trying again. I0410 16:38:20.428820 1 gather_cluster_operator_pods_and_events.go:408] Failed to fetch log for router-default-b7c798989-r74cd pod in namespace openshift-ingress for failing operator router (previous: false): "container \"router\" in pod \"router-default-b7c798989-r74cd\" is waiting to start: ContainerCreating" I0410 16:38:20.428844 1 gather_cluster_operator_pods_and_events.go:280] Error: "container \"router\" in pod \"router-default-b7c798989-r74cd\" is waiting to start: ContainerCreating" I0410 16:38:20.428857 1 gather_cluster_operator_pods_and_events.go:365] Fetching logs for serve-healthcheck-canary container ingress-canary-mj4fr pod in namespace openshift-ingress-canary (previous: false). I0410 16:38:20.628981 1 gather_cluster_operator_pods_and_events.go:408] Failed to fetch log for ingress-canary-mj4fr pod in namespace openshift-ingress-canary for failing operator serve-healthcheck-canary (previous: false): "container \"serve-healthcheck-canary\" in pod \"ingress-canary-mj4fr\" is waiting to start: ContainerCreating" I0410 16:38:20.629002 1 gather_cluster_operator_pods_and_events.go:280] Error: "container \"serve-healthcheck-canary\" in pod \"ingress-canary-mj4fr\" is waiting to start: ContainerCreating" I0410 16:38:20.629013 1 gather_cluster_operator_pods_and_events.go:365] Fetching logs for migrator container migrator-7d5f866c57-jpdgg pod in namespace openshift-kube-storage-version-migrator (previous: false). I0410 16:38:20.829721 1 gather_cluster_operator_pods_and_events.go:365] Fetching logs for graceful-termination container migrator-7d5f866c57-jpdgg pod in namespace openshift-kube-storage-version-migrator (previous: false). I0410 16:38:21.028661 1 gather_cluster_operator_pods_and_events.go:365] Fetching logs for kube-storage-version-migrator-operator container kube-storage-version-migrator-operator-74848b4cb9-kk5rd pod in namespace openshift-kube-storage-version-migrator-operator (previous: false). I0410 16:38:21.230901 1 tasks_processing.go:74] worker 26 stopped. I0410 16:38:21.231012 1 recorder.go:75] Recording events/openshift-dns-operator with fingerprint=e0421effe9bd23debdf33c2bd94306d083fe8358f02e98a04cea494239595fa2 I0410 16:38:21.231048 1 recorder.go:75] Recording events/openshift-dns with fingerprint=95f89d8d1e31537e1cb1d12c67433a6eb18ce0cf56ec60421d4ec6bc5fc61a75 I0410 16:38:21.231120 1 recorder.go:75] Recording events/openshift-image-registry with fingerprint=e87b64cbc52f098826f4401fc0284e934fc7bc635b54cdfb71d4300a1584d231 I0410 16:38:21.231157 1 recorder.go:75] Recording events/openshift-ingress-operator with fingerprint=f70c2024640af01897633a1e1d73c75336d6fabd77fd5bb86039c6adddcca187 I0410 16:38:21.231210 1 recorder.go:75] Recording events/openshift-ingress with fingerprint=bb265f92fd3df7fd99a581b2863fea8efdcbe76042c6b74e085b471141580f17 I0410 16:38:21.231229 1 recorder.go:75] Recording events/openshift-ingress-canary with fingerprint=eff785438dfaea7a4782d9cb4ffa8e14aab113f3cde1e03bf603fd3c2d517ddb I0410 16:38:21.231252 1 recorder.go:75] Recording events/openshift-kube-storage-version-migrator with fingerprint=b3f0bcbbb140cb5a5e658713d1df1e6e8fa8227c91ad151776c135ce6ead36a2 I0410 16:38:21.231321 1 recorder.go:75] Recording events/openshift-kube-storage-version-migrator-operator with fingerprint=588cafe677526caaf30cd220bbd9aa905bb2303ed89a34ad67af50c7548266a6 I0410 16:38:21.231344 1 recorder.go:75] Recording config/pod/openshift-kube-storage-version-migrator/logs/migrator-7d5f866c57-jpdgg/migrator_current.log with fingerprint=e08f516f247ed3b6e70fe148d13a3fc93503f1a1a94ae65ebce6f710a7ce5e93 I0410 16:38:21.231350 1 recorder.go:75] Recording config/pod/openshift-kube-storage-version-migrator/logs/migrator-7d5f866c57-jpdgg/graceful-termination_current.log with fingerprint=dd4bc7020a2d4a6bb1359d745158b1ed0cdb37099d1b40d44102bc966fcc700f I0410 16:38:21.231440 1 recorder.go:75] Recording config/pod/openshift-kube-storage-version-migrator-operator/logs/kube-storage-version-migrator-operator-74848b4cb9-kk5rd/kube-storage-version-migrator-operator_current.log with fingerprint=bf14947bdd1b45fb0a1490e7dd5fefc7ae648164bce135f843e0942422ecccb6 I0410 16:38:21.231449 1 gather.go:177] gatherer "clusterconfig" function "operators_pods_and_events" took 4.051297271s to process 11 records W0410 16:38:21.251753 1 gather_dvo_metrics.go:210] Failed to read the DVO metrics. Trying again. W0410 16:38:22.248535 1 gather_dvo_metrics.go:117] Unable to read metrics from endpoint "http://deployment-validation-operator-metrics.openshift-deployment-validation-operator.svc:8383": DVO metrics service was not available within the 5s timeout: context deadline exceeded I0410 16:38:22.248566 1 tasks_processing.go:74] worker 11 stopped. E0410 16:38:22.248579 1 gather.go:140] gatherer "clusterconfig" function "dvo_metrics" failed with the error: DVO metrics service was not available within the 5s timeout: context deadline exceeded I0410 16:38:22.248593 1 recorder.go:75] Recording config/dvo_metrics with fingerprint=e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855 W0410 16:38:22.248609 1 gather.go:155] issue recording gatherer "clusterconfig" function "dvo_metrics" result "config/dvo_metrics" because of the warning: warning: the record with the same fingerprint "e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855" was already recorded at path "config/configmaps/openshift-config/openshift-service-ca.crt/service-ca.crt", recording another one with a different path "config/dvo_metrics" I0410 16:38:22.248621 1 gather.go:177] gatherer "clusterconfig" function "dvo_metrics" took 5.067305219s to process 1 records I0410 16:38:29.821891 1 tasks_processing.go:74] worker 1 stopped. I0410 16:38:29.821933 1 recorder.go:75] Recording config/installplans with fingerprint=7b887df561a3a9e6ef0dc672845aa5d56e348505006b7496d3a2f83892b0c95b I0410 16:38:29.821946 1 gather.go:177] gatherer "clusterconfig" function "install_plans" took 12.642570267s to process 1 records I0410 16:38:29.936821 1 configmapobserver.go:84] configmaps "insights-config" not found I0410 16:38:30.070870 1 configmapobserver.go:84] configmaps "insights-config" not found I0410 16:38:30.137892 1 configmapobserver.go:84] configmaps "insights-config" not found I0410 16:38:30.591019 1 tasks_processing.go:74] worker 40 stopped. I0410 16:38:30.591286 1 recorder.go:75] Recording config/serviceaccounts with fingerprint=e03550ff5cb13d4cd718a41a9755571c983753e7ec8a789b8fb921884bbb0f64 I0410 16:38:30.591301 1 gather.go:177] gatherer "clusterconfig" function "service_accounts" took 13.411701034s to process 1 records E0410 16:38:30.591356 1 periodic.go:247] "Unhandled Error" err="clusterconfig failed after 13.412s with: function \"pod_network_connectivity_checks\" failed with an error, function \"machine_healthchecks\" failed with an error, function \"machines\" failed with an error, function \"support_secret\" failed with an error, function \"config_maps\" failed with an error, function \"ingress_certificates\" failed with an error, function \"dvo_metrics\" failed with an error" I0410 16:38:30.592441 1 controllerstatus.go:89] name=periodic-clusterconfig healthy=false reason=PeriodicGatherFailed message=Source clusterconfig could not be retrieved: function "pod_network_connectivity_checks" failed with an error, function "machine_healthchecks" failed with an error, function "machines" failed with an error, function "support_secret" failed with an error, function "config_maps" failed with an error, function "ingress_certificates" failed with an error, function "dvo_metrics" failed with an error I0410 16:38:30.592457 1 periodic.go:209] Running workloads gatherer I0410 16:38:30.592470 1 tasks_processing.go:45] number of workers: 2 I0410 16:38:30.592479 1 tasks_processing.go:69] worker 1 listening for tasks. I0410 16:38:30.592483 1 tasks_processing.go:71] worker 1 working on workload_info task. I0410 16:38:30.592491 1 tasks_processing.go:69] worker 0 listening for tasks. I0410 16:38:30.592510 1 tasks_processing.go:71] worker 0 working on helmchart_info task. I0410 16:38:30.612378 1 gather_workloads_info.go:278] Loaded pods in 0s, will wait 19s for image data I0410 16:38:30.618881 1 tasks_processing.go:74] worker 0 stopped. I0410 16:38:30.618900 1 gather.go:177] gatherer "workloads" function "helmchart_info" took 26.34789ms to process 0 records I0410 16:38:30.621069 1 gather_workloads_info.go:387] No image sha256:a498046d64605bcccee2440aa4f04a4602baaae263cf01d977ec5208e876b1fd (9ms) I0410 16:38:30.629966 1 gather_workloads_info.go:387] No image sha256:4556896f77307821531ef91b7b7faccb82b824ea695693b2989f597f0deca038 (9ms) I0410 16:38:30.637981 1 gather_workloads_info.go:387] No image sha256:5a95c19d82767e0235b4edb4a0536482c816904897aae1dc3eb255cb52b87a9f (8ms) I0410 16:38:30.646174 1 gather_workloads_info.go:387] No image sha256:a258c226562adb14e3a163a1940938526ee6a0928982a7667d85d9a7334ce639 (8ms) I0410 16:38:30.654016 1 gather_workloads_info.go:387] No image sha256:2904a78e2eb73fd6a9bb94c105c2a056831fb4113fbb7b0607c50adc9d879c9b (8ms) I0410 16:38:30.661878 1 gather_workloads_info.go:387] No image sha256:a0105d1eb62cf6ac9e5e2ef28d3e89bf6dc514bc594fc7090fe5a5ee18a09c87 (8ms) I0410 16:38:30.669739 1 gather_workloads_info.go:387] No image sha256:765f0d23b637f685f98a31bd47c131b03cf72a40761a3f9a9d6320faa3c33733 (8ms) I0410 16:38:30.677537 1 gather_workloads_info.go:387] No image sha256:695cf2f0cc07683c2a3ce1eaf3e56fe18abc6e2bac716f7d9843f5d173b9df52 (8ms) I0410 16:38:30.685476 1 gather_workloads_info.go:387] No image sha256:5f0b67cfbbc381243fb91ccc17345b56d05f4d717c667e8c644e5bf05633ba71 (8ms) I0410 16:38:30.692840 1 gather_workloads_info.go:387] No image sha256:943018739e3db1763c3184b460dbc409e058abbac76d57b9927faad317be85e4 (7ms) I0410 16:38:30.721514 1 gather_workloads_info.go:387] No image sha256:521712486e2c6e3c020dad6a1cb340db8e55665b69f7c208fab9cd9e965fd588 (29ms) I0410 16:38:30.821058 1 gather_workloads_info.go:387] No image sha256:653c666f842c13e0baae2e89a9b1efe0e2ef56f621ffb5b32005115d2a26ab8c (100ms) I0410 16:38:30.921678 1 gather_workloads_info.go:387] No image sha256:36b9e89c3cfcf1ab9ae500486e38afb6862cba48cb0b4d84a09508ab8f3d299f (101ms) I0410 16:38:31.023019 1 gather_workloads_info.go:387] No image sha256:03cf4cd7ef1518610c6c7b3ad27d1622d82e98e3dc6e3f8e5d0fceb5c8d3786e (101ms) I0410 16:38:31.122247 1 gather_workloads_info.go:387] No image sha256:1a2532940843248c57d52141185dd71fbc393ab28b65d48f682038632c1dbbad (99ms) I0410 16:38:31.221929 1 gather_workloads_info.go:387] No image sha256:91828234f107c068c8a4966d08370ae7b73e637651dbc6d92c18c4553402c22c (100ms) I0410 16:38:31.321887 1 gather_workloads_info.go:387] No image sha256:c15ca0c0ad60fe8757c2d5d1723fcdd7a1ed6c0251a90d22a7e6cae6811d01aa (100ms) I0410 16:38:31.421830 1 gather_workloads_info.go:387] No image sha256:ca1344cb64140188b7cae7bbc51fb751566c0b0c97d5e39b5850e628032c4a5e (100ms) I0410 16:38:31.522188 1 gather_workloads_info.go:387] No image sha256:7adc1eab05d6724c76ba751f6df816b08d6e70b78dee9eb94fa6fd9690542c98 (100ms) I0410 16:38:31.622265 1 gather_workloads_info.go:387] No image sha256:289816958633a763a72dbc44e1dad40466223164e7e253039514f0d974ea5d21 (100ms) I0410 16:38:31.720987 1 gather_workloads_info.go:387] No image sha256:b3909bf664c77097f75b3768830863d642eed3815dab2bfb4415c771ca2d5007 (99ms) I0410 16:38:31.820753 1 gather_workloads_info.go:387] No image sha256:2e57e192c3c1240fd935dcd55c8fde5e70e78bf81d6176c96edf21fafe59f8ba (100ms) I0410 16:38:31.921851 1 gather_workloads_info.go:387] No image sha256:7b31223098f08328f5ddea8e5b871dbbd5f5a61ec550e8956f66793c0c6031a9 (101ms) I0410 16:38:32.021846 1 gather_workloads_info.go:387] No image sha256:2e564f336c77116053f34d4201d364d8da04e789cfffa0ea422574c95f2d6404 (100ms) I0410 16:38:32.122186 1 gather_workloads_info.go:387] No image sha256:56a85660a445eced5c79a595a0eccf590087c5672d50f49d4c25ad52f9a44f04 (100ms) I0410 16:38:32.221480 1 gather_workloads_info.go:387] No image sha256:e84cb128d930bd1ab867cc89b7b7bf2b2c0e41105ab93b5381069945b3ee9c57 (99ms) I0410 16:38:32.321238 1 gather_workloads_info.go:387] No image sha256:a56211d075aa43cbb491f669a5b2e46ee023dc95b7d51dbac28f463948c5ad61 (100ms) I0410 16:38:32.420508 1 gather_workloads_info.go:387] No image sha256:0a99240166165eb5718e7516a43282fe32df9c7c5e809b31b58abe44e42ff94d (99ms) I0410 16:38:32.521153 1 gather_workloads_info.go:387] No image sha256:c940ea87e7d133d75ba0002ef00c0806825eed3db8094cdb260d1bac18127733 (101ms) I0410 16:38:32.624308 1 gather_workloads_info.go:387] No image sha256:47154813651033d59751fb655a384dbffb64dd26f10bd7f3be0c3128d0486356 (103ms) I0410 16:38:32.721241 1 gather_workloads_info.go:387] No image sha256:2598489729a4b258e4ecda4a06f6875133f2a10ced5c5241f8a57a8a05418e36 (97ms) I0410 16:38:32.721270 1 tasks_processing.go:74] worker 1 stopped. E0410 16:38:32.721290 1 gather.go:140] gatherer "workloads" function "workload_info" failed with the error: no running pods found for the insights-runtime-extractor statefulset I0410 16:38:32.721614 1 recorder.go:75] Recording config/workload_info with fingerprint=08ec1d92b54febe7abccb501315e6698892e086203b1f1a60071412a56a4ec2c I0410 16:38:32.721634 1 gather.go:177] gatherer "workloads" function "workload_info" took 2.128780403s to process 1 records E0410 16:38:32.721667 1 periodic.go:247] "Unhandled Error" err="workloads failed after 2.129s with: function \"workload_info\" failed with an error" I0410 16:38:32.722768 1 controllerstatus.go:89] name=periodic-workloads healthy=false reason=PeriodicGatherFailed message=Source workloads could not be retrieved: function "workload_info" failed with an error I0410 16:38:32.722781 1 periodic.go:209] Running conditional gatherer I0410 16:38:32.727789 1 requests.go:294] Making HTTP GET request at: https://console.redhat.com/api/gathering/v2/4.20.8/gathering_rules I0410 16:38:32.734242 1 conditional_gatherer.go:107] Get "https://console.redhat.com/api/gathering/v2/4.20.8/gathering_rules": dial tcp: lookup console.redhat.com on 172.30.0.10:53: read udp 10.128.0.10:49802->172.30.0.10:53: read: connection refused E0410 16:38:32.734509 1 conditional_gatherer.go:322] unable to update alerts cache: open /var/run/configmaps/service-ca-bundle/service-ca.crt: no such file or directory I0410 16:38:32.734567 1 conditional_gatherer.go:384] updating version cache for conditional gatherer I0410 16:38:32.739788 1 conditional_gatherer.go:392] cluster version is '4.20.8' E0410 16:38:32.739801 1 conditional_gatherer.go:209] error checking conditions for a gathering rule: alerts cache is missing E0410 16:38:32.739806 1 conditional_gatherer.go:209] error checking conditions for a gathering rule: alerts cache is missing E0410 16:38:32.739810 1 conditional_gatherer.go:209] error checking conditions for a gathering rule: alerts cache is missing E0410 16:38:32.739813 1 conditional_gatherer.go:209] error checking conditions for a gathering rule: alerts cache is missing E0410 16:38:32.739816 1 conditional_gatherer.go:209] error checking conditions for a gathering rule: alerts cache is missing E0410 16:38:32.739819 1 conditional_gatherer.go:209] error checking conditions for a gathering rule: alerts cache is missing E0410 16:38:32.739821 1 conditional_gatherer.go:209] error checking conditions for a gathering rule: alerts cache is missing E0410 16:38:32.739824 1 conditional_gatherer.go:209] error checking conditions for a gathering rule: alerts cache is missing E0410 16:38:32.739827 1 conditional_gatherer.go:209] error checking conditions for a gathering rule: alerts cache is missing I0410 16:38:32.739842 1 tasks_processing.go:45] number of workers: 3 I0410 16:38:32.739847 1 tasks_processing.go:69] worker 2 listening for tasks. I0410 16:38:32.739851 1 tasks_processing.go:71] worker 2 working on conditional_gatherer_rules task. I0410 16:38:32.739860 1 tasks_processing.go:69] worker 0 listening for tasks. I0410 16:38:32.739871 1 tasks_processing.go:69] worker 1 listening for tasks. I0410 16:38:32.739875 1 tasks_processing.go:71] worker 0 working on remote_configuration task. I0410 16:38:32.739878 1 tasks_processing.go:74] worker 1 stopped. I0410 16:38:32.739886 1 tasks_processing.go:71] worker 2 working on rapid_container_logs task. I0410 16:38:32.739924 1 recorder.go:75] Recording insights-operator/conditional-gatherer-rules with fingerprint=7034af97e7e41c22e4b775abdd4b9066c8ebb19da33eb7f69f39bfd2eb5f6406 I0410 16:38:32.739935 1 gather.go:177] gatherer "conditional" function "conditional_gatherer_rules" took 1.024µs to process 1 records I0410 16:38:32.739964 1 recorder.go:75] Recording insights-operator/remote-configuration with fingerprint=0394430c431eec4d48bb1811a90918e95161d2282c59af26f2473613cc0959db I0410 16:38:32.739971 1 gather.go:177] gatherer "conditional" function "remote_configuration" took 1.37µs to process 1 records I0410 16:38:32.739977 1 tasks_processing.go:74] worker 0 stopped. I0410 16:38:32.740154 1 tasks_processing.go:74] worker 2 stopped. I0410 16:38:32.740167 1 gather.go:177] gatherer "conditional" function "rapid_container_logs" took 257.182µs to process 0 records I0410 16:38:32.740185 1 controllerstatus.go:89] name=periodic-conditional healthy=false reason=NotAvailable message=Get "https://console.redhat.com/api/gathering/v2/4.20.8/gathering_rules": dial tcp: lookup console.redhat.com on 172.30.0.10:53: read udp 10.128.0.10:49802->172.30.0.10:53: read: connection refused I0410 16:38:32.740202 1 recorder.go:75] Recording insights-operator/remote-configuration.json with fingerprint=359de9c990c741675cec72fda96b5c3682221efdb4799f5eaa6e9805bcd3b5c1 I0410 16:38:32.760868 1 recorder.go:75] Recording insights-operator/gathers with fingerprint=c5bc97d70fae8a6f6c001ca904b3a5c49736cc1bfe3d648b6211d20a9ce913d0 I0410 16:38:32.760993 1 diskrecorder.go:70] Writing 104 records to /var/lib/insights-operator/insights-2026-04-10-163832.tar.gz I0410 16:38:32.767487 1 diskrecorder.go:51] Wrote 104 records to disk in 6ms I0410 16:38:32.767513 1 periodic.go:278] Gathering cluster info every 2h0m0s I0410 16:38:32.767529 1 periodic.go:279] Configuration is dataReporting: interval: 2h0m0s, uploadEndpoint: https://console.redhat.com/api/ingress/v1/upload, storagePath: /var/lib/insights-operator, downloadEndpoint: https://console.redhat.com/api/insights-results-aggregator/v2/cluster/%s/reports, conditionalGathererEndpoint: https://console.redhat.com/api/gathering/v2/%s/gathering_rules, obfuscation: [] sca: disabled: false, endpoint: https://api.openshift.com/api/accounts_mgmt/v1/entitlement_certificates, interval: 8h0m0s alerting: disabled: false clusterTransfer: endpoint: https://api.openshift.com/api/accounts_mgmt/v1/cluster_transfers/, interval: 12h0m0s proxy: httpProxy: , httpsProxy: , noProxy: I0410 16:38:41.808530 1 configmapobserver.go:84] configmaps "insights-config" not found I0410 16:39:31.516937 1 observer_polling.go:111] Observed file "/var/run/configmaps/service-ca-bundle/service-ca.crt" has been created (hash="f8e3ad604c04e67ff04f81774519ce4a072c72b7d626c913cfd98a30ffbf7c3b") W0410 16:39:31.516976 1 builder.go:160] Restart triggered because of file /var/run/configmaps/service-ca-bundle/service-ca.crt was created I0410 16:39:31.517029 1 observer_polling.go:111] Observed file "/var/run/secrets/serving-cert/tls.crt" has been created (hash="55c97b8b4b3f1388fa5f6fac1c165de2f08e1ca03b0fc6def2f07d012f95fd1c") I0410 16:39:31.517045 1 genericapiserver.go:548] "[graceful-termination] shutdown event" name="ShutdownInitiated" I0410 16:39:31.517069 1 observer_polling.go:111] Observed file "/var/run/secrets/serving-cert/tls.key" has been created (hash="846da692ae0d99b349197f5e4bf23a0787b686173bf59f97db79851d17f8a7e7") I0410 16:39:31.517080 1 periodic.go:170] Shutting down I0410 16:39:31.517093 1 base_controller.go:123] Shutting down worker of LoggingSyncer controller ... I0410 16:39:31.517109 1 simple_featuregate_reader.go:177] Shutting down feature-gate-detector I0410 16:39:31.517056 1 base_controller.go:181] Shutting down LoggingSyncer ... I0410 16:39:31.517128 1 base_controller.go:113] All LoggingSyncer workers have been terminated I0410 16:39:31.517133 1 genericapiserver.go:693] "[graceful-termination] pre-shutdown hooks completed" name="PreShutdownHooksStopped" I0410 16:39:31.517145 1 base_controller.go:181] Shutting down ConfigController ...