W0704 18:14:47.828900 1 cmd.go:257] Using insecure, self-signed certificates I0704 18:14:48.270314 1 start.go:138] Unable to read service ca bundle: open /var/run/configmaps/service-ca-bundle/service-ca.crt: no such file or directory I0704 18:14:48.270795 1 observer_polling.go:159] Starting file observer I0704 18:14:48.848395 1 operator.go:60] Starting insights-operator v0.0.0-master+$Format:%H$ I0704 18:14:48.848681 1 legacy_config.go:327] Current config: {"report":false,"storagePath":"/var/lib/insights-operator","interval":"2h","endpoint":"https://console.redhat.com/api/ingress/v1/upload","conditionalGathererEndpoint":"https://console.redhat.com/api/gathering/v2/%s/gathering_rules","pull_report":{"endpoint":"https://console.redhat.com/api/insights-results-aggregator/v2/cluster/%s/reports","delay":"60s","timeout":"3000s","min_retry":"30s"},"impersonate":"system:serviceaccount:openshift-insights:gather","enableGlobalObfuscation":false,"ocm":{"scaEndpoint":"https://api.openshift.com/api/accounts_mgmt/v1/entitlement_certificates","scaInterval":"8h","scaDisabled":false,"clusterTransferEndpoint":"https://api.openshift.com/api/accounts_mgmt/v1/cluster_transfers/","clusterTransferInterval":"12h"},"disableInsightsAlerts":false,"processingStatusEndpoint":"https://console.redhat.com/api/insights-results-aggregator/v2/cluster/%s/request/%s/status","reportEndpointTechPreview":"https://console.redhat.com/api/insights-results-aggregator/v2/cluster/%s/request/%s/report"} I0704 18:14:48.849704 1 secure_serving.go:57] Forcing use of http/1.1 only W0704 18:14:48.849885 1 secure_serving.go:69] Use of insecure cipher 'TLS_ECDHE_ECDSA_WITH_AES_128_CBC_SHA256' detected. W0704 18:14:48.849916 1 secure_serving.go:69] Use of insecure cipher 'TLS_ECDHE_RSA_WITH_AES_128_CBC_SHA256' detected. W0704 18:14:48.849927 1 secure_serving.go:69] Use of insecure cipher 'TLS_RSA_WITH_AES_128_GCM_SHA256' detected. W0704 18:14:48.849932 1 secure_serving.go:69] Use of insecure cipher 'TLS_RSA_WITH_AES_256_GCM_SHA384' detected. W0704 18:14:48.849936 1 secure_serving.go:69] Use of insecure cipher 'TLS_RSA_WITH_AES_128_CBC_SHA' detected. W0704 18:14:48.849939 1 secure_serving.go:69] Use of insecure cipher 'TLS_RSA_WITH_AES_256_CBC_SHA' detected. I0704 18:14:48.849811 1 simple_featuregate_reader.go:171] Starting feature-gate-detector I0704 18:14:48.853189 1 operator.go:125] FeatureGates initialized: knownFeatureGates=[AdditionalRoutingCapabilities AdminNetworkPolicy AlibabaPlatform AzureWorkloadIdentity BuildCSIVolumes CPMSMachineNamePrefix ConsolePluginContentSecurityPolicy ExternalOIDC ExternalOIDCWithUIDAndExtraClaimMappings GatewayAPI GatewayAPIController HighlyAvailableArbiter ImageVolume IngressControllerLBSubnetsAWS KMSv1 MachineConfigNodes ManagedBootImages ManagedBootImagesAWS MetricsCollectionProfiles NetworkDiagnosticsConfig NetworkLiveMigration NetworkSegmentation PinnedImages ProcMountType RouteAdvertisements RouteExternalCertificate ServiceAccountTokenNodeBinding SetEIPForNLBIngressController SigstoreImageVerification StoragePerformantSecurityPolicy UpgradeStatus UserNamespacesPodSecurityStandards UserNamespacesSupport VSphereMultiDisk VSphereMultiNetworks AWSClusterHostedDNS AWSClusterHostedDNSInstall AWSDedicatedHosts AWSServiceLBNetworkSecurityGroup AutomatedEtcdBackup AzureClusterHostedDNSInstall AzureDedicatedHosts AzureMultiDisk BootImageSkewEnforcement BootcNodeManagement ClusterAPIInstall ClusterAPIInstallIBMCloud ClusterMonitoringConfig ClusterVersionOperatorConfiguration DNSNameResolver DualReplica DyanmicServiceEndpointIBMCloud DynamicResourceAllocation EtcdBackendQuota EventedPLEG Example Example2 ExternalSnapshotMetadata GCPClusterHostedDNS GCPClusterHostedDNSInstall GCPCustomAPIEndpoints GCPCustomAPIEndpointsInstall ImageModeStatusReporting ImageStreamImportMode IngressControllerDynamicConfigurationManager InsightsConfig InsightsConfigAPI InsightsOnDemandDataGather IrreconcilableMachineConfig KMSEncryptionProvider MachineAPIMigration MachineAPIOperatorDisableMachineHealthCheckController ManagedBootImagesAzure ManagedBootImagesvSphere MaxUnavailableStatefulSet MinimumKubeletVersion MixedCPUsAllocation MultiArchInstallAzure MultiDiskSetup MutatingAdmissionPolicy NewOLM NewOLMCatalogdAPIV1Metas NewOLMOwnSingleNamespace NewOLMPreflightPermissionChecks NewOLMWebhookProviderOpenshiftServiceCA NoRegistryClusterOperations NodeSwap NutanixMultiSubnets OVNObservability OpenShiftPodSecurityAdmission PreconfiguredUDNAddresses SELinuxMount ShortCertRotation SignatureStores SigstoreImageVerificationPKI TranslateStreamCloseWebsocketRequests VSphereConfigurableMaxAllowedBlockVolumesPerNode VSphereHostVMGroupZonal VSphereMixedNodeEnv VolumeAttributesClass VolumeGroupSnapshot] I0704 18:14:48.853277 1 event.go:377] Event(v1.ObjectReference{Kind:"Deployment", Namespace:"openshift-insights", Name:"insights-operator", UID:"ab1a42d4-ed64-4d4e-8ba1-e9da502519c2", APIVersion:"apps/v1", ResourceVersion:"", FieldPath:""}): type: 'Normal' reason: 'FeatureGatesInitialized' FeatureGates updated to featuregates.Features{Enabled:[]v1.FeatureGateName{"AdditionalRoutingCapabilities", "AdminNetworkPolicy", "AlibabaPlatform", "AzureWorkloadIdentity", "BuildCSIVolumes", "CPMSMachineNamePrefix", "ConsolePluginContentSecurityPolicy", "ExternalOIDC", "ExternalOIDCWithUIDAndExtraClaimMappings", "GatewayAPI", "GatewayAPIController", "HighlyAvailableArbiter", "ImageVolume", "IngressControllerLBSubnetsAWS", "KMSv1", "MachineConfigNodes", "ManagedBootImages", "ManagedBootImagesAWS", "MetricsCollectionProfiles", "NetworkDiagnosticsConfig", "NetworkLiveMigration", "NetworkSegmentation", "PinnedImages", "ProcMountType", "RouteAdvertisements", "RouteExternalCertificate", "ServiceAccountTokenNodeBinding", "SetEIPForNLBIngressController", "SigstoreImageVerification", "StoragePerformantSecurityPolicy", "UpgradeStatus", "UserNamespacesPodSecurityStandards", "UserNamespacesSupport", "VSphereMultiDisk", "VSphereMultiNetworks"}, Disabled:[]v1.FeatureGateName{"AWSClusterHostedDNS", "AWSClusterHostedDNSInstall", "AWSDedicatedHosts", "AWSServiceLBNetworkSecurityGroup", "AutomatedEtcdBackup", "AzureClusterHostedDNSInstall", "AzureDedicatedHosts", "AzureMultiDisk", "BootImageSkewEnforcement", "BootcNodeManagement", "ClusterAPIInstall", "ClusterAPIInstallIBMCloud", "ClusterMonitoringConfig", "ClusterVersionOperatorConfiguration", "DNSNameResolver", "DualReplica", "DyanmicServiceEndpointIBMCloud", "DynamicResourceAllocation", "EtcdBackendQuota", "EventedPLEG", "Example", "Example2", "ExternalSnapshotMetadata", "GCPClusterHostedDNS", "GCPClusterHostedDNSInstall", "GCPCustomAPIEndpoints", "GCPCustomAPIEndpointsInstall", "ImageModeStatusReporting", "ImageStreamImportMode", "IngressControllerDynamicConfigurationManager", "InsightsConfig", "InsightsConfigAPI", "InsightsOnDemandDataGather", "IrreconcilableMachineConfig", "KMSEncryptionProvider", "MachineAPIMigration", "MachineAPIOperatorDisableMachineHealthCheckController", "ManagedBootImagesAzure", "ManagedBootImagesvSphere", "MaxUnavailableStatefulSet", "MinimumKubeletVersion", "MixedCPUsAllocation", "MultiArchInstallAzure", "MultiDiskSetup", "MutatingAdmissionPolicy", "NewOLM", "NewOLMCatalogdAPIV1Metas", "NewOLMOwnSingleNamespace", "NewOLMPreflightPermissionChecks", "NewOLMWebhookProviderOpenshiftServiceCA", "NoRegistryClusterOperations", "NodeSwap", "NutanixMultiSubnets", "OVNObservability", "OpenShiftPodSecurityAdmission", "PreconfiguredUDNAddresses", "SELinuxMount", "ShortCertRotation", "SignatureStores", "SigstoreImageVerificationPKI", "TranslateStreamCloseWebsocketRequests", "VSphereConfigurableMaxAllowedBlockVolumesPerNode", "VSphereHostVMGroupZonal", "VSphereMixedNodeEnv", "VolumeAttributesClass", "VolumeGroupSnapshot"}} I0704 18:14:48.854421 1 requestheader_controller.go:180] Starting RequestHeaderAuthRequestController I0704 18:14:48.854474 1 shared_informer.go:313] Waiting for caches to sync for RequestHeaderAuthRequestController I0704 18:14:48.854532 1 configmap_cafile_content.go:205] "Starting controller" name="client-ca::kube-system::extension-apiserver-authentication::client-ca-file" I0704 18:14:48.854972 1 shared_informer.go:313] Waiting for caches to sync for client-ca::kube-system::extension-apiserver-authentication::client-ca-file I0704 18:14:48.854579 1 configmap_cafile_content.go:205] "Starting controller" name="client-ca::kube-system::extension-apiserver-authentication::requestheader-client-ca-file" I0704 18:14:48.855047 1 shared_informer.go:313] Waiting for caches to sync for client-ca::kube-system::extension-apiserver-authentication::requestheader-client-ca-file I0704 18:14:48.854892 1 dynamic_serving_content.go:135] "Starting controller" name="serving-cert::/tmp/serving-cert-561975089/tls.crt::/tmp/serving-cert-561975089/tls.key" I0704 18:14:48.855100 1 tlsconfig.go:243] "Starting DynamicServingCertificateController" I0704 18:14:48.855086 1 secure_serving.go:213] Serving securely on [::]:8443 W0704 18:14:48.858320 1 configmapobserver.go:64] Cannot get the configuration config map: configmaps "insights-config" not found. Default configuration is used. I0704 18:14:48.858346 1 secretconfigobserver.go:216] Legacy configuration set: enabled=false endpoint=https://console.redhat.com/api/ingress/v1/upload conditional_gatherer_endpoint=https://console.redhat.com/api/gathering/v2/%s/gathering_rules interval=2h0m0s token=false reportEndpoint=https://console.redhat.com/api/insights-results-aggregator/v2/cluster/%s/reports initialPollingDelay=1m0s minRetryTime=30s pollingTimeout=50m0s processingStatusEndpoint=https://console.redhat.com/api/insights-results-aggregator/v2/cluster/%s/request/%s/status I0704 18:14:48.858546 1 base_controller.go:76] Waiting for caches to sync for ConfigController I0704 18:14:48.865395 1 secretconfigobserver.go:249] Found cloud.openshift.com token I0704 18:14:48.865463 1 secretconfigobserver.go:204] Legacy configuration updated: enabled=true endpoint=https://console.redhat.com/api/ingress/v1/upload conditional_gatherer_endpoint=https://console.redhat.com/api/gathering/v2/%s/gathering_rules interval=2h0m0s token=true reportEndpoint=https://console.redhat.com/api/insights-results-aggregator/v2/cluster/%s/reports initialPollingDelay=1m0s minRetryTime=30s pollingTimeout=50m0s processingStatusEndpoint=https://console.redhat.com/api/insights-results-aggregator/v2/cluster/%s/request/%s/status I0704 18:14:48.870435 1 secretconfigobserver.go:119] support secret does not exist I0704 18:14:48.874628 1 secretconfigobserver.go:249] Found cloud.openshift.com token I0704 18:14:48.878113 1 secretconfigobserver.go:119] support secret does not exist I0704 18:14:48.880960 1 recorder.go:161] Pruning old reports every 4h14m55s, max age is 288h0m0s I0704 18:14:48.884612 1 controllerstatus.go:80] name=insightsuploader healthy=true reason= message= I0704 18:14:48.884636 1 insightsuploader.go:86] Reporting status periodically to https://console.redhat.com/api/ingress/v1/upload every 2h0m0s, starting in 1m30s I0704 18:14:48.884663 1 controllerstatus.go:80] name=insightsreport healthy=true reason= message= I0704 18:14:48.884671 1 insightsreport.go:296] Starting report retriever I0704 18:14:48.884677 1 insightsreport.go:298] Insights analysis reports will be downloaded from the https://console.redhat.com/api/insights-results-aggregator/v2/cluster/%s/reports endpoint with a delay of 1m0s I0704 18:14:48.884716 1 periodic.go:209] Running clusterconfig gatherer I0704 18:14:48.884786 1 tasks_processing.go:45] number of workers: 32 I0704 18:14:48.884826 1 tasks_processing.go:69] worker 0 listening for tasks. I0704 18:14:48.884870 1 tasks_processing.go:69] worker 27 listening for tasks. I0704 18:14:48.884916 1 tasks_processing.go:69] worker 8 listening for tasks. I0704 18:14:48.884949 1 tasks_processing.go:71] worker 8 working on mutating_webhook_configurations task. I0704 18:14:48.884872 1 tasks_processing.go:69] worker 3 listening for tasks. I0704 18:14:48.884917 1 tasks_processing.go:69] worker 14 listening for tasks. I0704 18:14:48.884838 1 tasks_processing.go:69] worker 17 listening for tasks. I0704 18:14:48.885069 1 tasks_processing.go:71] worker 14 working on node_logs task. I0704 18:14:48.885071 1 tasks_processing.go:71] worker 17 working on support_secret task. I0704 18:14:48.884832 1 tasks_processing.go:69] worker 1 listening for tasks. I0704 18:14:48.885161 1 tasks_processing.go:71] worker 1 working on qemu_kubevirt_launcher_logs task. I0704 18:14:48.884847 1 tasks_processing.go:69] worker 18 listening for tasks. I0704 18:14:48.884850 1 tasks_processing.go:69] worker 24 listening for tasks. I0704 18:14:48.884854 1 tasks_processing.go:69] worker 25 listening for tasks. I0704 18:14:48.884852 1 tasks_processing.go:69] worker 16 listening for tasks. I0704 18:14:48.884858 1 tasks_processing.go:69] worker 19 listening for tasks. I0704 18:14:48.884861 1 tasks_processing.go:69] worker 26 listening for tasks. I0704 18:14:48.884863 1 tasks_processing.go:69] worker 2 listening for tasks. I0704 18:14:48.884865 1 tasks_processing.go:69] worker 20 listening for tasks. I0704 18:14:48.884874 1 tasks_processing.go:69] worker 21 listening for tasks. I0704 18:14:48.884876 1 tasks_processing.go:71] worker 0 working on certificate_signing_requests task. I0704 18:14:48.885574 1 tasks_processing.go:71] worker 21 working on tsdb_status task. W0704 18:14:48.885606 1 gather_prometheus_tsdb_status.go:38] Unable to load metrics client, tsdb status cannot be collected: open /var/run/configmaps/service-ca-bundle/service-ca.crt: no such file or directory I0704 18:14:48.885617 1 tasks_processing.go:71] worker 21 working on image_pruners task. I0704 18:14:48.885632 1 tasks_processing.go:71] worker 19 working on operators_pods_and_events task. I0704 18:14:48.884830 1 tasks_processing.go:69] worker 31 listening for tasks. I0704 18:14:48.884881 1 tasks_processing.go:69] worker 4 listening for tasks. I0704 18:14:48.884881 1 tasks_processing.go:69] worker 22 listening for tasks. I0704 18:14:48.884888 1 tasks_processing.go:69] worker 5 listening for tasks. I0704 18:14:48.884888 1 tasks_processing.go:69] worker 29 listening for tasks. I0704 18:14:48.884889 1 tasks_processing.go:69] worker 23 listening for tasks. I0704 18:14:48.884895 1 tasks_processing.go:69] worker 6 listening for tasks. I0704 18:14:48.885826 1 tasks_processing.go:71] worker 20 working on machine_config_pools task. I0704 18:14:48.885869 1 tasks_processing.go:71] worker 16 working on install_plans task. I0704 18:14:48.885910 1 tasks_processing.go:71] worker 6 working on validating_webhook_configurations task. I0704 18:14:48.885975 1 tasks_processing.go:71] worker 22 working on openshift_machine_api_events task. I0704 18:14:48.884898 1 tasks_processing.go:69] worker 11 listening for tasks. I0704 18:14:48.886112 1 tasks_processing.go:71] worker 11 working on container_runtime_configs task. I0704 18:14:48.884898 1 tasks_processing.go:69] worker 28 listening for tasks. I0704 18:14:48.884902 1 tasks_processing.go:69] worker 12 listening for tasks. I0704 18:14:48.884906 1 tasks_processing.go:69] worker 7 listening for tasks. I0704 18:14:48.884908 1 tasks_processing.go:69] worker 9 listening for tasks. I0704 18:14:48.884910 1 tasks_processing.go:69] worker 13 listening for tasks. I0704 18:14:48.884913 1 tasks_processing.go:69] worker 10 listening for tasks. I0704 18:14:48.884922 1 tasks_processing.go:71] worker 27 working on metrics task. W0704 18:14:48.886205 1 gather_most_recent_metrics.go:64] Unable to load metrics client, no metrics will be collected: open /var/run/configmaps/service-ca-bundle/service-ca.crt: no such file or directory I0704 18:14:48.884922 1 tasks_processing.go:69] worker 15 listening for tasks. I0704 18:14:48.884929 1 tasks_processing.go:69] worker 30 listening for tasks. I0704 18:14:48.885053 1 tasks_processing.go:71] worker 3 working on openstack_dataplanenodesets task. I0704 18:14:48.886339 1 tasks_processing.go:71] worker 30 working on openstack_version task. I0704 18:14:48.886410 1 tasks_processing.go:71] worker 9 working on openstack_controlplanes task. I0704 18:14:48.885817 1 tasks_processing.go:71] worker 26 working on cost_management_metrics_configs task. I0704 18:14:48.886492 1 tasks_processing.go:71] worker 23 working on feature_gates task. I0704 18:14:48.886494 1 tasks_processing.go:71] worker 29 working on number_of_pods_and_netnamespaces_with_sdn_annotations task. I0704 18:14:48.886573 1 tasks_processing.go:71] worker 28 working on monitoring_persistent_volumes task. I0704 18:14:48.885822 1 tasks_processing.go:71] worker 2 working on service_accounts task. I0704 18:14:48.886684 1 tasks_processing.go:71] worker 13 working on nodenetworkconfigurationpolicies task. I0704 18:14:48.886891 1 tasks_processing.go:71] worker 12 working on nodes task. I0704 18:14:48.886950 1 tasks_processing.go:71] worker 31 working on image task. I0704 18:14:48.886969 1 tasks_processing.go:71] worker 10 working on storage_classes task. I0704 18:14:48.887071 1 tasks_processing.go:71] worker 5 working on dvo_metrics task. I0704 18:14:48.887566 1 tasks_processing.go:71] worker 15 working on openstack_dataplanedeployments task. I0704 18:14:48.885842 1 gather.go:177] gatherer "clusterconfig" function "tsdb_status" took 29.957µs to process 0 records I0704 18:14:48.887876 1 tasks_processing.go:71] worker 4 working on machine_configs task. I0704 18:14:48.887914 1 tasks_processing.go:71] worker 27 working on machines task. I0704 18:14:48.885853 1 tasks_processing.go:71] worker 24 working on schedulers task. I0704 18:14:48.888175 1 tasks_processing.go:71] worker 7 working on cluster_apiserver task. I0704 18:14:48.885858 1 tasks_processing.go:71] worker 18 working on sap_pods task. I0704 18:14:48.887904 1 gather.go:177] gatherer "clusterconfig" function "metrics" took 28.888µs to process 0 records I0704 18:14:48.885864 1 tasks_processing.go:71] worker 25 working on jaegers task. I0704 18:14:48.889294 1 controller.go:128] Initializing last reported time to 0001-01-01T00:00:00Z I0704 18:14:48.889314 1 controller.go:212] Source periodic-clusterconfig *controllerstatus.Simple is not ready I0704 18:14:48.889320 1 controller.go:212] Source periodic-conditional *controllerstatus.Simple is not ready I0704 18:14:48.889324 1 controller.go:212] Source periodic-workloads *controllerstatus.Simple is not ready I0704 18:14:48.889342 1 controller.go:489] The operator is still being initialized I0704 18:14:48.889350 1 controller.go:512] The operator is healthy I0704 18:14:48.891214 1 tasks_processing.go:71] worker 8 working on networks task. I0704 18:14:48.891569 1 recorder.go:75] Recording config/mutatingwebhookconfigurations/aws-pod-identity with fingerprint=c1664d6400997576eeda39b19f8d67b603e7eb908cf41821fe6694535108788c I0704 18:14:48.891621 1 recorder.go:75] Recording config/mutatingwebhookconfigurations/sre-podimagespec-mutation with fingerprint=782888244acb9e79ef630bfd091c1269b5e65a952661872e13205e7f1786142c I0704 18:14:48.891663 1 recorder.go:75] Recording config/mutatingwebhookconfigurations/sre-service-mutation with fingerprint=782970e4a79d516b354afd2ef112e76e0d1d94b1bb7125c7bff94089e7bd0c78 I0704 18:14:48.891675 1 gather.go:177] gatherer "clusterconfig" function "mutating_webhook_configurations" took 6.22478ms to process 3 records I0704 18:14:48.899979 1 tasks_processing.go:71] worker 17 working on oauths task. E0704 18:14:48.900001 1 gather.go:140] gatherer "clusterconfig" function "support_secret" failed with the error: secrets "support" not found I0704 18:14:48.900019 1 gather.go:177] gatherer "clusterconfig" function "support_secret" took 14.896527ms to process 0 records I0704 18:14:48.900064 1 gather.go:177] gatherer "clusterconfig" function "openstack_controlplanes" took 13.640903ms to process 0 records I0704 18:14:48.900131 1 gather.go:177] gatherer "clusterconfig" function "cost_management_metrics_configs" took 13.69426ms to process 0 records I0704 18:14:48.900193 1 gather.go:177] gatherer "clusterconfig" function "openshift_machine_api_events" took 14.147921ms to process 0 records I0704 18:14:48.900219 1 tasks_processing.go:71] worker 26 working on infrastructures task. I0704 18:14:48.900326 1 tasks_processing.go:71] worker 9 working on active_alerts task. I0704 18:14:48.900224 1 gather.go:177] gatherer "clusterconfig" function "openstack_dataplanedeployments" took 12.609368ms to process 0 records I0704 18:14:48.900230 1 tasks_processing.go:71] worker 15 working on ingress_certificates task. I0704 18:14:48.900236 1 tasks_processing.go:71] worker 22 working on crds task. W0704 18:14:48.900403 1 gather_active_alerts.go:54] Unable to load alerts client, no alerts will be collected: open /var/run/configmaps/service-ca-bundle/service-ca.crt: no such file or directory I0704 18:14:48.900444 1 gather.go:177] gatherer "clusterconfig" function "openstack_dataplanenodesets" took 14.07285ms to process 0 records I0704 18:14:48.900635 1 gather.go:177] gatherer "clusterconfig" function "nodenetworkconfigurationpolicies" took 13.62308ms to process 0 records I0704 18:14:48.901407 1 gather.go:177] gatherer "clusterconfig" function "jaegers" took 11.845612ms to process 0 records I0704 18:14:48.901466 1 gather.go:177] gatherer "clusterconfig" function "node_logs" took 15.311467ms to process 0 records I0704 18:14:48.900454 1 tasks_processing.go:71] worker 3 working on ingress task. I0704 18:14:48.901565 1 tasks_processing.go:71] worker 25 working on storage_cluster task. I0704 18:14:48.901495 1 gather.go:177] gatherer "clusterconfig" function "openstack_version" took 14.09614ms to process 0 records I0704 18:14:48.901511 1 tasks_processing.go:71] worker 30 working on config_maps task. I0704 18:14:48.902253 1 tasks_processing.go:71] worker 13 working on container_images task. I0704 18:14:48.902433 1 recorder.go:75] Recording config/featuregate with fingerprint=a263834283590a306805dbe214b2ffaa6bbfa8c2f038d9a684088163092c23f1 I0704 18:14:48.902473 1 gather.go:177] gatherer "clusterconfig" function "feature_gates" took 14.060106ms to process 1 records I0704 18:14:48.901518 1 tasks_processing.go:71] worker 14 working on pod_network_connectivity_checks task. I0704 18:14:48.902504 1 gather.go:177] gatherer "clusterconfig" function "active_alerts" took 238.622µs to process 0 records I0704 18:14:48.902594 1 tasks_processing.go:71] worker 9 working on operators task. I0704 18:14:48.902802 1 tasks_processing.go:71] worker 23 working on overlapping_namespace_uids task. I0704 18:14:48.904055 1 tasks_processing.go:71] worker 18 working on sap_datahubs task. I0704 18:14:48.903856 1 gather.go:177] gatherer "clusterconfig" function "sap_pods" took 13.09412ms to process 0 records I0704 18:14:48.904191 1 gather.go:177] gatherer "clusterconfig" function "container_runtime_configs" took 15.376046ms to process 0 records I0704 18:14:48.904207 1 tasks_processing.go:71] worker 27 working on openshift_logging task. I0704 18:14:48.904259 1 tasks_processing.go:71] worker 11 working on machine_sets task. E0704 18:14:48.904198 1 gather.go:140] gatherer "clusterconfig" function "machines" failed with the error: machines.machine.openshift.io is forbidden: User "system:serviceaccount:openshift-insights:gather" cannot list resource "machines" in API group "machine.openshift.io" at the cluster scope I0704 18:14:48.905520 1 gather.go:177] gatherer "clusterconfig" function "machines" took 14.530206ms to process 0 records I0704 18:14:48.905620 1 gather_logs.go:145] no pods in namespace were found I0704 18:14:48.905673 1 recorder.go:75] Recording config/schedulers/cluster with fingerprint=e3f06220bceeab695f868f497fb1b48911ba7f4ee04d2b1b946479e84b999c53 I0704 18:14:48.905703 1 gather.go:177] gatherer "clusterconfig" function "schedulers" took 16.970291ms to process 1 records I0704 18:14:48.905836 1 tasks_processing.go:71] worker 24 working on aggregated_monitoring_cr_names task. I0704 18:14:48.906131 1 tasks_processing.go:71] worker 21 working on machine_autoscalers task. I0704 18:14:48.906166 1 recorder.go:75] Recording config/clusteroperator/imageregistry.operator.openshift.io/imagepruner/cluster with fingerprint=8caf1cf46da7795536ab97ddd550bf5912526e5585f168858b874a2afd620d2c I0704 18:14:48.906287 1 gather.go:177] gatherer "clusterconfig" function "image_pruners" took 19.63908ms to process 1 records I0704 18:14:48.906518 1 recorder.go:75] Recording config/validatingwebhookconfigurations/multus.openshift.io with fingerprint=5605c743b6dae1b32b6e32f4ec22ecd0a25c65af817506c070b1cccfc86f9dfa I0704 18:14:48.906542 1 tasks_processing.go:71] worker 6 working on silenced_alerts task. W0704 18:14:48.906777 1 gather_silenced_alerts.go:38] Unable to load alerts client, no alerts will be collected: open /var/run/configmaps/service-ca-bundle/service-ca.crt: no such file or directory I0704 18:14:48.906715 1 recorder.go:75] Recording config/validatingwebhookconfigurations/network-node-identity.openshift.io with fingerprint=69edebc306e4b40e86922834a45d5b86a9be3e730ad1a812288fdb59d98a772c I0704 18:14:48.906938 1 recorder.go:75] Recording config/validatingwebhookconfigurations/performance-addon-operator with fingerprint=bf44fa355d8b2fbe75c30af16dcca3b1c26ad743e777ea2612d4d74bdd7f0632 I0704 18:14:48.906991 1 recorder.go:75] Recording config/validatingwebhookconfigurations/sre-clusterrolebindings-validation with fingerprint=361df9cccf677804c2bbbe94f226a2b6a9617679e07e9c04a24edcb91b10900b I0704 18:14:48.907029 1 recorder.go:75] Recording config/validatingwebhookconfigurations/sre-clusterroles-validation with fingerprint=730f89ece56b00da1953ae4d29248ca135af11e20590e3ba9da8bf4355efa07a I0704 18:14:48.907068 1 recorder.go:75] Recording config/validatingwebhookconfigurations/sre-ingress-config-validation with fingerprint=da5b9ded72d53269ed9bf97e104462d9f764cbdd6f925a6a14ef68548bf6cd0b I0704 18:14:48.907104 1 recorder.go:75] Recording config/validatingwebhookconfigurations/sre-network-operator-validation with fingerprint=6e8d1619fe092849c3ea9d6f43efc236a3847f54ce7aa1a670de604314173f7d I0704 18:14:48.907158 1 recorder.go:75] Recording config/validatingwebhookconfigurations/sre-regular-user-validation with fingerprint=55e3b81d6f5ebe5dd602e2fe36e9ca5dc78d049eaf2b0229f288b8f191f1389b I0704 18:14:48.907194 1 recorder.go:75] Recording config/validatingwebhookconfigurations/sre-scc-validation with fingerprint=1f609fb7527b74c7978250630f3fac1385ad8863401cb44bdc9933ef986016f5 I0704 18:14:48.907237 1 recorder.go:75] Recording config/validatingwebhookconfigurations/sre-serviceaccount-validation with fingerprint=3b9fcc3c8f5fc2ef8b3ea116fa548ab29c4d1595fbc0075e368f4fb45ae531d9 I0704 18:14:48.907271 1 recorder.go:75] Recording config/validatingwebhookconfigurations/sre-techpreviewnoupgrade-validation with fingerprint=a79ec5c1463bcd4a1b8664a54e2594151992e16a33ab995281b36cbf5a90c9d2 I0704 18:14:48.907280 1 gather.go:177] gatherer "clusterconfig" function "validating_webhook_configurations" took 19.385649ms to process 11 records I0704 18:14:48.907369 1 tasks_processing.go:71] worker 10 working on machine_healthchecks task. I0704 18:14:48.907387 1 recorder.go:75] Recording config/storage/storageclasses/gp2-csi with fingerprint=8fca1e3c9cdad1c7ba2a1337ea824423dab0ccbbb8b0b8c5c6573d7be00c49ca I0704 18:14:48.907407 1 recorder.go:75] Recording config/storage/storageclasses/gp3-csi with fingerprint=107074f2314c36307e750c5b37c8cca22c56f4707337067786984edf9317d032 I0704 18:14:48.907422 1 gather.go:177] gatherer "clusterconfig" function "storage_classes" took 18.416359ms to process 2 records I0704 18:14:48.907541 1 recorder.go:75] Recording config/apiserver with fingerprint=d6f6a02297bbe11995326a5e2bc41f0ac4564a6ee81982a764c9749f84f061bd I0704 18:14:48.907553 1 gather.go:177] gatherer "clusterconfig" function "cluster_apiserver" took 17.163751ms to process 1 records I0704 18:14:48.907560 1 gather.go:177] gatherer "clusterconfig" function "qemu_kubevirt_launcher_logs" took 20.433219ms to process 0 records I0704 18:14:48.907566 1 tasks_processing.go:71] worker 7 working on clusterroles task. I0704 18:14:48.907590 1 tasks_processing.go:71] worker 1 working on lokistack task. I0704 18:14:48.907647 1 recorder.go:75] Recording config/image with fingerprint=c22937246384c8bcdfff9cfa803189d9e1622a1273bd12b88a0cf4fcd5d75fd7 I0704 18:14:48.907656 1 gather.go:177] gatherer "clusterconfig" function "image" took 19.575373ms to process 1 records I0704 18:14:48.907666 1 gather.go:177] gatherer "clusterconfig" function "silenced_alerts" took 98.305µs to process 0 records I0704 18:14:48.907672 1 tasks_processing.go:71] worker 31 working on ceph_cluster task. I0704 18:14:48.907868 1 tasks_processing.go:71] worker 6 working on olm_operators task. I0704 18:14:48.907998 1 recorder.go:75] Recording config/node/ip-10-0-1-178.ec2.internal with fingerprint=381a87d3824eb0f361186ba4aa0a0065ad0006fbfc12a0393668837a4c67931b I0704 18:14:48.908110 1 recorder.go:75] Recording config/node/ip-10-0-2-42.ec2.internal with fingerprint=d1d07613939d7050f155e2b479e8c1c59fea0648dcc30b3cfe948d43e81d8c53 I0704 18:14:48.908017 1 tasks_processing.go:71] worker 12 working on proxies task. I0704 18:14:48.908123 1 gather.go:177] gatherer "clusterconfig" function "nodes" took 19.956912ms to process 2 records I0704 18:14:48.908265 1 tasks_processing.go:71] worker 0 working on sap_config task. I0704 18:14:48.908414 1 gather.go:177] gatherer "clusterconfig" function "certificate_signing_requests" took 22.819039ms to process 0 records I0704 18:14:48.908473 1 gather.go:177] gatherer "clusterconfig" function "monitoring_persistent_volumes" took 21.886131ms to process 0 records I0704 18:14:48.908504 1 tasks_processing.go:71] worker 28 working on image_registries task. I0704 18:14:48.911217 1 tasks_processing.go:71] worker 25 working on pdbs task. I0704 18:14:48.911308 1 gather.go:177] gatherer "clusterconfig" function "storage_cluster" took 9.61323ms to process 0 records I0704 18:14:48.911430 1 tasks_processing.go:71] worker 8 working on version task. I0704 18:14:48.911561 1 recorder.go:75] Recording config/network with fingerprint=d40fbedec8ac1a192be8c06835da3280cfdb456acb3eb4e4931144c9bfac137b I0704 18:14:48.911620 1 gather.go:177] gatherer "clusterconfig" function "networks" took 20.110943ms to process 1 records W0704 18:14:48.911582 1 gather_dvo_metrics.go:210] Failed to read the DVO metrics. Trying again. I0704 18:14:48.922084 1 tasks_processing.go:71] worker 31 working on authentication task. I0704 18:14:48.922122 1 gather.go:177] gatherer "clusterconfig" function "ceph_cluster" took 14.399259ms to process 0 records I0704 18:14:48.922294 1 tasks_processing.go:71] worker 18 working on nodenetworkstates task. I0704 18:14:48.922328 1 gather.go:177] gatherer "clusterconfig" function "sap_datahubs" took 18.157764ms to process 0 records E0704 18:14:48.922388 1 gather.go:140] gatherer "clusterconfig" function "pod_network_connectivity_checks" failed with the error: the server could not find the requested resource (get podnetworkconnectivitychecks.controlplane.operator.openshift.io) I0704 18:14:48.922421 1 gather.go:177] gatherer "clusterconfig" function "pod_network_connectivity_checks" took 18.532761ms to process 0 records I0704 18:14:48.922495 1 tasks_processing.go:74] worker 14 stopped. I0704 18:14:48.922558 1 tasks_processing.go:74] worker 3 stopped. I0704 18:14:48.922663 1 recorder.go:75] Recording config/ingress with fingerprint=4966ecbc5be945a172b7bf97269243c4e13585e36b6d54f26a5c47fe97b36bc4 I0704 18:14:48.922713 1 gather.go:177] gatherer "clusterconfig" function "ingress" took 20.894858ms to process 1 records I0704 18:14:48.922817 1 tasks_processing.go:74] worker 17 stopped. I0704 18:14:48.923078 1 recorder.go:75] Recording config/oauth with fingerprint=21ef8a5f557f5467a5daaba6b636a3578be3feb5702468de5d33f9c1f8d69160 I0704 18:14:48.923120 1 sca.go:136] Pulling SCA certificates from https://api.openshift.com/api/accounts_mgmt/v1/entitlement_certificates. Next check is in 8h0m0s I0704 18:14:48.923149 1 cluster_transfer.go:83] checking the availability of cluster transfer. Next check is in 12h0m0s I0704 18:14:48.923129 1 gather.go:177] gatherer "clusterconfig" function "oauths" took 22.479652ms to process 1 records W0704 18:14:48.923238 1 operator.go:288] started I0704 18:14:48.923259 1 base_controller.go:76] Waiting for caches to sync for LoggingSyncer I0704 18:14:48.923407 1 tasks_processing.go:74] worker 26 stopped. I0704 18:14:48.923921 1 recorder.go:75] Recording config/infrastructure with fingerprint=57282429e7cbd290602cd1d1ba6e0a40e670d1b066ea4b0046dbe8c76009097a I0704 18:14:48.923935 1 gather.go:177] gatherer "clusterconfig" function "infrastructures" took 22.88148ms to process 1 records I0704 18:14:48.929503 1 tasks_processing.go:74] worker 1 stopped. I0704 18:14:48.929528 1 gather.go:177] gatherer "clusterconfig" function "lokistack" took 21.899155ms to process 0 records E0704 18:14:48.929633 1 gather.go:140] gatherer "clusterconfig" function "machine_healthchecks" failed with the error: machinehealthchecks.machine.openshift.io is forbidden: User "system:serviceaccount:openshift-insights:gather" cannot list resource "machinehealthchecks" in API group "machine.openshift.io" at the cluster scope I0704 18:14:48.929647 1 gather.go:177] gatherer "clusterconfig" function "machine_healthchecks" took 22.159951ms to process 0 records I0704 18:14:48.929655 1 gather.go:177] gatherer "clusterconfig" function "machine_autoscalers" took 23.361639ms to process 0 records I0704 18:14:48.929661 1 gather.go:177] gatherer "clusterconfig" function "machine_sets" took 25.281994ms to process 0 records I0704 18:14:48.929668 1 tasks_processing.go:74] worker 11 stopped. I0704 18:14:48.929676 1 tasks_processing.go:74] worker 21 stopped. I0704 18:14:48.929682 1 tasks_processing.go:74] worker 10 stopped. I0704 18:14:48.929828 1 tasks_processing.go:74] worker 0 stopped. I0704 18:14:48.929844 1 gather.go:177] gatherer "clusterconfig" function "sap_config" took 21.547572ms to process 0 records I0704 18:14:48.929980 1 tasks_processing.go:74] worker 27 stopped. I0704 18:14:48.929999 1 gather.go:177] gatherer "clusterconfig" function "openshift_logging" took 25.75725ms to process 0 records I0704 18:14:48.954766 1 shared_informer.go:320] Caches are synced for RequestHeaderAuthRequestController I0704 18:14:48.955901 1 shared_informer.go:320] Caches are synced for client-ca::kube-system::extension-apiserver-authentication::requestheader-client-ca-file I0704 18:14:48.955915 1 shared_informer.go:320] Caches are synced for client-ca::kube-system::extension-apiserver-authentication::client-ca-file I0704 18:14:48.959066 1 base_controller.go:82] Caches are synced for ConfigController I0704 18:14:48.959079 1 base_controller.go:119] Starting #1 worker of ConfigController controller ... I0704 18:14:48.971713 1 tasks_processing.go:74] worker 18 stopped. I0704 18:14:48.971755 1 gather.go:177] gatherer "clusterconfig" function "nodenetworkstates" took 49.376004ms to process 0 records I0704 18:14:48.971987 1 tasks_processing.go:74] worker 20 stopped. I0704 18:14:48.972011 1 gather.go:177] gatherer "clusterconfig" function "machine_config_pools" took 86.112478ms to process 0 records I0704 18:14:48.972094 1 recorder.go:75] Recording config/proxy with fingerprint=8afc9f73909b5d3c32d034c68f01205c85609f91f2db0f14a529c31219bd31be I0704 18:14:48.972112 1 gather.go:177] gatherer "clusterconfig" function "proxies" took 63.861338ms to process 1 records I0704 18:14:48.972121 1 tasks_processing.go:74] worker 12 stopped. I0704 18:14:48.972348 1 tasks_processing.go:74] worker 7 stopped. I0704 18:14:48.972602 1 recorder.go:75] Recording cluster-scoped-resources/rbac.authorization.k8s.io/clusterroles/admin with fingerprint=f4cb3105184588643e9e912e1855f85949b90060ff843307d823e20a74888896 I0704 18:14:48.972827 1 recorder.go:75] Recording cluster-scoped-resources/rbac.authorization.k8s.io/clusterroles/edit with fingerprint=b0f3460196c1b86a5db1ced348b162616d187ebc3d8b8b5a35afbc6fece08a0a I0704 18:14:48.972875 1 gather.go:177] gatherer "clusterconfig" function "clusterroles" took 64.769168ms to process 2 records I0704 18:14:48.972894 1 gather.go:177] gatherer "clusterconfig" function "number_of_pods_and_netnamespaces_with_sdn_annotations" took 86.113474ms to process 0 records I0704 18:14:48.972903 1 tasks_processing.go:74] worker 29 stopped. I0704 18:14:48.973357 1 tasks_processing.go:74] worker 28 stopped. I0704 18:14:48.973683 1 recorder.go:75] Recording config/clusteroperator/imageregistry.operator.openshift.io/config/cluster with fingerprint=cbfa3cee48b5b48f09bc97ab459d80649620d46a107ce687ef7fb1f09ac4fd75 I0704 18:14:48.973695 1 gather.go:177] gatherer "clusterconfig" function "image_registries" took 64.837955ms to process 1 records E0704 18:14:48.981599 1 cluster_transfer.go:95] failed to pull cluster transfer: unable to retrieve cluster transfer data from https://api.openshift.com/api/accounts_mgmt/v1/cluster_transfers/: Get "https://api.openshift.com/api/accounts_mgmt/v1/cluster_transfers/?search=cluster_uuid+is+%27feb53f94-da09-459e-9b02-5a19cc5fc927%27+and+status+is+%27accepted%27": dial tcp: lookup api.openshift.com on 172.30.0.10:53: read udp 10.129.0.12:53014->172.30.0.10:53: read: connection refused I0704 18:14:48.981614 1 controllerstatus.go:80] name=clusterTransferController healthy=true reason=Disconnected message=failed to pull cluster transfer: unable to retrieve cluster transfer data from https://api.openshift.com/api/accounts_mgmt/v1/cluster_transfers/: Get "https://api.openshift.com/api/accounts_mgmt/v1/cluster_transfers/?search=cluster_uuid+is+%27feb53f94-da09-459e-9b02-5a19cc5fc927%27+and+status+is+%27accepted%27": dial tcp: lookup api.openshift.com on 172.30.0.10:53: read udp 10.129.0.12:53014->172.30.0.10:53: read: connection refused I0704 18:14:48.982810 1 prometheus_rules.go:88] Prometheus rules successfully created I0704 18:14:48.982882 1 controller.go:212] Source periodic-clusterconfig *controllerstatus.Simple is not ready I0704 18:14:48.982899 1 controller.go:212] Source periodic-conditional *controllerstatus.Simple is not ready I0704 18:14:48.982906 1 controller.go:212] Source periodic-workloads *controllerstatus.Simple is not ready I0704 18:14:48.982913 1 controller.go:212] Source scaController *sca.Controller is not ready I0704 18:14:48.982937 1 controller.go:489] The operator is still being initialized I0704 18:14:48.982948 1 controller.go:512] The operator is healthy I0704 18:14:48.985686 1 configmapobserver.go:84] configmaps "insights-config" not found I0704 18:14:49.000124 1 tasks_processing.go:74] worker 31 stopped. I0704 18:14:49.000406 1 recorder.go:75] Recording config/authentication with fingerprint=c5c018a33a1b44fc3a4537f08d9b68b406ed2f96b024ecaa989d6230236e1ada I0704 18:14:49.000454 1 gather.go:177] gatherer "clusterconfig" function "authentication" took 78.016171ms to process 1 records I0704 18:14:49.000503 1 tasks_processing.go:74] worker 22 stopped. I0704 18:14:49.003466 1 recorder.go:75] Recording config/crd/volumesnapshots.snapshot.storage.k8s.io with fingerprint=8ae66d4ae73028aa71213ef7e7e704fe69489cfb21985f3006e3399c60ab4e6c I0704 18:14:49.003923 1 recorder.go:75] Recording config/crd/volumesnapshotcontents.snapshot.storage.k8s.io with fingerprint=536e197f8e175828fd3cde2d7da8e0aa3f4fb4edd50810ac89b258f2963612fe I0704 18:14:49.004040 1 gather.go:177] gatherer "clusterconfig" function "crds" took 100.023722ms to process 2 records I0704 18:14:49.005145 1 tasks_processing.go:74] worker 25 stopped. I0704 18:14:49.005328 1 recorder.go:75] Recording config/pdbs/openshift-image-registry/image-registry with fingerprint=c4e5c5803255367d9ef01e6d47b851ea12b787de0e31453586fb28f0f1186687 I0704 18:14:49.005362 1 recorder.go:75] Recording config/pdbs/openshift-ingress/router-default with fingerprint=ddcd5dd8b0c9b07a0891ed087496d982a8b643c7c82a2545a2f4f777acafcf03 I0704 18:14:49.005385 1 recorder.go:75] Recording config/pdbs/openshift-operator-lifecycle-manager/packageserver-pdb with fingerprint=be9a86c98c82151a06a180e7511064d22f33a6a5df96d03be726ffd29fde239c I0704 18:14:49.005394 1 gather.go:177] gatherer "clusterconfig" function "pdbs" took 93.857512ms to process 3 records I0704 18:14:49.007902 1 tasks_processing.go:74] worker 4 stopped. I0704 18:14:49.008012 1 recorder.go:75] Recording aggregated/unused_machine_configs_count with fingerprint=4bfc9fa984e5dfcd45848faaf05269de7619bf42edf9f781751af5ee05c1a499 I0704 18:14:49.008029 1 gather.go:177] gatherer "clusterconfig" function "machine_configs" took 119.978698ms to process 1 records I0704 18:14:49.014993 1 tasks_processing.go:74] worker 8 stopped. I0704 18:14:49.015377 1 recorder.go:75] Recording config/version with fingerprint=e94fa02b738244a2970b3603b516f6a8b2b1df987d305fe4bd2c8304144432c0 I0704 18:14:49.015393 1 recorder.go:75] Recording config/id with fingerprint=a7867e71b65c466072972171bf1ad8d4c4fb062600832135be6b3d3dee27a968 I0704 18:14:49.015400 1 gather.go:177] gatherer "clusterconfig" function "version" took 103.539803ms to process 2 records I0704 18:14:49.015929 1 tasks_processing.go:74] worker 23 stopped. I0704 18:14:49.015959 1 recorder.go:75] Recording config/namespaces_with_overlapping_uids with fingerprint=4f53cda18c2baa0c0354bb5f9a3ecbe5ed12ab4d8e11ba873c2f11161202b945 I0704 18:14:49.015970 1 gather.go:177] gatherer "clusterconfig" function "overlapping_namespace_uids" took 111.833405ms to process 1 records I0704 18:14:49.022593 1 requests.go:205] Asking for SCA certificate with "{"arch": ["x86_64"]}" payload I0704 18:14:49.023455 1 base_controller.go:82] Caches are synced for LoggingSyncer I0704 18:14:49.023522 1 base_controller.go:119] Starting #1 worker of LoggingSyncer controller ... W0704 18:14:49.028132 1 sca.go:161] Failed to pull SCA certs from https://api.openshift.com/api/accounts_mgmt/v1/entitlement_certificates: unable to retrieve SCA certs data from https://api.openshift.com/api/accounts_mgmt/v1/entitlement_certificates: Post "https://api.openshift.com/api/accounts_mgmt/v1/entitlement_certificates": dial tcp: lookup api.openshift.com on 172.30.0.10:53: read udp 10.129.0.12:60732->172.30.0.10:53: read: connection refused I0704 18:14:49.028150 1 controllerstatus.go:80] name=scaController healthy=true reason=NonHTTPError message=Failed to pull SCA certs from https://api.openshift.com/api/accounts_mgmt/v1/entitlement_certificates: unable to retrieve SCA certs data from https://api.openshift.com/api/accounts_mgmt/v1/entitlement_certificates: Post "https://api.openshift.com/api/accounts_mgmt/v1/entitlement_certificates": dial tcp: lookup api.openshift.com on 172.30.0.10:53: read udp 10.129.0.12:60732->172.30.0.10:53: read: connection refused I0704 18:14:49.028282 1 tasks_processing.go:74] worker 30 stopped. E0704 18:14:49.028300 1 gather.go:140] gatherer "clusterconfig" function "config_maps" failed with the error: configmaps "cluster-monitoring-config" not found E0704 18:14:49.028312 1 gather.go:140] gatherer "clusterconfig" function "config_maps" failed with the error: configmaps "gateway-mode-config" not found E0704 18:14:49.028318 1 gather.go:140] gatherer "clusterconfig" function "config_maps" failed with the error: configmaps "insights-config" not found I0704 18:14:49.028333 1 recorder.go:75] Recording config/configmaps/openshift-config/installer-images/images.json with fingerprint=26b6661162b099a0f5a279859b4f46c867929a79d9a4a41fde4be4e6fe138018 I0704 18:14:49.028787 1 recorder.go:75] Recording config/configmaps/openshift-config/kube-root-ca.crt/ca.crt with fingerprint=f73fcd48d6f9ebbc83253a1b5f23ef59d0683465ec46e474389103c717467dfb I0704 18:14:49.028809 1 recorder.go:75] Recording config/configmaps/openshift-config/openshift-install/invoker with fingerprint=76b482f683cd3ef9da02debac5b26080a5aeb06ff768ee5c21117514dff29d8a I0704 18:14:49.028818 1 recorder.go:75] Recording config/configmaps/openshift-config/openshift-install/version with fingerprint=c93090eb0d2a4736885abeb79c91680cfd01fda46464f83456b085d4dc8239f0 I0704 18:14:49.028825 1 recorder.go:75] Recording config/configmaps/openshift-config/openshift-service-ca.crt/service-ca.crt with fingerprint=e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855 I0704 18:14:49.028888 1 recorder.go:75] Recording config/configmaps/openshift-config/rosa-brand-logo/rosa-brand-logo.svg with fingerprint=6ed8ca4dd7a8eee7249182bc006e9649ce84d76c551ddfaaa33e55d8c4cc1ed0 I0704 18:14:49.028926 1 recorder.go:75] Recording config/configmaps/kube-system/cluster-config-v1/install-config with fingerprint=ab3811c6b83fd7b8e920094cfa3080d1b4ee3c35ec4c8379437b21d27bd6608d I0704 18:14:49.028963 1 gather.go:177] gatherer "clusterconfig" function "config_maps" took 126.024507ms to process 7 records I0704 18:14:49.034260 1 tasks_processing.go:74] worker 13 stopped. I0704 18:14:49.036821 1 recorder.go:75] Recording config/pod/openshift-console-operator/console-operator-575cd97545-cqt8b with fingerprint=f637ca1340ecc050e64627b460e6cb923bb06e705676669c73dee350fe10c353 I0704 18:14:49.037397 1 recorder.go:75] Recording config/pod/openshift-ovn-kubernetes/ovnkube-node-pdmxz with fingerprint=0424d4a1d8fcebbcccc1847c3d40efe2cbd8a0dcfb03dbf2c3ae0b548f4e0638 I0704 18:14:49.037493 1 recorder.go:75] Recording config/running_containers with fingerprint=6d6c409ac2285e492ce0af704dafcdad5cc5e735a92f03b24b04fd06defd0345 I0704 18:14:49.037548 1 gather.go:177] gatherer "clusterconfig" function "container_images" took 131.987472ms to process 3 records E0704 18:14:49.037570 1 gather.go:140] gatherer "clusterconfig" function "ingress_certificates" failed with the error: failed to get secret 'router-certs-default' in namespace 'openshift-ingress': secrets "router-certs-default" not found E0704 18:14:49.037579 1 gather.go:140] gatherer "clusterconfig" function "ingress_certificates" failed with the error: failed to get secret '2rbfm8mkhd6428s0ui2gban8fum4jbk0-primary-cert-bundle-secret' in namespace 'openshift-ingress-operator': secrets "2rbfm8mkhd6428s0ui2gban8fum4jbk0-primary-cert-bundle-secret" not found I0704 18:14:49.037679 1 recorder.go:75] Recording aggregated/ingress_controllers_certs with fingerprint=aca2b9eaa176ad1276d351dc4ec8342f4ea5e257c15ec4f067311ab05623da84 I0704 18:14:49.037693 1 gather.go:177] gatherer "clusterconfig" function "ingress_certificates" took 134.980541ms to process 1 records I0704 18:14:49.037589 1 tasks_processing.go:74] worker 15 stopped. I0704 18:14:49.051622 1 gather_cluster_operators.go:184] Unable to get dnsrecords.ingress.operator.openshift.io resource due to: dnsrecords.ingress.operator.openshift.io "default" not found I0704 18:14:49.062060 1 gather_cluster_operators.go:184] Unable to get dnsrecords.ingress.operator.openshift.io resource due to: dnsrecords.ingress.operator.openshift.io "default" not found I0704 18:14:49.117018 1 tasks_processing.go:74] worker 24 stopped. I0704 18:14:49.117041 1 gather.go:177] gatherer "clusterconfig" function "aggregated_monitoring_cr_names" took 211.153565ms to process 0 records I0704 18:14:49.321375 1 gather_cluster_operator_pods_and_events.go:121] Found 14 pods with 16 containers I0704 18:14:49.321389 1 gather_cluster_operator_pods_and_events.go:235] Maximum buffer size: 1572864 bytes I0704 18:14:49.321412 1 gather_cluster_operator_pods_and_events.go:365] Fetching logs for dns container dns-default-jwvqm pod in namespace openshift-dns (previous: false). I0704 18:14:49.553769 1 gather_cluster_operator_pods_and_events.go:408] Failed to fetch log for dns-default-jwvqm pod in namespace openshift-dns for failing operator dns (previous: false): "container \"dns\" in pod \"dns-default-jwvqm\" is waiting to start: ContainerCreating" I0704 18:14:49.553787 1 gather_cluster_operator_pods_and_events.go:280] Error: "container \"dns\" in pod \"dns-default-jwvqm\" is waiting to start: ContainerCreating" I0704 18:14:49.553797 1 gather_cluster_operator_pods_and_events.go:365] Fetching logs for kube-rbac-proxy container dns-default-jwvqm pod in namespace openshift-dns (previous: false). I0704 18:14:49.733246 1 gather_cluster_operator_pods_and_events.go:408] Failed to fetch log for dns-default-jwvqm pod in namespace openshift-dns for failing operator kube-rbac-proxy (previous: false): "container \"kube-rbac-proxy\" in pod \"dns-default-jwvqm\" is waiting to start: ContainerCreating" I0704 18:14:49.733267 1 gather_cluster_operator_pods_and_events.go:280] Error: "container \"kube-rbac-proxy\" in pod \"dns-default-jwvqm\" is waiting to start: ContainerCreating" I0704 18:14:49.733278 1 gather_cluster_operator_pods_and_events.go:365] Fetching logs for dns container dns-default-r75zz pod in namespace openshift-dns (previous: false). W0704 18:14:49.909603 1 gather_dvo_metrics.go:210] Failed to read the DVO metrics. Trying again. I0704 18:14:49.953241 1 gather_cluster_operator_pods_and_events.go:408] Failed to fetch log for dns-default-r75zz pod in namespace openshift-dns for failing operator dns (previous: false): "container \"dns\" in pod \"dns-default-r75zz\" is waiting to start: ContainerCreating" I0704 18:14:49.953260 1 gather_cluster_operator_pods_and_events.go:280] Error: "container \"dns\" in pod \"dns-default-r75zz\" is waiting to start: ContainerCreating" I0704 18:14:49.953271 1 gather_cluster_operator_pods_and_events.go:365] Fetching logs for kube-rbac-proxy container dns-default-r75zz pod in namespace openshift-dns (previous: false). I0704 18:14:50.018287 1 tasks_processing.go:74] worker 6 stopped. I0704 18:14:50.018307 1 gather.go:177] gatherer "clusterconfig" function "olm_operators" took 1.110384341s to process 0 records I0704 18:14:50.127098 1 gather_cluster_operator_pods_and_events.go:408] Failed to fetch log for dns-default-r75zz pod in namespace openshift-dns for failing operator kube-rbac-proxy (previous: false): "container \"kube-rbac-proxy\" in pod \"dns-default-r75zz\" is waiting to start: ContainerCreating" I0704 18:14:50.127118 1 gather_cluster_operator_pods_and_events.go:280] Error: "container \"kube-rbac-proxy\" in pod \"dns-default-r75zz\" is waiting to start: ContainerCreating" I0704 18:14:50.127131 1 gather_cluster_operator_pods_and_events.go:365] Fetching logs for dns-node-resolver container node-resolver-fg4pw pod in namespace openshift-dns (previous: false). I0704 18:14:50.331598 1 gather_cluster_operator_pods_and_events.go:280] Error: "log buffer is empty" I0704 18:14:50.331619 1 gather_cluster_operator_pods_and_events.go:365] Fetching logs for dns-node-resolver container node-resolver-nbqb5 pod in namespace openshift-dns (previous: false). I0704 18:14:50.527264 1 gather_cluster_operator_pods_and_events.go:280] Error: "log buffer is empty" I0704 18:14:50.527329 1 gather_cluster_operator_pods_and_events.go:365] Fetching logs for registry container image-registry-5664dddb59-2qxg5 pod in namespace openshift-image-registry (previous: false). I0704 18:14:50.608187 1 gather_cluster_operators.go:184] Unable to get configs.samples.operator.openshift.io resource due to: configs.samples.operator.openshift.io "cluster" not found I0704 18:14:50.720274 1 gather_cluster_operator_pods_and_events.go:280] Error: "log buffer is empty" I0704 18:14:50.720324 1 gather_cluster_operator_pods_and_events.go:365] Fetching logs for registry container image-registry-5664dddb59-x6b2v pod in namespace openshift-image-registry (previous: false). W0704 18:14:50.911187 1 gather_dvo_metrics.go:210] Failed to read the DVO metrics. Trying again. I0704 18:14:50.939678 1 gather_cluster_operator_pods_and_events.go:408] Failed to fetch log for image-registry-5664dddb59-x6b2v pod in namespace openshift-image-registry for failing operator registry (previous: false): "container \"registry\" in pod \"image-registry-5664dddb59-x6b2v\" is waiting to start: ContainerCreating" I0704 18:14:50.939701 1 gather_cluster_operator_pods_and_events.go:280] Error: "container \"registry\" in pod \"image-registry-5664dddb59-x6b2v\" is waiting to start: ContainerCreating" I0704 18:14:50.940016 1 gather_cluster_operator_pods_and_events.go:365] Fetching logs for registry container image-registry-8575cbc98d-bslfc pod in namespace openshift-image-registry (previous: false). I0704 18:14:51.003891 1 tasks_processing.go:74] worker 9 stopped. I0704 18:14:51.003961 1 recorder.go:75] Recording config/clusteroperator/console with fingerprint=cdae15bde1b143901ca300a625617a1616b27ef3793f54fbe5247b63c94e1ba5 I0704 18:14:51.004174 1 recorder.go:75] Recording config/clusteroperator/csi-snapshot-controller with fingerprint=d060998b1be78c70904b5ff0e8c78110b98ee430861509804a3f34f9203b888d I0704 18:14:51.004266 1 recorder.go:75] Recording config/clusteroperator/operator.openshift.io/csisnapshotcontroller/cluster with fingerprint=5adc514f4b63e2f1ecc68bf6f9c0af70c5eea04522a49524e102721b1c41f80e I0704 18:14:51.004320 1 recorder.go:75] Recording config/clusteroperator/dns with fingerprint=b79826099207f88cc1cba657a3338ed126497c742b6a4ce1ef9f2362e77189ce I0704 18:14:51.004372 1 recorder.go:75] Recording config/clusteroperator/operator.openshift.io/dns/default with fingerprint=9e7b4ce029030d3d8c3b49af92c556acdcc415000b40d3f969dbdc42c432b47f I0704 18:14:51.004448 1 recorder.go:75] Recording config/clusteroperator/image-registry with fingerprint=297c46b7f0dc9adf8a8c80444036520194f87e954253e9016af2d60e31b7d9c8 I0704 18:14:51.004517 1 recorder.go:75] Recording config/clusteroperator/ingress with fingerprint=2fa262c03c910cfce617695909916dd8a3efb9d01d50890ce608fa383077e6a3 I0704 18:14:51.004559 1 recorder.go:75] Recording config/clusteroperator/operator.openshift.io/ingresscontroller/openshift-ingress-operator/default with fingerprint=355bb94179aba09b7fbb7daea8086d7be06e0c7447bcbf39d53625ce7778063d I0704 18:14:51.004611 1 recorder.go:75] Recording config/clusteroperator/insights with fingerprint=8b88f1e34b032f34290ec278d30d5704fd6bc1fa5b8e3c4de2d30baeb8f869ca I0704 18:14:51.004648 1 recorder.go:75] Recording config/clusteroperator/operator.openshift.io/insightsoperator/cluster with fingerprint=e5ff11d57817f84a678f6fa9565af55bd1120227c16a21933637ab62675a6d70 I0704 18:14:51.004695 1 recorder.go:75] Recording config/clusteroperator/kube-apiserver with fingerprint=87f0617ab07b732d3247e3f25d38957fde52ff03e709c401e4e1050a116a51b1 I0704 18:14:51.004746 1 recorder.go:75] Recording config/clusteroperator/operator.openshift.io/kubeapiserver/cluster with fingerprint=51503bf0b784fcf65ea46bcaf1f72ac1a5c4d5dc211934f18f27871efed05762 I0704 18:14:51.004867 1 recorder.go:75] Recording config/clusteroperator/kube-controller-manager with fingerprint=91e9cedd1bf770c37dada749517c1c1addc1b25bb837eaddbae541c836eb1f66 I0704 18:14:51.004935 1 recorder.go:75] Recording config/clusteroperator/operator.openshift.io/kubecontrollermanager/cluster with fingerprint=ce90c0d4f367d7da085074268031798382ae7c54fdcb0a21f15a4818fe308c11 I0704 18:14:51.005006 1 recorder.go:75] Recording config/clusteroperator/kube-scheduler with fingerprint=595fed4c18151c7b82ae3a693caaf348a0e5f3b26013cf92747b21ed159e95a9 I0704 18:14:51.005094 1 recorder.go:75] Recording config/clusteroperator/operator.openshift.io/kubescheduler/cluster with fingerprint=f2940fb9fd20c19951dfc295eb363b7fba0c505f5ae61f01967a063099e6b60a I0704 18:14:51.005123 1 recorder.go:75] Recording config/clusteroperator/kube-storage-version-migrator with fingerprint=bf3ee966253eafde315905a0ee0b9bd5f54094b89626b9e806d932d5c8f1547e I0704 18:14:51.005135 1 recorder.go:75] Recording config/clusteroperator/operator.openshift.io/kubestorageversionmigrator/cluster with fingerprint=9351181aa7e6ada41ef581ab31e13516c6b934cc95710154bafb2eb222cb58db I0704 18:14:51.005169 1 recorder.go:75] Recording config/clusteroperator/monitoring with fingerprint=64587192642257122c917dfab521acf7045276cbb904d15ba22e77127930f943 I0704 18:14:51.005377 1 recorder.go:75] Recording config/clusteroperator/network with fingerprint=c1138a4f8c3f3971561fbf0cb8cd6f7c2cc6823532311e7a23d9193d52fabbb2 I0704 18:14:51.005390 1 recorder.go:75] Recording config/clusteroperator/network.operator.openshift.io/operatorpki/openshift-ovn-kubernetes/ovn with fingerprint=626a89d20e0deaed5b6dfb533acfe65f4bb1618bd200a703b62e60c5d16d94ab I0704 18:14:51.005401 1 recorder.go:75] Recording config/clusteroperator/network.operator.openshift.io/operatorpki/openshift-ovn-kubernetes/signer with fingerprint=90410b16914712b85b3c4578716ad8c0ae072e688f4cd1e022bf76f20da3506d I0704 18:14:51.005435 1 recorder.go:75] Recording config/clusteroperator/operator.openshift.io/network/cluster with fingerprint=a93d15eaecb455a0e40ecb2826eeecc1533899204ddd3c3921d15ab70af7ae75 I0704 18:14:51.005498 1 recorder.go:75] Recording config/clusteroperator/node-tuning with fingerprint=4efada0a3082c54dd9b9507a9b4f1e585301eb17c22503601491de01f73e7a6a I0704 18:14:51.005540 1 recorder.go:75] Recording config/clusteroperator/openshift-apiserver with fingerprint=456ac6a8b845bf5f153459225fbb330685be6477e3f8c09be3537243027f6ae2 I0704 18:14:51.005554 1 recorder.go:75] Recording config/clusteroperator/operator.openshift.io/openshiftapiserver/cluster with fingerprint=e712e6cf27339b441e4ed1f4cde91dbde7e952698ba93407e4457db63a4a4c76 I0704 18:14:51.005579 1 recorder.go:75] Recording config/clusteroperator/openshift-controller-manager with fingerprint=efbbf5b3aac7b28473b30fc6506454fbccfe61f00cbf685af2c7f5753843bf8a I0704 18:14:51.005591 1 recorder.go:75] Recording config/clusteroperator/operator.openshift.io/openshiftcontrollermanager/cluster with fingerprint=d71a0f4672f9b45d9fc8293bf1687afc650fd28d32e2e30de27523fe7b4eadf7 I0704 18:14:51.005611 1 recorder.go:75] Recording config/clusteroperator/openshift-samples with fingerprint=aa44429cddf3ed5fbb8477dafb882900b0043a2a4fc7da444fd51035ceb88a06 I0704 18:14:51.005716 1 recorder.go:75] Recording config/clusteroperator/operator-lifecycle-manager with fingerprint=12290029e4ff3924275e8d5d41cfe510b6350d0aa1fb520289910bf9b51007f3 I0704 18:14:51.005759 1 recorder.go:75] Recording config/clusteroperator/operator-lifecycle-manager-catalog with fingerprint=c85ed591e7abfae85c66b035eea542331bdd756970312292b2986a4e59608b9f I0704 18:14:51.005803 1 recorder.go:75] Recording config/clusteroperator/operator-lifecycle-manager-packageserver with fingerprint=9c7c3e8ea984282d6f72aed733f9d0a3a8802529382ed998053675b80cfd6f9b I0704 18:14:51.005852 1 recorder.go:75] Recording config/clusteroperator/service-ca with fingerprint=248d6d1a75986b9ead342de0d73333026cc9082b7b3aad27e54cb597eca0f3a6 I0704 18:14:51.005928 1 recorder.go:75] Recording config/clusteroperator/storage with fingerprint=3720d08e9b531c74690c100ddf577ff950bbbd1b83e43d7289130d6fe2c51695 I0704 18:14:51.005982 1 recorder.go:75] Recording config/clusteroperator/operator.openshift.io/clustercsidriver/ebs.csi.aws.com with fingerprint=510064d6f6bcced87ab5bd2ddaff3d0edd7f93f4a4f7af2641f29fc53ffab21e I0704 18:14:51.006031 1 recorder.go:75] Recording config/clusteroperator/operator.openshift.io/storage/cluster with fingerprint=8e480f8c1ce1b39baac42d8ec780c57c2592929ae0c801b61ffad49ba13f33ad I0704 18:14:51.006045 1 gather.go:177] gatherer "clusterconfig" function "operators" took 2.099952674s to process 36 records I0704 18:14:51.129221 1 gather_cluster_operator_pods_and_events.go:408] Failed to fetch log for image-registry-8575cbc98d-bslfc pod in namespace openshift-image-registry for failing operator registry (previous: false): "container \"registry\" in pod \"image-registry-8575cbc98d-bslfc\" is waiting to start: ContainerCreating" I0704 18:14:51.129314 1 gather_cluster_operator_pods_and_events.go:280] Error: "container \"registry\" in pod \"image-registry-8575cbc98d-bslfc\" is waiting to start: ContainerCreating" I0704 18:14:51.129353 1 gather_cluster_operator_pods_and_events.go:365] Fetching logs for node-ca container node-ca-wqpnq pod in namespace openshift-image-registry (previous: false). I0704 18:14:51.326389 1 gather_cluster_operator_pods_and_events.go:280] Error: "log buffer is empty" I0704 18:14:51.326408 1 gather_cluster_operator_pods_and_events.go:365] Fetching logs for node-ca container node-ca-x6gxq pod in namespace openshift-image-registry (previous: false). I0704 18:14:51.529274 1 gather_cluster_operator_pods_and_events.go:280] Error: "log buffer is empty" I0704 18:14:51.529296 1 gather_cluster_operator_pods_and_events.go:365] Fetching logs for router container router-default-555df4478-25st5 pod in namespace openshift-ingress (previous: false). I0704 18:14:51.725951 1 gather_cluster_operator_pods_and_events.go:408] Failed to fetch log for router-default-555df4478-25st5 pod in namespace openshift-ingress for failing operator router (previous: false): "container \"router\" in pod \"router-default-555df4478-25st5\" is waiting to start: ContainerCreating" I0704 18:14:51.725970 1 gather_cluster_operator_pods_and_events.go:280] Error: "container \"router\" in pod \"router-default-555df4478-25st5\" is waiting to start: ContainerCreating" I0704 18:14:51.725981 1 gather_cluster_operator_pods_and_events.go:365] Fetching logs for router container router-default-555df4478-5d7c8 pod in namespace openshift-ingress (previous: false). W0704 18:14:51.911237 1 gather_dvo_metrics.go:210] Failed to read the DVO metrics. Trying again. I0704 18:14:51.931456 1 gather_cluster_operator_pods_and_events.go:408] Failed to fetch log for router-default-555df4478-5d7c8 pod in namespace openshift-ingress for failing operator router (previous: false): "container \"router\" in pod \"router-default-555df4478-5d7c8\" is waiting to start: ContainerCreating" I0704 18:14:51.931539 1 gather_cluster_operator_pods_and_events.go:280] Error: "container \"router\" in pod \"router-default-555df4478-5d7c8\" is waiting to start: ContainerCreating" I0704 18:14:51.931560 1 gather_cluster_operator_pods_and_events.go:365] Fetching logs for router container router-default-567659f4f5-6h6sx pod in namespace openshift-ingress (previous: false). I0704 18:14:52.127569 1 gather_cluster_operator_pods_and_events.go:408] Failed to fetch log for router-default-567659f4f5-6h6sx pod in namespace openshift-ingress for failing operator router (previous: false): "container \"router\" in pod \"router-default-567659f4f5-6h6sx\" is waiting to start: ContainerCreating" I0704 18:14:52.127636 1 gather_cluster_operator_pods_and_events.go:280] Error: "container \"router\" in pod \"router-default-567659f4f5-6h6sx\" is waiting to start: ContainerCreating" I0704 18:14:52.127673 1 gather_cluster_operator_pods_and_events.go:365] Fetching logs for serve-healthcheck-canary container ingress-canary-dpqq4 pod in namespace openshift-ingress-canary (previous: false). I0704 18:14:52.329207 1 gather_cluster_operator_pods_and_events.go:408] Failed to fetch log for ingress-canary-dpqq4 pod in namespace openshift-ingress-canary for failing operator serve-healthcheck-canary (previous: false): "container \"serve-healthcheck-canary\" in pod \"ingress-canary-dpqq4\" is waiting to start: ContainerCreating" I0704 18:14:52.329227 1 gather_cluster_operator_pods_and_events.go:280] Error: "container \"serve-healthcheck-canary\" in pod \"ingress-canary-dpqq4\" is waiting to start: ContainerCreating" I0704 18:14:52.329241 1 gather_cluster_operator_pods_and_events.go:365] Fetching logs for serve-healthcheck-canary container ingress-canary-k75v4 pod in namespace openshift-ingress-canary (previous: false). I0704 18:14:52.526768 1 gather_cluster_operator_pods_and_events.go:408] Failed to fetch log for ingress-canary-k75v4 pod in namespace openshift-ingress-canary for failing operator serve-healthcheck-canary (previous: false): "container \"serve-healthcheck-canary\" in pod \"ingress-canary-k75v4\" is waiting to start: ContainerCreating" I0704 18:14:52.526785 1 gather_cluster_operator_pods_and_events.go:280] Error: "container \"serve-healthcheck-canary\" in pod \"ingress-canary-k75v4\" is waiting to start: ContainerCreating" I0704 18:14:52.526805 1 tasks_processing.go:74] worker 19 stopped. I0704 18:14:52.526907 1 recorder.go:75] Recording events/openshift-dns-operator with fingerprint=338b5f41432833087fe3ed9f7346b2668ee200660e46ac9b06f80c4895e58975 I0704 18:14:52.526947 1 recorder.go:75] Recording events/openshift-dns with fingerprint=5d3f862fdf444b63b146761ad1a95973d7d1eda079dcabaaca347d4e79f08a4f I0704 18:14:52.527053 1 recorder.go:75] Recording events/openshift-image-registry with fingerprint=97859fd6eba956180e8a2908596b9b22ddef2272e07375fc74388a72648f93df I0704 18:14:52.527099 1 recorder.go:75] Recording events/openshift-ingress-operator with fingerprint=6c99c8fc13f6cf2097757870013d01c60a64cb286d0eb44e28ec29707cc12b25 I0704 18:14:52.527148 1 recorder.go:75] Recording events/openshift-ingress with fingerprint=8103e54f1079983cdc552bc7b8986e07eab2db954254448156cb983d52ceac43 I0704 18:14:52.527162 1 recorder.go:75] Recording events/openshift-ingress-canary with fingerprint=26e54736f3e44c88a022d981a7fd815280584a91067f100d26fb588d7ad7a931 I0704 18:14:52.527323 1 recorder.go:75] Recording config/pod/openshift-image-registry/image-registry-5664dddb59-2qxg5 with fingerprint=fc0f5edf46de2d7fa9b8150a608beed1704cd0d0e41c09fb32e95d04838e196b I0704 18:14:52.527449 1 recorder.go:75] Recording config/pod/openshift-image-registry/image-registry-5664dddb59-x6b2v with fingerprint=8fe04e3d0caf58fe1678077e91b19fcc3d35dc5a21b4256c66a0bd739437ffb9 I0704 18:14:52.527574 1 recorder.go:75] Recording config/pod/openshift-image-registry/image-registry-8575cbc98d-bslfc with fingerprint=e6584df18353f7852a93327193d11c3b56f9d15396caee5cbecfddf14c7df2b2 I0704 18:14:52.527598 1 gather.go:177] gatherer "clusterconfig" function "operators_pods_and_events" took 3.64115839s to process 9 records W0704 18:14:52.910745 1 gather_dvo_metrics.go:210] Failed to read the DVO metrics. Trying again. W0704 18:14:53.906325 1 gather_dvo_metrics.go:117] Unable to read metrics from endpoint "http://deployment-validation-operator-metrics.openshift-deployment-validation-operator.svc:8383": DVO metrics service was not available within the 5s timeout: context deadline exceeded I0704 18:14:53.906358 1 tasks_processing.go:74] worker 5 stopped. E0704 18:14:53.906373 1 gather.go:140] gatherer "clusterconfig" function "dvo_metrics" failed with the error: DVO metrics service was not available within the 5s timeout: context deadline exceeded I0704 18:14:53.906388 1 recorder.go:75] Recording config/dvo_metrics with fingerprint=e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855 W0704 18:14:53.906410 1 gather.go:155] issue recording gatherer "clusterconfig" function "dvo_metrics" result "config/dvo_metrics" because of the warning: warning: the record with the same fingerprint "e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855" was already recorded at path "config/configmaps/openshift-config/openshift-service-ca.crt/service-ca.crt", recording another one with a different path "config/dvo_metrics" I0704 18:14:53.906425 1 gather.go:177] gatherer "clusterconfig" function "dvo_metrics" took 5.019207739s to process 1 records I0704 18:15:01.510395 1 tasks_processing.go:74] worker 16 stopped. I0704 18:15:01.510434 1 recorder.go:75] Recording config/installplans with fingerprint=7b887df561a3a9e6ef0dc672845aa5d56e348505006b7496d3a2f83892b0c95b I0704 18:15:01.510447 1 gather.go:177] gatherer "clusterconfig" function "install_plans" took 12.624484992s to process 1 records I0704 18:15:02.053756 1 configmapobserver.go:84] configmaps "insights-config" not found I0704 18:15:02.293332 1 tasks_processing.go:74] worker 2 stopped. I0704 18:15:02.293759 1 recorder.go:75] Recording config/serviceaccounts with fingerprint=6eea9bca9c2d016e4128bf2b374fefdd9610f4d08552d21add7f6bb399a3af40 I0704 18:15:02.293780 1 gather.go:177] gatherer "clusterconfig" function "service_accounts" took 13.406679667s to process 1 records E0704 18:15:02.293855 1 periodic.go:247] "Unhandled Error" err="clusterconfig failed after 13.409s with: function \"support_secret\" failed with an error, function \"machines\" failed with an error, function \"pod_network_connectivity_checks\" failed with an error, function \"machine_healthchecks\" failed with an error, function \"config_maps\" failed with an error, function \"ingress_certificates\" failed with an error, function \"dvo_metrics\" failed with an error" I0704 18:15:02.294963 1 controllerstatus.go:89] name=periodic-clusterconfig healthy=false reason=PeriodicGatherFailed message=Source clusterconfig could not be retrieved: function "support_secret" failed with an error, function "machines" failed with an error, function "pod_network_connectivity_checks" failed with an error, function "machine_healthchecks" failed with an error, function "config_maps" failed with an error, function "ingress_certificates" failed with an error, function "dvo_metrics" failed with an error I0704 18:15:02.294977 1 periodic.go:209] Running workloads gatherer I0704 18:15:02.294995 1 tasks_processing.go:45] number of workers: 2 I0704 18:15:02.295010 1 tasks_processing.go:69] worker 1 listening for tasks. I0704 18:15:02.295016 1 tasks_processing.go:71] worker 1 working on workload_info task. I0704 18:15:02.295019 1 tasks_processing.go:69] worker 0 listening for tasks. I0704 18:15:02.295095 1 tasks_processing.go:71] worker 0 working on helmchart_info task. I0704 18:15:02.318221 1 tasks_processing.go:74] worker 0 stopped. I0704 18:15:02.318238 1 gather.go:177] gatherer "workloads" function "helmchart_info" took 23.104327ms to process 0 records I0704 18:15:02.320013 1 gather_workloads_info.go:278] Loaded pods in 0s, will wait 20s for image data I0704 18:15:02.327051 1 gather_workloads_info.go:387] No image sha256:2904a78e2eb73fd6a9bb94c105c2a056831fb4113fbb7b0607c50adc9d879c9b (8ms) I0704 18:15:02.334145 1 gather_workloads_info.go:387] No image sha256:04c87c054a3f366a7dfbe0a93ebb0c80a098ee16842c5794b67c1202eec61996 (7ms) I0704 18:15:02.340383 1 gather_workloads_info.go:387] No image sha256:765f0d23b637f685f98a31bd47c131b03cf72a40761a3f9a9d6320faa3c33733 (6ms) I0704 18:15:02.346842 1 gather_workloads_info.go:387] No image sha256:c940ea87e7d133d75ba0002ef00c0806825eed3db8094cdb260d1bac18127733 (6ms) I0704 18:15:02.353086 1 gather_workloads_info.go:387] No image sha256:ca1344cb64140188b7cae7bbc51fb751566c0b0c97d5e39b5850e628032c4a5e (6ms) I0704 18:15:02.359040 1 gather_workloads_info.go:387] No image sha256:943018739e3db1763c3184b460dbc409e058abbac76d57b9927faad317be85e4 (6ms) I0704 18:15:02.365415 1 gather_workloads_info.go:387] No image sha256:1a2532940843248c57d52141185dd71fbc393ab28b65d48f682038632c1dbbad (6ms) I0704 18:15:02.371966 1 gather_workloads_info.go:387] No image sha256:5a95c19d82767e0235b4edb4a0536482c816904897aae1dc3eb255cb52b87a9f (7ms) I0704 18:15:02.378490 1 gather_workloads_info.go:387] No image sha256:0a99240166165eb5718e7516a43282fe32df9c7c5e809b31b58abe44e42ff94d (7ms) I0704 18:15:02.390351 1 gather_workloads_info.go:387] No image sha256:03cf4cd7ef1518610c6c7b3ad27d1622d82e98e3dc6e3f8e5d0fceb5c8d3786e (12ms) I0704 18:15:02.426503 1 gather_workloads_info.go:387] No image sha256:289816958633a763a72dbc44e1dad40466223164e7e253039514f0d974ea5d21 (36ms) I0704 18:15:02.528245 1 gather_workloads_info.go:387] No image sha256:b3909bf664c77097f75b3768830863d642eed3815dab2bfb4415c771ca2d5007 (102ms) I0704 18:15:02.627775 1 gather_workloads_info.go:387] No image sha256:a498046d64605bcccee2440aa4f04a4602baaae263cf01d977ec5208e876b1fd (100ms) I0704 18:15:02.727934 1 gather_workloads_info.go:387] No image sha256:91828234f107c068c8a4966d08370ae7b73e637651dbc6d92c18c4553402c22c (100ms) I0704 18:15:02.828807 1 gather_workloads_info.go:387] No image sha256:36b9e89c3cfcf1ab9ae500486e38afb6862cba48cb0b4d84a09508ab8f3d299f (101ms) I0704 18:15:02.929515 1 gather_workloads_info.go:387] No image sha256:4556896f77307821531ef91b7b7faccb82b824ea695693b2989f597f0deca038 (101ms) I0704 18:15:03.027224 1 gather_workloads_info.go:387] No image sha256:95418d0ba6fc0aa96647d97cf6530967f7d7ae97a82ce7f7e41c8e44310b0b72 (98ms) I0704 18:15:03.131784 1 gather_workloads_info.go:387] No image sha256:2e57e192c3c1240fd935dcd55c8fde5e70e78bf81d6176c96edf21fafe59f8ba (105ms) I0704 18:15:03.227841 1 gather_workloads_info.go:387] No image sha256:5f0b67cfbbc381243fb91ccc17345b56d05f4d717c667e8c644e5bf05633ba71 (96ms) I0704 18:15:03.327805 1 gather_workloads_info.go:387] No image sha256:a0105d1eb62cf6ac9e5e2ef28d3e89bf6dc514bc594fc7090fe5a5ee18a09c87 (100ms) I0704 18:15:03.427755 1 gather_workloads_info.go:387] No image sha256:521712486e2c6e3c020dad6a1cb340db8e55665b69f7c208fab9cd9e965fd588 (100ms) I0704 18:15:03.527823 1 gather_workloads_info.go:387] No image sha256:56a85660a445eced5c79a595a0eccf590087c5672d50f49d4c25ad52f9a44f04 (100ms) I0704 18:15:03.628038 1 gather_workloads_info.go:387] No image sha256:7b31223098f08328f5ddea8e5b871dbbd5f5a61ec550e8956f66793c0c6031a9 (100ms) I0704 18:15:03.728875 1 gather_workloads_info.go:387] No image sha256:a56211d075aa43cbb491f669a5b2e46ee023dc95b7d51dbac28f463948c5ad61 (101ms) I0704 18:15:03.828101 1 gather_workloads_info.go:387] No image sha256:2598489729a4b258e4ecda4a06f6875133f2a10ced5c5241f8a57a8a05418e36 (99ms) I0704 18:15:03.927470 1 gather_workloads_info.go:387] No image sha256:a258c226562adb14e3a163a1940938526ee6a0928982a7667d85d9a7334ce639 (99ms) I0704 18:15:04.027100 1 gather_workloads_info.go:387] No image sha256:653c666f842c13e0baae2e89a9b1efe0e2ef56f621ffb5b32005115d2a26ab8c (100ms) I0704 18:15:04.127986 1 gather_workloads_info.go:387] No image sha256:e84cb128d930bd1ab867cc89b7b7bf2b2c0e41105ab93b5381069945b3ee9c57 (101ms) I0704 18:15:04.229103 1 gather_workloads_info.go:387] No image sha256:2e564f336c77116053f34d4201d364d8da04e789cfffa0ea422574c95f2d6404 (101ms) I0704 18:15:04.328300 1 gather_workloads_info.go:387] No image sha256:695cf2f0cc07683c2a3ce1eaf3e56fe18abc6e2bac716f7d9843f5d173b9df52 (99ms) I0704 18:15:04.427313 1 gather_workloads_info.go:387] No image sha256:47154813651033d59751fb655a384dbffb64dd26f10bd7f3be0c3128d0486356 (99ms) I0704 18:15:04.527064 1 gather_workloads_info.go:387] No image sha256:7adc1eab05d6724c76ba751f6df816b08d6e70b78dee9eb94fa6fd9690542c98 (100ms) I0704 18:15:04.527096 1 tasks_processing.go:74] worker 1 stopped. E0704 18:15:04.527107 1 gather.go:140] gatherer "workloads" function "workload_info" failed with the error: no running pods found for the insights-runtime-extractor statefulset I0704 18:15:04.527475 1 recorder.go:75] Recording config/workload_info with fingerprint=8c5e64581cc8a52bd8af91ed93dae6d27a9920692498bc810a8c00f76fd5834f I0704 18:15:04.527492 1 gather.go:177] gatherer "workloads" function "workload_info" took 2.232072152s to process 1 records E0704 18:15:04.527521 1 periodic.go:247] "Unhandled Error" err="workloads failed after 2.232s with: function \"workload_info\" failed with an error" I0704 18:15:04.528632 1 controllerstatus.go:89] name=periodic-workloads healthy=false reason=PeriodicGatherFailed message=Source workloads could not be retrieved: function "workload_info" failed with an error I0704 18:15:04.528648 1 periodic.go:209] Running conditional gatherer I0704 18:15:04.534833 1 requests.go:294] Making HTTP GET request at: https://console.redhat.com/api/gathering/v2/4.20.8/gathering_rules I0704 18:15:04.541637 1 conditional_gatherer.go:107] Get "https://console.redhat.com/api/gathering/v2/4.20.8/gathering_rules": dial tcp: lookup console.redhat.com on 172.30.0.10:53: read udp 10.129.0.12:35720->172.30.0.10:53: read: connection refused E0704 18:15:04.541929 1 conditional_gatherer.go:322] unable to update alerts cache: open /var/run/configmaps/service-ca-bundle/service-ca.crt: no such file or directory I0704 18:15:04.541994 1 conditional_gatherer.go:384] updating version cache for conditional gatherer I0704 18:15:04.549295 1 conditional_gatherer.go:392] cluster version is '4.20.8' E0704 18:15:04.549308 1 conditional_gatherer.go:209] error checking conditions for a gathering rule: alerts cache is missing E0704 18:15:04.549313 1 conditional_gatherer.go:209] error checking conditions for a gathering rule: alerts cache is missing E0704 18:15:04.549316 1 conditional_gatherer.go:209] error checking conditions for a gathering rule: alerts cache is missing E0704 18:15:04.549320 1 conditional_gatherer.go:209] error checking conditions for a gathering rule: alerts cache is missing E0704 18:15:04.549323 1 conditional_gatherer.go:209] error checking conditions for a gathering rule: alerts cache is missing E0704 18:15:04.549326 1 conditional_gatherer.go:209] error checking conditions for a gathering rule: alerts cache is missing E0704 18:15:04.549329 1 conditional_gatherer.go:209] error checking conditions for a gathering rule: alerts cache is missing E0704 18:15:04.549332 1 conditional_gatherer.go:209] error checking conditions for a gathering rule: alerts cache is missing E0704 18:15:04.549335 1 conditional_gatherer.go:209] error checking conditions for a gathering rule: alerts cache is missing I0704 18:15:04.549349 1 tasks_processing.go:45] number of workers: 3 I0704 18:15:04.549361 1 tasks_processing.go:69] worker 2 listening for tasks. I0704 18:15:04.549365 1 tasks_processing.go:71] worker 2 working on conditional_gatherer_rules task. I0704 18:15:04.549371 1 tasks_processing.go:69] worker 0 listening for tasks. I0704 18:15:04.549381 1 tasks_processing.go:69] worker 1 listening for tasks. I0704 18:15:04.549386 1 tasks_processing.go:71] worker 0 working on remote_configuration task. I0704 18:15:04.549388 1 tasks_processing.go:74] worker 1 stopped. I0704 18:15:04.549407 1 tasks_processing.go:71] worker 2 working on rapid_container_logs task. I0704 18:15:04.549458 1 recorder.go:75] Recording insights-operator/conditional-gatherer-rules with fingerprint=7034af97e7e41c22e4b775abdd4b9066c8ebb19da33eb7f69f39bfd2eb5f6406 I0704 18:15:04.549471 1 gather.go:177] gatherer "conditional" function "conditional_gatherer_rules" took 833ns to process 1 records I0704 18:15:04.549508 1 recorder.go:75] Recording insights-operator/remote-configuration with fingerprint=0394430c431eec4d48bb1811a90918e95161d2282c59af26f2473613cc0959db I0704 18:15:04.549524 1 gather.go:177] gatherer "conditional" function "remote_configuration" took 1.299µs to process 1 records I0704 18:15:04.549530 1 tasks_processing.go:74] worker 0 stopped. I0704 18:15:04.549701 1 tasks_processing.go:74] worker 2 stopped. I0704 18:15:04.549722 1 gather.go:177] gatherer "conditional" function "rapid_container_logs" took 271.435µs to process 0 records I0704 18:15:04.549767 1 controllerstatus.go:89] name=periodic-conditional healthy=false reason=NotAvailable message=Get "https://console.redhat.com/api/gathering/v2/4.20.8/gathering_rules": dial tcp: lookup console.redhat.com on 172.30.0.10:53: read udp 10.129.0.12:35720->172.30.0.10:53: read: connection refused I0704 18:15:04.549783 1 recorder.go:75] Recording insights-operator/remote-configuration.json with fingerprint=359de9c990c741675cec72fda96b5c3682221efdb4799f5eaa6e9805bcd3b5c1 I0704 18:15:04.572450 1 recorder.go:75] Recording insights-operator/gathers with fingerprint=fd938f7cffaef8d9e0774a62f79a95c8fc230a74e3f4e277868d6e8e86dbda79 I0704 18:15:04.572587 1 diskrecorder.go:70] Writing 104 records to /var/lib/insights-operator/insights-2026-07-04-181504.tar.gz I0704 18:15:04.579937 1 diskrecorder.go:51] Wrote 104 records to disk in 7ms I0704 18:15:04.579971 1 periodic.go:278] Gathering cluster info every 2h0m0s I0704 18:15:04.579985 1 periodic.go:279] Configuration is dataReporting: interval: 2h0m0s, uploadEndpoint: https://console.redhat.com/api/ingress/v1/upload, storagePath: /var/lib/insights-operator, downloadEndpoint: https://console.redhat.com/api/insights-results-aggregator/v2/cluster/%s/reports, conditionalGathererEndpoint: https://console.redhat.com/api/gathering/v2/%s/gathering_rules, obfuscation: [] sca: disabled: false, endpoint: https://api.openshift.com/api/accounts_mgmt/v1/entitlement_certificates, interval: 8h0m0s alerting: disabled: false clusterTransfer: endpoint: https://api.openshift.com/api/accounts_mgmt/v1/cluster_transfers/, interval: 12h0m0s proxy: httpProxy: , httpsProxy: , noProxy: I0704 18:15:05.593448 1 configmapobserver.go:84] configmaps "insights-config" not found I0704 18:15:05.794069 1 configmapobserver.go:84] configmaps "insights-config" not found I0704 18:15:23.258197 1 configmapobserver.go:84] configmaps "insights-config" not found I0704 18:16:13.271180 1 observer_polling.go:111] Observed file "/var/run/secrets/serving-cert/tls.crt" has been created (hash="99ee009fd90b33b59f9f17482040b153c955b64572c400174088fef7171e688f") W0704 18:16:13.271214 1 builder.go:160] Restart triggered because of file /var/run/secrets/serving-cert/tls.crt was created I0704 18:16:13.271262 1 observer_polling.go:111] Observed file "/var/run/secrets/serving-cert/tls.key" has been created (hash="9c9c193911f08702a4650281e9f7966d2710f6e740d47db16cd0bfd4a2b4bb1f") I0704 18:16:13.271286 1 genericapiserver.go:693] "[graceful-termination] pre-shutdown hooks completed" name="PreShutdownHooksStopped" I0704 18:16:13.271326 1 simple_featuregate_reader.go:177] Shutting down feature-gate-detector I0704 18:16:13.271333 1 genericapiserver.go:548] "[graceful-termination] shutdown event" name="ShutdownInitiated" I0704 18:16:13.271346 1 observer_polling.go:111] Observed file "/var/run/configmaps/service-ca-bundle/service-ca.crt" has been created (hash="81dcf046d90ce453a52a34ab25d332c17253dad2726944de3f184dc9a39d922d") I0704 18:16:13.271363 1 periodic.go:170] Shutting down I0704 18:16:13.271367 1 genericapiserver.go:651] "[graceful-termination] not going to wait for active watch request(s) to drain" I0704 18:16:13.271370 1 base_controller.go:181] Shutting down ConfigController ... I0704 18:16:13.271387 1 base_controller.go:123] Shutting down worker of ConfigController controller ... I0704 18:16:13.271396 1 base_controller.go:113] All ConfigController workers have been terminated I0704 18:16:13.271407 1 base_controller.go:181] Shutting down LoggingSyncer ...