W0512 17:39:20.496683 1 cmd.go:257] Using insecure, self-signed certificates I0512 17:39:21.407430 1 start.go:138] Unable to read service ca bundle: open /var/run/configmaps/service-ca-bundle/service-ca.crt: no such file or directory I0512 17:39:21.407747 1 observer_polling.go:159] Starting file observer I0512 17:39:22.024269 1 operator.go:76] Starting insights-operator v0.0.0-master+$Format:%H$ I0512 17:39:22.024466 1 legacy_config.go:327] Current config: {"report":false,"storagePath":"/var/lib/insights-operator","interval":"2h","endpoint":"https://console.redhat.com/api/ingress/v1/upload","conditionalGathererEndpoint":"https://console.redhat.com/api/gathering/v2/%s/gathering_rules","pull_report":{"endpoint":"https://console.redhat.com/api/insights-results-aggregator/v2/cluster/%s/reports","delay":"60s","timeout":"3000s","min_retry":"30s"},"impersonate":"system:serviceaccount:openshift-insights:gather","enableGlobalObfuscation":false,"ocm":{"scaEndpoint":"https://api.openshift.com/api/accounts_mgmt/v1/entitlement_certificates","scaInterval":"8h","scaDisabled":false,"clusterTransferEndpoint":"https://api.openshift.com/api/accounts_mgmt/v1/cluster_transfers/","clusterTransferInterval":"12h"},"disableInsightsAlerts":false,"processingStatusEndpoint":"https://console.redhat.com/api/insights-results-aggregator/v2/cluster/%s/request/%s/status","reportEndpointTechPreview":"https://console.redhat.com/api/insights-results-aggregator/v2/cluster/%s/request/%s/report"} I0512 17:39:22.025029 1 secure_serving.go:57] Forcing use of http/1.1 only W0512 17:39:22.025052 1 secure_serving.go:69] Use of insecure cipher 'TLS_ECDHE_ECDSA_WITH_AES_128_CBC_SHA256' detected. W0512 17:39:22.025056 1 secure_serving.go:69] Use of insecure cipher 'TLS_ECDHE_RSA_WITH_AES_128_CBC_SHA256' detected. W0512 17:39:22.025061 1 secure_serving.go:69] Use of insecure cipher 'TLS_RSA_WITH_AES_128_GCM_SHA256' detected. W0512 17:39:22.025064 1 secure_serving.go:69] Use of insecure cipher 'TLS_RSA_WITH_AES_256_GCM_SHA384' detected. W0512 17:39:22.025066 1 secure_serving.go:69] Use of insecure cipher 'TLS_RSA_WITH_AES_128_CBC_SHA' detected. W0512 17:39:22.025069 1 secure_serving.go:69] Use of insecure cipher 'TLS_RSA_WITH_AES_256_CBC_SHA' detected. I0512 17:39:22.025101 1 simple_featuregate_reader.go:171] Starting feature-gate-detector I0512 17:39:22.029552 1 operator.go:141] FeatureGates initialized: knownFeatureGates=[AdditionalRoutingCapabilities AdminNetworkPolicy AlibabaPlatform AzureWorkloadIdentity BuildCSIVolumes CPMSMachineNamePrefix ConsolePluginContentSecurityPolicy ExternalOIDC ExternalOIDCWithUIDAndExtraClaimMappings GCPClusterHostedDNSInstall GatewayAPI GatewayAPIController HighlyAvailableArbiter HyperShiftOnlyDynamicResourceAllocation ImageStreamImportMode ImageVolume KMSv1 MachineConfigNodes ManagedBootImages ManagedBootImagesAWS ManagedBootImagesAzure ManagedBootImagesvSphere MetricsCollectionProfiles NetworkDiagnosticsConfig NetworkLiveMigration NetworkSegmentation PinnedImages PreconfiguredUDNAddresses ProcMountType RouteAdvertisements RouteExternalCertificate ServiceAccountTokenNodeBinding SigstoreImageVerification SigstoreImageVerificationPKI StoragePerformantSecurityPolicy UpgradeStatus UserNamespacesPodSecurityStandards UserNamespacesSupport VSphereMultiDisk VSphereMultiNetworks VolumeAttributesClass AWSClusterHostedDNS AWSClusterHostedDNSInstall AWSDedicatedHosts AWSDualStackInstall AWSServiceLBNetworkSecurityGroup AutomatedEtcdBackup AzureClusterHostedDNSInstall AzureDedicatedHosts AzureDualStackInstall AzureMultiDisk BootImageSkewEnforcement BootcNodeManagement CBORServingAndStorage CRDCompatibilityRequirementOperator ClientsAllowCBOR ClientsPreferCBOR ClusterAPIInstall ClusterAPIInstallIBMCloud ClusterAPIMachineManagement ClusterAPIMachineManagementVSphere ClusterMonitoringConfig ClusterVersionOperatorConfiguration DNSNameResolver DualReplica DyanmicServiceEndpointIBMCloud EtcdBackendQuota EventTTL EventedPLEG Example Example2 ExternalSnapshotMetadata GCPClusterHostedDNS GCPCustomAPIEndpoints GCPCustomAPIEndpointsInstall GCPDualStackInstall ImageModeStatusReporting IngressControllerDynamicConfigurationManager InsightsConfig InsightsOnDemandDataGather IrreconcilableMachineConfig KMSEncryption KMSEncryptionProvider MachineAPIMigration MachineAPIOperatorDisableMachineHealthCheckController ManagedBootImagesCPMS MaxUnavailableStatefulSet MinimumKubeletVersion MixedCPUsAllocation MultiArchInstallAzure MultiDiskSetup MutableCSINodeAllocatableCount MutatingAdmissionPolicy NewOLM NewOLMBoxCutterRuntime NewOLMCatalogdAPIV1Metas NewOLMOwnSingleNamespace NewOLMPreflightPermissionChecks NewOLMWebhookProviderOpenshiftServiceCA NoRegistryClusterInstall NutanixMultiSubnets OSStreams OVNObservability OnPremDNSRecords OpenShiftPodSecurityAdmission ProvisioningRequestAvailable SELinuxMount ShortCertRotation SignatureStores TranslateStreamCloseWebsocketRequests VSphereConfigurableMaxAllowedBlockVolumesPerNode VSphereHostVMGroupZonal VSphereMixedNodeEnv VolumeGroupSnapshot] I0512 17:39:22.029621 1 event.go:377] Event(v1.ObjectReference{Kind:"Deployment", Namespace:"openshift-insights", Name:"insights-operator", UID:"d2ef06bd-138d-4867-96bc-1df2a57df207", APIVersion:"apps/v1", ResourceVersion:"", FieldPath:""}): type: 'Normal' reason: 'FeatureGatesInitialized' FeatureGates updated to featuregates.Features{Enabled:[]v1.FeatureGateName{"AdditionalRoutingCapabilities", "AdminNetworkPolicy", "AlibabaPlatform", "AzureWorkloadIdentity", "BuildCSIVolumes", "CPMSMachineNamePrefix", "ConsolePluginContentSecurityPolicy", "ExternalOIDC", "ExternalOIDCWithUIDAndExtraClaimMappings", "GCPClusterHostedDNSInstall", "GatewayAPI", "GatewayAPIController", "HighlyAvailableArbiter", "HyperShiftOnlyDynamicResourceAllocation", "ImageStreamImportMode", "ImageVolume", "KMSv1", "MachineConfigNodes", "ManagedBootImages", "ManagedBootImagesAWS", "ManagedBootImagesAzure", "ManagedBootImagesvSphere", "MetricsCollectionProfiles", "NetworkDiagnosticsConfig", "NetworkLiveMigration", "NetworkSegmentation", "PinnedImages", "PreconfiguredUDNAddresses", "ProcMountType", "RouteAdvertisements", "RouteExternalCertificate", "ServiceAccountTokenNodeBinding", "SigstoreImageVerification", "SigstoreImageVerificationPKI", "StoragePerformantSecurityPolicy", "UpgradeStatus", "UserNamespacesPodSecurityStandards", "UserNamespacesSupport", "VSphereMultiDisk", "VSphereMultiNetworks", "VolumeAttributesClass"}, Disabled:[]v1.FeatureGateName{"AWSClusterHostedDNS", "AWSClusterHostedDNSInstall", "AWSDedicatedHosts", "AWSDualStackInstall", "AWSServiceLBNetworkSecurityGroup", "AutomatedEtcdBackup", "AzureClusterHostedDNSInstall", "AzureDedicatedHosts", "AzureDualStackInstall", "AzureMultiDisk", "BootImageSkewEnforcement", "BootcNodeManagement", "CBORServingAndStorage", "CRDCompatibilityRequirementOperator", "ClientsAllowCBOR", "ClientsPreferCBOR", "ClusterAPIInstall", "ClusterAPIInstallIBMCloud", "ClusterAPIMachineManagement", "ClusterAPIMachineManagementVSphere", "ClusterMonitoringConfig", "ClusterVersionOperatorConfiguration", "DNSNameResolver", "DualReplica", "DyanmicServiceEndpointIBMCloud", "EtcdBackendQuota", "EventTTL", "EventedPLEG", "Example", "Example2", "ExternalSnapshotMetadata", "GCPClusterHostedDNS", "GCPCustomAPIEndpoints", "GCPCustomAPIEndpointsInstall", "GCPDualStackInstall", "ImageModeStatusReporting", "IngressControllerDynamicConfigurationManager", "InsightsConfig", "InsightsOnDemandDataGather", "IrreconcilableMachineConfig", "KMSEncryption", "KMSEncryptionProvider", "MachineAPIMigration", "MachineAPIOperatorDisableMachineHealthCheckController", "ManagedBootImagesCPMS", "MaxUnavailableStatefulSet", "MinimumKubeletVersion", "MixedCPUsAllocation", "MultiArchInstallAzure", "MultiDiskSetup", "MutableCSINodeAllocatableCount", "MutatingAdmissionPolicy", "NewOLM", "NewOLMBoxCutterRuntime", "NewOLMCatalogdAPIV1Metas", "NewOLMOwnSingleNamespace", "NewOLMPreflightPermissionChecks", "NewOLMWebhookProviderOpenshiftServiceCA", "NoRegistryClusterInstall", "NutanixMultiSubnets", "OSStreams", "OVNObservability", "OnPremDNSRecords", "OpenShiftPodSecurityAdmission", "ProvisioningRequestAvailable", "SELinuxMount", "ShortCertRotation", "SignatureStores", "TranslateStreamCloseWebsocketRequests", "VSphereConfigurableMaxAllowedBlockVolumesPerNode", "VSphereHostVMGroupZonal", "VSphereMixedNodeEnv", "VolumeGroupSnapshot"}} I0512 17:39:22.030628 1 requestheader_controller.go:180] Starting RequestHeaderAuthRequestController I0512 17:39:22.030706 1 shared_informer.go:313] Waiting for caches to sync for RequestHeaderAuthRequestController I0512 17:39:22.030767 1 configmap_cafile_content.go:205] "Starting controller" name="client-ca::kube-system::extension-apiserver-authentication::client-ca-file" I0512 17:39:22.030803 1 shared_informer.go:313] Waiting for caches to sync for client-ca::kube-system::extension-apiserver-authentication::client-ca-file I0512 17:39:22.030833 1 configmap_cafile_content.go:205] "Starting controller" name="client-ca::kube-system::extension-apiserver-authentication::requestheader-client-ca-file" I0512 17:39:22.030856 1 shared_informer.go:313] Waiting for caches to sync for client-ca::kube-system::extension-apiserver-authentication::requestheader-client-ca-file I0512 17:39:22.031060 1 dynamic_serving_content.go:135] "Starting controller" name="serving-cert::/tmp/serving-cert-1898473089/tls.crt::/tmp/serving-cert-1898473089/tls.key" I0512 17:39:22.031315 1 secure_serving.go:213] Serving securely on [::]:8443 I0512 17:39:22.031347 1 tlsconfig.go:243] "Starting DynamicServingCertificateController" W0512 17:39:22.035366 1 configmapobserver.go:64] Cannot get the configuration config map: configmaps "insights-config" not found. Default configuration is used. I0512 17:39:22.035404 1 secretconfigobserver.go:216] Legacy configuration set: enabled=false endpoint=https://console.redhat.com/api/ingress/v1/upload conditional_gatherer_endpoint=https://console.redhat.com/api/gathering/v2/%s/gathering_rules interval=2h0m0s token=false reportEndpoint=https://console.redhat.com/api/insights-results-aggregator/v2/cluster/%s/reports initialPollingDelay=1m0s minRetryTime=30s pollingTimeout=50m0s processingStatusEndpoint=https://console.redhat.com/api/insights-results-aggregator/v2/cluster/%s/request/%s/status I0512 17:39:22.035530 1 base_controller.go:76] Waiting for caches to sync for ConfigController I0512 17:39:22.040539 1 secretconfigobserver.go:249] Found cloud.openshift.com token I0512 17:39:22.040559 1 secretconfigobserver.go:204] Legacy configuration updated: enabled=true endpoint=https://console.redhat.com/api/ingress/v1/upload conditional_gatherer_endpoint=https://console.redhat.com/api/gathering/v2/%s/gathering_rules interval=2h0m0s token=true reportEndpoint=https://console.redhat.com/api/insights-results-aggregator/v2/cluster/%s/reports initialPollingDelay=1m0s minRetryTime=30s pollingTimeout=50m0s processingStatusEndpoint=https://console.redhat.com/api/insights-results-aggregator/v2/cluster/%s/request/%s/status I0512 17:39:22.044738 1 secretconfigobserver.go:119] support secret does not exist I0512 17:39:22.049248 1 secretconfigobserver.go:249] Found cloud.openshift.com token I0512 17:39:22.053343 1 secretconfigobserver.go:119] support secret does not exist I0512 17:39:22.058681 1 recorder.go:176] Pruning old reports every 5h9m51s, max age is 288h0m0s I0512 17:39:22.063694 1 controllerstatus.go:80] name=insightsuploader healthy=true reason= message= I0512 17:39:22.063708 1 controllerstatus.go:80] name=insightsreport healthy=true reason= message= I0512 17:39:22.063718 1 insightsuploader.go:86] Reporting status periodically to https://console.redhat.com/api/ingress/v1/upload every 2h0m0s, starting in 1m30s I0512 17:39:22.063723 1 insightsreport.go:296] Starting report retriever I0512 17:39:22.063733 1 insightsreport.go:298] Insights analysis reports will be downloaded from the https://console.redhat.com/api/insights-results-aggregator/v2/cluster/%s/reports endpoint with a delay of 1m0s I0512 17:39:22.063743 1 periodic.go:216] Running clusterconfig gatherer I0512 17:39:22.063809 1 tasks_processing.go:45] number of workers: 64 I0512 17:39:22.063832 1 tasks_processing.go:69] worker 1 listening for tasks. I0512 17:39:22.063840 1 tasks_processing.go:69] worker 9 listening for tasks. I0512 17:39:22.063842 1 tasks_processing.go:69] worker 0 listening for tasks. I0512 17:39:22.063847 1 tasks_processing.go:69] worker 6 listening for tasks. I0512 17:39:22.063849 1 tasks_processing.go:69] worker 7 listening for tasks. I0512 17:39:22.063851 1 tasks_processing.go:69] worker 5 listening for tasks. I0512 17:39:22.063857 1 tasks_processing.go:69] worker 30 listening for tasks. I0512 17:39:22.063860 1 tasks_processing.go:71] worker 5 working on version task. I0512 17:39:22.063862 1 tasks_processing.go:71] worker 30 working on openstack_dataplanenodesets task. I0512 17:39:22.063863 1 tasks_processing.go:69] worker 31 listening for tasks. I0512 17:39:22.063867 1 tasks_processing.go:69] worker 3 listening for tasks. I0512 17:39:22.063870 1 tasks_processing.go:69] worker 32 listening for tasks. I0512 17:39:22.063871 1 tasks_processing.go:69] worker 8 listening for tasks. I0512 17:39:22.063874 1 tasks_processing.go:69] worker 33 listening for tasks. I0512 17:39:22.063884 1 tasks_processing.go:69] worker 4 listening for tasks. I0512 17:39:22.063882 1 tasks_processing.go:69] worker 34 listening for tasks. I0512 17:39:22.063884 1 tasks_processing.go:69] worker 55 listening for tasks. I0512 17:39:22.063889 1 tasks_processing.go:69] worker 35 listening for tasks. I0512 17:39:22.063895 1 tasks_processing.go:69] worker 20 listening for tasks. I0512 17:39:22.063895 1 tasks_processing.go:69] worker 42 listening for tasks. I0512 17:39:22.063897 1 tasks_processing.go:69] worker 36 listening for tasks. I0512 17:39:22.063903 1 tasks_processing.go:69] worker 10 listening for tasks. I0512 17:39:22.063903 1 tasks_processing.go:69] worker 43 listening for tasks. I0512 17:39:22.063910 1 tasks_processing.go:69] worker 44 listening for tasks. I0512 17:39:22.063912 1 tasks_processing.go:69] worker 40 listening for tasks. I0512 17:39:22.063917 1 tasks_processing.go:69] worker 45 listening for tasks. I0512 17:39:22.063910 1 tasks_processing.go:69] worker 49 listening for tasks. I0512 17:39:22.063923 1 tasks_processing.go:69] worker 50 listening for tasks. I0512 17:39:22.063918 1 tasks_processing.go:69] worker 11 listening for tasks. I0512 17:39:22.063918 1 tasks_processing.go:69] worker 62 listening for tasks. I0512 17:39:22.063934 1 tasks_processing.go:69] worker 47 listening for tasks. I0512 17:39:22.063937 1 tasks_processing.go:69] worker 13 listening for tasks. I0512 17:39:22.063938 1 tasks_processing.go:69] worker 48 listening for tasks. I0512 17:39:22.063938 1 tasks_processing.go:69] worker 38 listening for tasks. I0512 17:39:22.063955 1 tasks_processing.go:69] worker 53 listening for tasks. I0512 17:39:22.063905 1 tasks_processing.go:69] worker 37 listening for tasks. I0512 17:39:22.063963 1 tasks_processing.go:69] worker 18 listening for tasks. I0512 17:39:22.063927 1 tasks_processing.go:69] worker 46 listening for tasks. I0512 17:39:22.063965 1 tasks_processing.go:69] worker 54 listening for tasks. I0512 17:39:22.063970 1 tasks_processing.go:69] worker 57 listening for tasks. I0512 17:39:22.063926 1 tasks_processing.go:69] worker 39 listening for tasks. I0512 17:39:22.063975 1 tasks_processing.go:69] worker 26 listening for tasks. I0512 17:39:22.063976 1 tasks_processing.go:69] worker 27 listening for tasks. I0512 17:39:22.063928 1 tasks_processing.go:69] worker 12 listening for tasks. I0512 17:39:22.063983 1 tasks_processing.go:69] worker 60 listening for tasks. I0512 17:39:22.063982 1 tasks_processing.go:69] worker 16 listening for tasks. I0512 17:39:22.063980 1 tasks_processing.go:69] worker 28 listening for tasks. I0512 17:39:22.063989 1 tasks_processing.go:69] worker 23 listening for tasks. I0512 17:39:22.063991 1 tasks_processing.go:69] worker 24 listening for tasks. I0512 17:39:22.063995 1 tasks_processing.go:69] worker 63 listening for tasks. I0512 17:39:22.063995 1 tasks_processing.go:71] worker 1 working on machine_healthchecks task. I0512 17:39:22.064018 1 tasks_processing.go:71] worker 7 working on schedulers task. I0512 17:39:22.063862 1 tasks_processing.go:69] worker 2 listening for tasks. I0512 17:39:22.063939 1 tasks_processing.go:69] worker 52 listening for tasks. I0512 17:39:22.064068 1 tasks_processing.go:71] worker 31 working on infrastructures task. I0512 17:39:22.064072 1 tasks_processing.go:71] worker 8 working on nodenetworkstates task. I0512 17:39:22.064076 1 tasks_processing.go:71] worker 13 working on openshift_logging task. I0512 17:39:22.064074 1 tasks_processing.go:71] worker 32 working on service_accounts task. I0512 17:39:22.064079 1 tasks_processing.go:71] worker 52 working on operators task. I0512 17:39:22.064083 1 tasks_processing.go:71] worker 4 working on openstack_controlplanes task. I0512 17:39:22.064105 1 tasks_processing.go:71] worker 45 working on ceph_cluster task. I0512 17:39:22.064082 1 tasks_processing.go:71] worker 44 working on oauths task. I0512 17:39:22.064168 1 tasks_processing.go:71] worker 11 working on olm_operators task. I0512 17:39:22.064380 1 tasks_processing.go:71] worker 55 working on storage_cluster task. I0512 17:39:22.064425 1 tasks_processing.go:71] worker 48 working on tsdb_status task. I0512 17:39:22.064469 1 tasks_processing.go:71] worker 50 working on dvo_metrics task. I0512 17:39:22.064519 1 tasks_processing.go:71] worker 53 working on image_pruners task. I0512 17:39:22.064079 1 tasks_processing.go:71] worker 43 working on sap_config task. I0512 17:39:22.064725 1 tasks_processing.go:71] worker 27 working on crds task. I0512 17:39:22.064796 1 tasks_processing.go:71] worker 42 working on lokistack task. I0512 17:39:22.065001 1 tasks_processing.go:71] worker 23 working on nodes task. I0512 17:39:22.065026 1 tasks_processing.go:71] worker 26 working on node_features task. I0512 17:39:22.065031 1 tasks_processing.go:71] worker 54 working on qemu_kubevirt_launcher_logs task. I0512 17:39:22.065053 1 tasks_processing.go:71] worker 60 working on openstack_version task. I0512 17:39:22.064077 1 tasks_processing.go:71] worker 49 working on cluster_apiserver task. I0512 17:39:22.065234 1 tasks_processing.go:71] worker 28 working on machine_sets task. I0512 17:39:22.064100 1 tasks_processing.go:71] worker 40 working on pdbs task. W0512 17:39:22.064500 1 gather_prometheus_tsdb_status.go:38] Unable to load metrics client, tsdb status cannot be collected: open /var/run/configmaps/service-ca-bundle/service-ca.crt: no such file or directory I0512 17:39:22.065610 1 tasks_processing.go:71] worker 48 working on storage_classes task. I0512 17:39:22.065011 1 tasks_processing.go:71] worker 39 working on cost_management_metrics_configs task. I0512 17:39:22.065849 1 gather.go:177] gatherer "clusterconfig" function "tsdb_status" took 1.135811ms to process 0 records I0512 17:39:22.064799 1 tasks_processing.go:71] worker 35 working on proxies task. I0512 17:39:22.063951 1 tasks_processing.go:69] worker 29 listening for tasks. I0512 17:39:22.066120 1 tasks_processing.go:71] worker 29 working on machine_config_pools task. I0512 17:39:22.063957 1 tasks_processing.go:69] worker 17 listening for tasks. I0512 17:39:22.063956 1 tasks_processing.go:69] worker 56 listening for tasks. I0512 17:39:22.063958 1 tasks_processing.go:69] worker 25 listening for tasks. I0512 17:39:22.063967 1 tasks_processing.go:69] worker 15 listening for tasks. I0512 17:39:22.063971 1 tasks_processing.go:69] worker 21 listening for tasks. I0512 17:39:22.063857 1 tasks_processing.go:69] worker 41 listening for tasks. I0512 17:39:22.063981 1 tasks_processing.go:69] worker 58 listening for tasks. I0512 17:39:22.063981 1 tasks_processing.go:69] worker 22 listening for tasks. I0512 17:39:22.063983 1 tasks_processing.go:71] worker 6 working on machine_autoscalers task. I0512 17:39:22.063932 1 tasks_processing.go:69] worker 51 listening for tasks. I0512 17:39:22.063988 1 tasks_processing.go:71] worker 0 working on config_maps task. I0512 17:39:22.063986 1 tasks_processing.go:69] worker 19 listening for tasks. I0512 17:39:22.063989 1 tasks_processing.go:71] worker 9 working on ingress task. I0512 17:39:22.063992 1 tasks_processing.go:69] worker 59 listening for tasks. I0512 17:39:22.066509 1 tasks_processing.go:71] worker 51 working on overlapping_namespace_uids task. I0512 17:39:22.066519 1 tasks_processing.go:71] worker 59 working on pod_network_connectivity_checks task. I0512 17:39:22.066534 1 tasks_processing.go:71] worker 41 working on mutating_webhook_configurations task. I0512 17:39:22.063991 1 tasks_processing.go:69] worker 61 listening for tasks. I0512 17:39:22.064067 1 tasks_processing.go:71] worker 3 working on silenced_alerts task. W0512 17:39:22.066591 1 gather_silenced_alerts.go:38] Unable to load alerts client, no alerts will be collected: open /var/run/configmaps/service-ca-bundle/service-ca.crt: no such file or directory I0512 17:39:22.066613 1 gather.go:177] gatherer "clusterconfig" function "silenced_alerts" took 39.217µs to process 0 records I0512 17:39:22.064165 1 tasks_processing.go:71] worker 34 working on openstack_dataplanedeployments task. I0512 17:39:22.064068 1 tasks_processing.go:71] worker 33 working on container_runtime_configs task. I0512 17:39:22.064512 1 tasks_processing.go:71] worker 38 working on openshift_machine_api_events task. I0512 17:39:22.066808 1 tasks_processing.go:71] worker 19 working on machine_configs task. I0512 17:39:22.066856 1 tasks_processing.go:71] worker 61 working on sap_pods task. I0512 17:39:22.064806 1 tasks_processing.go:71] worker 36 working on image task. I0512 17:39:22.064859 1 tasks_processing.go:71] worker 20 working on image_registries task. I0512 17:39:22.064862 1 tasks_processing.go:71] worker 10 working on support_secret task. I0512 17:39:22.064893 1 tasks_processing.go:71] worker 37 working on aggregated_monitoring_cr_names task. I0512 17:39:22.064923 1 tasks_processing.go:71] worker 62 working on clusterroles task. I0512 17:39:22.064926 1 tasks_processing.go:71] worker 12 working on install_plans task. I0512 17:39:22.064934 1 tasks_processing.go:71] worker 18 working on number_of_pods_and_netnamespaces_with_sdn_annotations task. I0512 17:39:22.064958 1 tasks_processing.go:71] worker 47 working on networks task. I0512 17:39:22.064963 1 tasks_processing.go:71] worker 57 working on feature_gates task. I0512 17:39:22.064967 1 tasks_processing.go:71] worker 46 working on nodenetworkconfigurationpolicies task. I0512 17:39:22.065060 1 tasks_processing.go:71] worker 16 working on monitoring_persistent_volumes task. I0512 17:39:22.065114 1 tasks_processing.go:71] worker 63 working on jaegers task. I0512 17:39:22.065122 1 tasks_processing.go:71] worker 24 working on ingress_certificates task. I0512 17:39:22.065194 1 tasks_processing.go:71] worker 2 working on authentication task. I0512 17:39:22.066240 1 tasks_processing.go:71] worker 17 working on metrics task. W0512 17:39:22.067049 1 gather_most_recent_metrics.go:64] Unable to load metrics client, no metrics will be collected: open /var/run/configmaps/service-ca-bundle/service-ca.crt: no such file or directory I0512 17:39:22.067112 1 tasks_processing.go:71] worker 17 working on validating_webhook_configurations task. I0512 17:39:22.067167 1 gather.go:177] gatherer "clusterconfig" function "metrics" took 89.041µs to process 0 records I0512 17:39:22.066317 1 tasks_processing.go:71] worker 56 working on active_alerts task. W0512 17:39:22.067278 1 gather_active_alerts.go:54] Unable to load alerts client, no alerts will be collected: open /var/run/configmaps/service-ca-bundle/service-ca.crt: no such file or directory I0512 17:39:22.066415 1 tasks_processing.go:71] worker 25 working on machines task. I0512 17:39:22.067301 1 gather.go:177] gatherer "clusterconfig" function "active_alerts" took 29.561µs to process 0 records I0512 17:39:22.066438 1 tasks_processing.go:71] worker 15 working on sap_datahubs task. I0512 17:39:22.066480 1 tasks_processing.go:71] worker 21 working on certificate_signing_requests task. I0512 17:39:22.066487 1 tasks_processing.go:71] worker 58 working on node_logs task. I0512 17:39:22.066497 1 tasks_processing.go:71] worker 22 working on container_images task. I0512 17:39:22.066790 1 tasks_processing.go:71] worker 3 working on operators_pods_and_events task. I0512 17:39:22.068485 1 tasks_processing.go:74] worker 30 stopped. I0512 17:39:22.068501 1 gather.go:177] gatherer "clusterconfig" function "openstack_dataplanenodesets" took 4.613167ms to process 0 records I0512 17:39:22.063953 1 tasks_processing.go:69] worker 14 listening for tasks. I0512 17:39:22.068531 1 tasks_processing.go:74] worker 14 stopped. I0512 17:39:22.067292 1 tasks_processing.go:74] worker 56 stopped. I0512 17:39:22.069958 1 tasks_processing.go:74] worker 55 stopped. I0512 17:39:22.069989 1 gather.go:177] gatherer "clusterconfig" function "storage_cluster" took 5.526914ms to process 0 records I0512 17:39:22.070057 1 tasks_processing.go:74] worker 45 stopped. I0512 17:39:22.070072 1 gather.go:177] gatherer "clusterconfig" function "ceph_cluster" took 5.942856ms to process 0 records I0512 17:39:22.070107 1 gather.go:177] gatherer "clusterconfig" function "nodenetworkstates" took 5.977827ms to process 0 records I0512 17:39:22.070115 1 gather.go:177] gatherer "clusterconfig" function "openshift_logging" took 5.982718ms to process 0 records I0512 17:39:22.070120 1 tasks_processing.go:74] worker 13 stopped. I0512 17:39:22.070124 1 tasks_processing.go:74] worker 8 stopped. I0512 17:39:22.070271 1 controller.go:129] Initializing last reported time to 0001-01-01T00:00:00Z I0512 17:39:22.070287 1 controller.go:254] Source periodic-workloads *controllerstatus.Simple is not ready I0512 17:39:22.070298 1 controller.go:254] Source periodic-clusterconfig *controllerstatus.Simple is not ready I0512 17:39:22.070301 1 controller.go:254] Source periodic-conditional *controllerstatus.Simple is not ready I0512 17:39:22.070314 1 controller.go:531] The operator is still being initialized I0512 17:39:22.070319 1 controller.go:554] The operator is healthy I0512 17:39:22.070521 1 tasks_processing.go:74] worker 1 stopped. E0512 17:39:22.070535 1 gather.go:140] gatherer "clusterconfig" function "machine_healthchecks" failed with the error: machinehealthchecks.machine.openshift.io is forbidden: User "system:serviceaccount:openshift-insights:gather" cannot list resource "machinehealthchecks" in API group "machine.openshift.io" at the cluster scope I0512 17:39:22.070546 1 gather.go:177] gatherer "clusterconfig" function "machine_healthchecks" took 6.50886ms to process 0 records I0512 17:39:22.071228 1 tasks_processing.go:74] worker 39 stopped. I0512 17:39:22.071241 1 gather.go:177] gatherer "clusterconfig" function "cost_management_metrics_configs" took 5.444001ms to process 0 records I0512 17:39:22.075968 1 tasks_processing.go:74] worker 42 stopped. I0512 17:39:22.075985 1 gather.go:177] gatherer "clusterconfig" function "lokistack" took 10.969699ms to process 0 records I0512 17:39:22.090850 1 tasks_processing.go:74] worker 4 stopped. I0512 17:39:22.090865 1 gather.go:177] gatherer "clusterconfig" function "openstack_controlplanes" took 26.732353ms to process 0 records I0512 17:39:22.099865 1 tasks_processing.go:74] worker 53 stopped. I0512 17:39:22.100233 1 recorder.go:75] Recording config/clusteroperator/imageregistry.operator.openshift.io/imagepruner/cluster with fingerprint=89d687a2c7f5a5547b2e4cc377523c0ca5102b6f24773f9a866348af53c5c663 I0512 17:39:22.100288 1 gather.go:177] gatherer "clusterconfig" function "image_pruners" took 35.307087ms to process 1 records I0512 17:39:22.100302 1 gather.go:177] gatherer "clusterconfig" function "machine_config_pools" took 33.726985ms to process 0 records E0512 17:39:22.100311 1 gather.go:140] gatherer "clusterconfig" function "pod_network_connectivity_checks" failed with the error: the server could not find the requested resource (get podnetworkconnectivitychecks.controlplane.operator.openshift.io) I0512 17:39:22.100321 1 gather.go:177] gatherer "clusterconfig" function "pod_network_connectivity_checks" took 33.392779ms to process 0 records E0512 17:39:22.100329 1 gather.go:140] gatherer "clusterconfig" function "machines" failed with the error: machines.machine.openshift.io is forbidden: User "system:serviceaccount:openshift-insights:gather" cannot list resource "machines" in API group "machine.openshift.io" at the cluster scope I0512 17:39:22.100339 1 gather.go:177] gatherer "clusterconfig" function "machines" took 32.67628ms to process 0 records I0512 17:39:22.100340 1 tasks_processing.go:74] worker 29 stopped. I0512 17:39:22.100353 1 tasks_processing.go:74] worker 59 stopped. I0512 17:39:22.100359 1 tasks_processing.go:74] worker 25 stopped. I0512 17:39:22.100467 1 recorder.go:75] Recording config/storage/storageclasses/gp2-csi with fingerprint=7d1d7442594b6c4a9076672bb0904762c9057fb4f398383a5feac825252ce649 I0512 17:39:22.100472 1 tasks_processing.go:74] worker 48 stopped. I0512 17:39:22.100491 1 recorder.go:75] Recording config/storage/storageclasses/gp3-csi with fingerprint=9e0d1f8308f06fe1bc77caa73d0d8b82f61a37cc3c102032e02751c76b09d24e I0512 17:39:22.100500 1 gather.go:177] gatherer "clusterconfig" function "storage_classes" took 34.494628ms to process 2 records I0512 17:39:22.100597 1 tasks_processing.go:74] worker 40 stopped. I0512 17:39:22.100600 1 recorder.go:75] Recording config/pdbs/openshift-image-registry/image-registry with fingerprint=be2816f9fe66fc87ec666d80a2fe3baa80db1d3460217153560fa0d5959feda6 I0512 17:39:22.100673 1 recorder.go:75] Recording config/pdbs/openshift-ingress/router-default with fingerprint=6c5ed8d6392a5dbeaf613fee0858e00cd6b8a70c11f92f1fc9b6d2bffd99517d I0512 17:39:22.100703 1 recorder.go:75] Recording config/pdbs/openshift-operator-lifecycle-manager/packageserver-pdb with fingerprint=3dae71eb7ffc168a91784728c9f5455fb7e7c6c81eb54d6a01fa6af324d4eb35 I0512 17:39:22.100716 1 gather.go:177] gatherer "clusterconfig" function "pdbs" took 34.833222ms to process 3 records I0512 17:39:22.100796 1 tasks_processing.go:74] worker 7 stopped. I0512 17:39:22.100808 1 recorder.go:75] Recording config/schedulers/cluster with fingerprint=d113ca8aae9e1097d9139f4541fa22603a45422da43236be4903e1b7198c51c9 I0512 17:39:22.100819 1 gather.go:177] gatherer "clusterconfig" function "schedulers" took 36.404594ms to process 1 records I0512 17:39:22.100872 1 tasks_processing.go:74] worker 31 stopped. I0512 17:39:22.101590 1 recorder.go:75] Recording config/infrastructure with fingerprint=192ae505fdbb6f7e6a13faf67743e633c4176b663079e14dec282178d9cef429 I0512 17:39:22.101602 1 gather.go:177] gatherer "clusterconfig" function "infrastructures" took 36.370865ms to process 1 records I0512 17:39:22.101612 1 gather.go:177] gatherer "clusterconfig" function "olm_operators" took 36.078556ms to process 0 records I0512 17:39:22.101672 1 tasks_processing.go:74] worker 11 stopped. I0512 17:39:22.101738 1 tasks_processing.go:74] worker 49 stopped. I0512 17:39:22.101767 1 recorder.go:75] Recording config/apiserver with fingerprint=75102c0b02e9c3357e0ed83b0709866ccaab3f8766221e62c442fbbf29810063 I0512 17:39:22.101781 1 gather.go:177] gatherer "clusterconfig" function "cluster_apiserver" took 35.387796ms to process 1 records I0512 17:39:22.101790 1 gather.go:177] gatherer "clusterconfig" function "openstack_version" took 35.793175ms to process 0 records I0512 17:39:22.101820 1 tasks_processing.go:74] worker 60 stopped. I0512 17:39:22.101891 1 tasks_processing.go:74] worker 44 stopped. I0512 17:39:22.102046 1 recorder.go:75] Recording config/oauth with fingerprint=b5926e25b5f61d38318e7c8ffbb36692c3cd08b712a7bbc2ef34d7a946d26e13 I0512 17:39:22.102058 1 gather.go:177] gatherer "clusterconfig" function "oauths" took 35.908695ms to process 1 records E0512 17:39:22.102265 1 gather_node_features.go:86] GatherNodeFeatures: NodeFeatures resource not found in openshift-nfd namespace (may not be installed) I0512 17:39:22.102273 1 tasks_processing.go:74] worker 61 stopped. I0512 17:39:22.102288 1 gather.go:177] gatherer "clusterconfig" function "sap_pods" took 35.320503ms to process 0 records I0512 17:39:22.102297 1 gather.go:177] gatherer "clusterconfig" function "node_features" took 37.239905ms to process 0 records I0512 17:39:22.102305 1 tasks_processing.go:74] worker 26 stopped. I0512 17:39:22.102521 1 tasks_processing.go:74] worker 35 stopped. I0512 17:39:22.102595 1 recorder.go:75] Recording config/proxy with fingerprint=ea9523ffc4b516cc9c174336e5e3d0df341b68cc4ad8854a46cb719994a47c5c I0512 17:39:22.102609 1 gather.go:177] gatherer "clusterconfig" function "proxies" took 36.581155ms to process 1 records I0512 17:39:22.104804 1 tasks_processing.go:74] worker 6 stopped. I0512 17:39:22.104854 1 gather.go:177] gatherer "clusterconfig" function "machine_autoscalers" took 38.290974ms to process 0 records I0512 17:39:22.104873 1 gather.go:177] gatherer "clusterconfig" function "openstack_dataplanedeployments" took 38.200611ms to process 0 records I0512 17:39:22.104880 1 gather.go:177] gatherer "clusterconfig" function "sap_config" took 40.129021ms to process 0 records I0512 17:39:22.104888 1 tasks_processing.go:74] worker 43 stopped. I0512 17:39:22.104894 1 tasks_processing.go:74] worker 34 stopped. I0512 17:39:22.104898 1 tasks_processing.go:74] worker 46 stopped. I0512 17:39:22.104913 1 gather.go:177] gatherer "clusterconfig" function "nodenetworkconfigurationpolicies" took 37.769071ms to process 0 records I0512 17:39:22.105024 1 gather_logs.go:145] no pods in namespace were found I0512 17:39:22.105062 1 tasks_processing.go:74] worker 54 stopped. I0512 17:39:22.105110 1 gather.go:177] gatherer "clusterconfig" function "qemu_kubevirt_launcher_logs" took 40.0209ms to process 0 records I0512 17:39:22.105413 1 recorder.go:75] Recording config/ingress with fingerprint=0f3bcef6f6c0c650b705095ae0a7eb23d7a065104ed0e2e9da8589a7ad1da667 I0512 17:39:22.105470 1 gather.go:177] gatherer "clusterconfig" function "ingress" took 38.51832ms to process 1 records I0512 17:39:22.105431 1 tasks_processing.go:74] worker 9 stopped. I0512 17:39:22.105596 1 tasks_processing.go:74] worker 2 stopped. I0512 17:39:22.105749 1 recorder.go:75] Recording config/authentication with fingerprint=c1fc5a585874b664391fdd760fc58d80c71e369926c75c97383e95882215a378 I0512 17:39:22.105764 1 gather.go:177] gatherer "clusterconfig" function "authentication" took 38.030655ms to process 1 records I0512 17:39:22.105772 1 gather.go:177] gatherer "clusterconfig" function "jaegers" took 38.050147ms to process 0 records I0512 17:39:22.105779 1 gather.go:177] gatherer "clusterconfig" function "openshift_machine_api_events" took 38.482194ms to process 0 records I0512 17:39:22.105785 1 gather.go:177] gatherer "clusterconfig" function "container_runtime_configs" took 38.643024ms to process 0 records I0512 17:39:22.105786 1 tasks_processing.go:74] worker 63 stopped. I0512 17:39:22.105802 1 tasks_processing.go:74] worker 38 stopped. I0512 17:39:22.105843 1 tasks_processing.go:74] worker 33 stopped. I0512 17:39:22.105926 1 tasks_processing.go:74] worker 47 stopped. I0512 17:39:22.105941 1 recorder.go:75] Recording config/network with fingerprint=225e9dafbe4910272a66e9eea9f88f36bcbc67c7193381e2017640c65acd6ba4 I0512 17:39:22.105950 1 gather.go:177] gatherer "clusterconfig" function "networks" took 38.179943ms to process 1 records I0512 17:39:22.105957 1 gather.go:177] gatherer "clusterconfig" function "sap_datahubs" took 38.024806ms to process 0 records I0512 17:39:22.106047 1 recorder.go:75] Recording config/image with fingerprint=c61e51ea8f49f2c79dc9313f4608820079887cb37999625dc14df16163cf104d I0512 17:39:22.106065 1 gather.go:177] gatherer "clusterconfig" function "image" took 38.490663ms to process 1 records I0512 17:39:22.106111 1 tasks_processing.go:74] worker 15 stopped. I0512 17:39:22.106121 1 tasks_processing.go:74] worker 36 stopped. I0512 17:39:22.106192 1 tasks_processing.go:74] worker 17 stopped. I0512 17:39:22.106306 1 recorder.go:75] Recording config/validatingwebhookconfigurations/multus.openshift.io with fingerprint=30a2c5aa63ac3bcdf8a16513e59b12a51442c63c324c8f209e4777eafbc51df5 I0512 17:39:22.106437 1 recorder.go:75] Recording config/validatingwebhookconfigurations/network-node-identity.openshift.io with fingerprint=334d02f329d8be36bc1d7e737adfc42cc1bbf3a97b387d274889b9325d43a7f5 I0512 17:39:22.106508 1 recorder.go:75] Recording config/validatingwebhookconfigurations/performance-addon-operator with fingerprint=9e0ca97752d399b403c1ab1f9255e4af8cb48c49fd462867b80c4a6d6b33ac0e I0512 17:39:22.106550 1 recorder.go:75] Recording config/validatingwebhookconfigurations/sre-clusterrolebindings-validation with fingerprint=bc5f37115282c37d820d79864842845b8d784d867902138c821629f466a0d8ac I0512 17:39:22.106602 1 recorder.go:75] Recording config/validatingwebhookconfigurations/sre-clusterroles-validation with fingerprint=8ce8e98f893a86c2a9943796aa7c4b7e078cfbb4f7dc9da8cc20f8798818564a I0512 17:39:22.106642 1 recorder.go:75] Recording config/validatingwebhookconfigurations/sre-ingress-config-validation with fingerprint=e74d8b3602915cc1fd5582ea11ff6a7eefb7eb6059b4228fca60c3655bec2100 I0512 17:39:22.106682 1 recorder.go:75] Recording config/validatingwebhookconfigurations/sre-network-operator-validation with fingerprint=5640f71e82ea5e51dd1252ae1ea4253736146b53215eec13748804e569882e67 I0512 17:39:22.106737 1 recorder.go:75] Recording config/validatingwebhookconfigurations/sre-regular-user-validation with fingerprint=e5d4511adbabc86d0a9b944849d27ff834e372c8f21a43c2f44a03350ab35be9 I0512 17:39:22.106777 1 recorder.go:75] Recording config/validatingwebhookconfigurations/sre-scc-validation with fingerprint=fb73e0a4ba324346d7a3186f39355032a58d9907ff30711338655aefdaae4ad6 I0512 17:39:22.106816 1 recorder.go:75] Recording config/validatingwebhookconfigurations/sre-serviceaccount-validation with fingerprint=9370a371f127dec831530851a292e37b10aae12b26c4cba636b1286d1d92b65d I0512 17:39:22.106888 1 recorder.go:75] Recording config/validatingwebhookconfigurations/sre-techpreviewnoupgrade-validation with fingerprint=f3bf173d02482eded39db8e7b1cc747cd7534e7adadfd172bfbb34baeea65088 I0512 17:39:22.106928 1 gather.go:177] gatherer "clusterconfig" function "validating_webhook_configurations" took 38.38117ms to process 11 records I0512 17:39:22.106943 1 gather.go:177] gatherer "clusterconfig" function "node_logs" took 37.709176ms to process 0 records I0512 17:39:22.106953 1 gather.go:177] gatherer "clusterconfig" function "machine_sets" took 40.582101ms to process 0 records I0512 17:39:22.106977 1 tasks_processing.go:74] worker 58 stopped. I0512 17:39:22.106989 1 tasks_processing.go:74] worker 28 stopped. I0512 17:39:22.107057 1 tasks_processing.go:74] worker 57 stopped. I0512 17:39:22.107083 1 recorder.go:75] Recording config/featuregate with fingerprint=81f54ea5b7f24b59ff33787d1634b2f795b78e387d99663d4f33fec5c9b3bacb I0512 17:39:22.107160 1 gather.go:177] gatherer "clusterconfig" function "feature_gates" took 39.018088ms to process 1 records W0512 17:39:22.107174 1 gather_dvo_metrics.go:210] Failed to read the DVO metrics. Trying again. I0512 17:39:22.107249 1 tasks_processing.go:74] worker 20 stopped. I0512 17:39:22.108637 1 recorder.go:75] Recording config/clusteroperator/imageregistry.operator.openshift.io/config/cluster with fingerprint=0323e450cc296a4f410950900a2424e4e9d86132fbd56d07b71e955ce912ef20 I0512 17:39:22.108730 1 gather.go:177] gatherer "clusterconfig" function "image_registries" took 39.716997ms to process 1 records I0512 17:39:22.109405 1 tasks_processing.go:74] worker 41 stopped. I0512 17:39:22.109661 1 recorder.go:75] Recording config/mutatingwebhookconfigurations/aws-pod-identity with fingerprint=8374272c2189b1fdf5036cbe6e4105bd57ad6b930ac808c3e1dde8ade4e53a7f I0512 17:39:22.109782 1 recorder.go:75] Recording config/mutatingwebhookconfigurations/sre-podimagespec-mutation with fingerprint=c75dcd0ca03f15e0d01f058bd64ddb2730a232c532694f6776f99d338f8df0f3 I0512 17:39:22.109844 1 recorder.go:75] Recording config/mutatingwebhookconfigurations/sre-service-mutation with fingerprint=cd6de93983db31701b9edfb68f1dcb0e97fe037ab0c1d19652354948c79aa53a I0512 17:39:22.109855 1 gather.go:177] gatherer "clusterconfig" function "mutating_webhook_configurations" took 40.65567ms to process 3 records I0512 17:39:22.109939 1 tasks_processing.go:74] worker 10 stopped. E0512 17:39:22.109864 1 gather.go:140] gatherer "clusterconfig" function "support_secret" failed with the error: secrets "support" not found I0512 17:39:22.109973 1 gather.go:177] gatherer "clusterconfig" function "support_secret" took 40.991987ms to process 0 records I0512 17:39:22.110503 1 tasks_processing.go:74] worker 23 stopped. I0512 17:39:22.111158 1 recorder.go:75] Recording config/node/ip-10-0-0-127.ec2.internal with fingerprint=fcf24362a4a816a2f312d9db24e8a7cc9efc2aa5f72cc9f7b148abb3a8b2d2c0 I0512 17:39:22.111263 1 recorder.go:75] Recording config/node/ip-10-0-1-203.ec2.internal with fingerprint=9549cc3a9664dcd078c4d3d9b0bacb5fdca0b489da5d5d2741b4e3f1eba1bca5 I0512 17:39:22.111358 1 recorder.go:75] Recording config/node/ip-10-0-2-251.ec2.internal with fingerprint=36a7e21df439b46487684105d030434b1cfefb9e82d21c7c36b6fc15cc8ad4c7 I0512 17:39:22.111371 1 gather.go:177] gatherer "clusterconfig" function "nodes" took 43.641082ms to process 3 records E0512 17:39:22.111382 1 gather.go:140] gatherer "clusterconfig" function "machine_configs" failed with the error: getting MachineConfigPools failed: the server could not find the requested resource (get machineconfigpools.machineconfiguration.openshift.io) I0512 17:39:22.111427 1 recorder.go:75] Recording aggregated/unused_machine_configs_count with fingerprint=4bfc9fa984e5dfcd45848faaf05269de7619bf42edf9f781751af5ee05c1a499 I0512 17:39:22.111439 1 gather.go:177] gatherer "clusterconfig" function "machine_configs" took 42.717457ms to process 1 records I0512 17:39:22.111455 1 recorder.go:75] Recording config/namespaces_with_overlapping_uids with fingerprint=4f53cda18c2baa0c0354bb5f9a3ecbe5ed12ab4d8e11ba873c2f11161202b945 I0512 17:39:22.111465 1 gather.go:177] gatherer "clusterconfig" function "overlapping_namespace_uids" took 44.028892ms to process 1 records I0512 17:39:22.111466 1 tasks_processing.go:74] worker 19 stopped. I0512 17:39:22.111483 1 tasks_processing.go:74] worker 51 stopped. I0512 17:39:22.111653 1 tasks_processing.go:74] worker 16 stopped. I0512 17:39:22.111665 1 gather.go:177] gatherer "clusterconfig" function "monitoring_persistent_volumes" took 44.622771ms to process 0 records I0512 17:39:22.111759 1 tasks_processing.go:74] worker 21 stopped. I0512 17:39:22.111770 1 gather.go:177] gatherer "clusterconfig" function "certificate_signing_requests" took 43.787306ms to process 0 records I0512 17:39:22.117118 1 tasks_processing.go:74] worker 37 stopped. I0512 17:39:22.117139 1 gather.go:177] gatherer "clusterconfig" function "aggregated_monitoring_cr_names" took 50.124908ms to process 0 records I0512 17:39:22.117307 1 tasks_processing.go:74] worker 62 stopped. I0512 17:39:22.117581 1 recorder.go:75] Recording cluster-scoped-resources/rbac.authorization.k8s.io/clusterroles/admin with fingerprint=ae264212e3d270f3aeb9d27ca3c761c4f1ec1482114637c75b57a36656ef80a4 I0512 17:39:22.117740 1 recorder.go:75] Recording cluster-scoped-resources/rbac.authorization.k8s.io/clusterroles/edit with fingerprint=a6c5777b805bb6a21cec03038a9ca0b1695a6003c73b5b4f7d500893749325cb I0512 17:39:22.117756 1 gather.go:177] gatherer "clusterconfig" function "clusterroles" took 50.314942ms to process 2 records I0512 17:39:22.117814 1 tasks_processing.go:74] worker 27 stopped. I0512 17:39:22.118530 1 recorder.go:75] Recording config/crd/volumesnapshots.snapshot.storage.k8s.io with fingerprint=85ffb1079fc2a3e80588201b63e15f7d07e0718c098b45285a8726770bfaa197 I0512 17:39:22.118798 1 recorder.go:75] Recording config/crd/volumesnapshotcontents.snapshot.storage.k8s.io with fingerprint=fa968b425d3e12ffe947bdf3a1b231419b71a1b56410abc17277f3ddc23a7153 I0512 17:39:22.118816 1 gather.go:177] gatherer "clusterconfig" function "crds" took 53.032072ms to process 2 records I0512 17:39:22.118832 1 cluster_transfer.go:83] checking the availability of cluster transfer. Next check is in 12h0m0s I0512 17:39:22.118829 1 sca.go:136] Pulling SCA certificates from https://api.openshift.com/api/accounts_mgmt/v1/entitlement_certificates. Next check is in 8h0m0s W0512 17:39:22.118990 1 operator.go:328] started I0512 17:39:22.119012 1 base_controller.go:76] Waiting for caches to sync for LoggingSyncer I0512 17:39:22.122897 1 tasks_processing.go:74] worker 22 stopped. I0512 17:39:22.122964 1 recorder.go:75] Recording config/running_containers with fingerprint=40e8ad80a2f10d528de2449acda3254ebf9ede2bef414212e3348066db53b071 I0512 17:39:22.122977 1 gather.go:177] gatherer "clusterconfig" function "container_images" took 54.653954ms to process 1 records I0512 17:39:22.131189 1 shared_informer.go:320] Caches are synced for client-ca::kube-system::extension-apiserver-authentication::client-ca-file I0512 17:39:22.131197 1 shared_informer.go:320] Caches are synced for RequestHeaderAuthRequestController I0512 17:39:22.131221 1 shared_informer.go:320] Caches are synced for client-ca::kube-system::extension-apiserver-authentication::requestheader-client-ca-file I0512 17:39:22.136624 1 base_controller.go:82] Caches are synced for ConfigController I0512 17:39:22.136640 1 base_controller.go:119] Starting #1 worker of ConfigController controller ... I0512 17:39:22.141473 1 controller.go:254] Source periodic-workloads *controllerstatus.Simple is not ready I0512 17:39:22.141489 1 controller.go:254] Source scaController *sca.Controller is not ready I0512 17:39:22.141494 1 controller.go:254] Source clusterTransferController *clustertransfer.Controller is not ready I0512 17:39:22.141498 1 controller.go:254] Source periodic-clusterconfig *controllerstatus.Simple is not ready I0512 17:39:22.141503 1 controller.go:254] Source periodic-conditional *controllerstatus.Simple is not ready I0512 17:39:22.141525 1 controller.go:531] The operator is still being initialized I0512 17:39:22.141531 1 controller.go:554] The operator is healthy I0512 17:39:22.146179 1 prometheus_rules.go:88] Prometheus rules successfully created E0512 17:39:22.149410 1 cluster_transfer.go:95] failed to pull cluster transfer: unable to retrieve cluster transfer data from https://api.openshift.com/api/accounts_mgmt/v1/cluster_transfers/: Get "https://api.openshift.com/api/accounts_mgmt/v1/cluster_transfers/?search=cluster_uuid+is+%2770c09392-4440-48e9-8b33-9e36bdcab159%27+and+status+is+%27accepted%27": dial tcp: lookup api.openshift.com on 172.30.0.10:53: read udp 10.128.0.8:39449->172.30.0.10:53: read: connection refused I0512 17:39:22.149425 1 controllerstatus.go:80] name=clusterTransferController healthy=true reason=Disconnected message=failed to pull cluster transfer: unable to retrieve cluster transfer data from https://api.openshift.com/api/accounts_mgmt/v1/cluster_transfers/: Get "https://api.openshift.com/api/accounts_mgmt/v1/cluster_transfers/?search=cluster_uuid+is+%2770c09392-4440-48e9-8b33-9e36bdcab159%27+and+status+is+%27accepted%27": dial tcp: lookup api.openshift.com on 172.30.0.10:53: read udp 10.128.0.8:39449->172.30.0.10:53: read: connection refused I0512 17:39:22.151537 1 configmapobserver.go:84] configmaps "insights-config" not found I0512 17:39:22.153004 1 tasks_processing.go:74] worker 18 stopped. I0512 17:39:22.153021 1 gather.go:177] gatherer "clusterconfig" function "number_of_pods_and_netnamespaces_with_sdn_annotations" took 85.998504ms to process 0 records I0512 17:39:22.176109 1 tasks_processing.go:74] worker 0 stopped. E0512 17:39:22.176131 1 gather.go:140] gatherer "clusterconfig" function "config_maps" failed with the error: configmaps "cluster-monitoring-config" not found E0512 17:39:22.176140 1 gather.go:140] gatherer "clusterconfig" function "config_maps" failed with the error: configmaps "gateway-mode-config" not found E0512 17:39:22.176145 1 gather.go:140] gatherer "clusterconfig" function "config_maps" failed with the error: configmaps "insights-config" not found I0512 17:39:22.176161 1 recorder.go:75] Recording config/configmaps/openshift-config/installer-images/images.json with fingerprint=ad286723f58bdcfc37aeba3ec5b4110c08e1af59cd34d14b4bfaab02d18e4856 I0512 17:39:22.176219 1 recorder.go:75] Recording config/configmaps/openshift-config/kube-root-ca.crt/ca.crt with fingerprint=d476c7d3f5b104863f08f481b1264dcc68cc272ecefb0ecb709b18a6afab034d I0512 17:39:22.176230 1 recorder.go:75] Recording config/configmaps/openshift-config/openshift-install/invoker with fingerprint=76b482f683cd3ef9da02debac5b26080a5aeb06ff768ee5c21117514dff29d8a I0512 17:39:22.176236 1 recorder.go:75] Recording config/configmaps/openshift-config/openshift-install/version with fingerprint=2ab5ab7b1b10d7fcf1197bb24dea7c90f400e4effc18c7356873209d54fdf84b I0512 17:39:22.176242 1 recorder.go:75] Recording config/configmaps/openshift-config/openshift-service-ca.crt/service-ca.crt with fingerprint=e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855 I0512 17:39:22.176308 1 recorder.go:75] Recording config/configmaps/openshift-config/rosa-brand-logo/rosa-brand-logo.svg with fingerprint=6ed8ca4dd7a8eee7249182bc006e9649ce84d76c551ddfaaa33e55d8c4cc1ed0 I0512 17:39:22.176319 1 recorder.go:75] Recording config/configmaps/kube-system/cluster-config-v1/install-config with fingerprint=ab3811c6b83fd7b8e920094cfa3080d1b4ee3c35ec4c8379437b21d27bd6608d I0512 17:39:22.176327 1 gather.go:177] gatherer "clusterconfig" function "config_maps" took 109.572818ms to process 7 records I0512 17:39:22.193980 1 tasks_processing.go:74] worker 5 stopped. I0512 17:39:22.195016 1 recorder.go:75] Recording config/version with fingerprint=a1d6ae461424afc877983fe0df57e671bd367d546f4e75f9651ea62bef531559 I0512 17:39:22.195040 1 recorder.go:75] Recording config/id with fingerprint=bc90d896e287ee790cf6e7bec9df2b76d35702422c865fa689e9410c1f89a184 I0512 17:39:22.195058 1 gather.go:177] gatherer "clusterconfig" function "version" took 130.102765ms to process 2 records I0512 17:39:22.205381 1 requests.go:205] Asking for SCA certificate with "{"arch": ["x86_64"]}" payload W0512 17:39:22.209338 1 sca.go:161] Failed to pull SCA certs from https://api.openshift.com/api/accounts_mgmt/v1/entitlement_certificates: unable to retrieve SCA certs data from https://api.openshift.com/api/accounts_mgmt/v1/entitlement_certificates: Post "https://api.openshift.com/api/accounts_mgmt/v1/entitlement_certificates": dial tcp: lookup api.openshift.com on 172.30.0.10:53: read udp 10.128.0.8:49718->172.30.0.10:53: read: connection refused I0512 17:39:22.209361 1 controllerstatus.go:80] name=scaController healthy=true reason=NonHTTPError message=Failed to pull SCA certs from https://api.openshift.com/api/accounts_mgmt/v1/entitlement_certificates: unable to retrieve SCA certs data from https://api.openshift.com/api/accounts_mgmt/v1/entitlement_certificates: Post "https://api.openshift.com/api/accounts_mgmt/v1/entitlement_certificates": dial tcp: lookup api.openshift.com on 172.30.0.10:53: read udp 10.128.0.8:49718->172.30.0.10:53: read: connection refused I0512 17:39:22.219184 1 base_controller.go:82] Caches are synced for LoggingSyncer I0512 17:39:22.219199 1 base_controller.go:119] Starting #1 worker of LoggingSyncer controller ... I0512 17:39:22.219495 1 tasks_processing.go:74] worker 24 stopped. E0512 17:39:22.219517 1 gather.go:140] gatherer "clusterconfig" function "ingress_certificates" failed with the error: failed to get secret 'router-certs-default' in namespace 'openshift-ingress': secrets "router-certs-default" not found E0512 17:39:22.219526 1 gather.go:140] gatherer "clusterconfig" function "ingress_certificates" failed with the error: failed to get secret '2q8h84qmvn7vejalm2q7daaitdtc7meu-primary-cert-bundle-secret' in namespace 'openshift-ingress-operator': secrets "2q8h84qmvn7vejalm2q7daaitdtc7meu-primary-cert-bundle-secret" not found I0512 17:39:22.219609 1 recorder.go:75] Recording aggregated/ingress_controllers_certs with fingerprint=62b40885f1d0bf5d163f64925bdcdfba9e6c56dff50bf0cadb16f0369bf47e5d I0512 17:39:22.219630 1 gather.go:177] gatherer "clusterconfig" function "ingress_certificates" took 152.410021ms to process 1 records I0512 17:39:22.517708 1 gather_cluster_operator_pods_and_events.go:121] Found 14 pods with 15 containers I0512 17:39:22.517724 1 gather_cluster_operator_pods_and_events.go:235] Maximum buffer size: 1677721 bytes I0512 17:39:22.518859 1 gather_cluster_operator_pods_and_events.go:365] Fetching logs for dns container dns-default-hksvz pod in namespace openshift-dns (previous: false). I0512 17:39:22.723482 1 gather_cluster_operator_pods_and_events.go:408] Failed to fetch log for dns-default-hksvz pod in namespace openshift-dns for failing operator dns (previous: false): "container \"dns\" in pod \"dns-default-hksvz\" is waiting to start: ContainerCreating" I0512 17:39:22.723502 1 gather_cluster_operator_pods_and_events.go:280] Error: "container \"dns\" in pod \"dns-default-hksvz\" is waiting to start: ContainerCreating" I0512 17:39:22.723511 1 gather_cluster_operator_pods_and_events.go:365] Fetching logs for kube-rbac-proxy container dns-default-hksvz pod in namespace openshift-dns (previous: false). I0512 17:39:22.922717 1 gather_cluster_operator_pods_and_events.go:408] Failed to fetch log for dns-default-hksvz pod in namespace openshift-dns for failing operator kube-rbac-proxy (previous: false): "container \"kube-rbac-proxy\" in pod \"dns-default-hksvz\" is waiting to start: ContainerCreating" I0512 17:39:22.922737 1 gather_cluster_operator_pods_and_events.go:280] Error: "container \"kube-rbac-proxy\" in pod \"dns-default-hksvz\" is waiting to start: ContainerCreating" I0512 17:39:22.922749 1 gather_cluster_operator_pods_and_events.go:365] Fetching logs for dns-node-resolver container node-resolver-28x4p pod in namespace openshift-dns (previous: false). W0512 17:39:23.104617 1 gather_dvo_metrics.go:210] Failed to read the DVO metrics. Trying again. I0512 17:39:23.223445 1 gather_cluster_operator_pods_and_events.go:280] Error: "log buffer is empty" I0512 17:39:23.223466 1 gather_cluster_operator_pods_and_events.go:365] Fetching logs for dns-node-resolver container node-resolver-8tmnz pod in namespace openshift-dns (previous: false). I0512 17:39:23.365673 1 gather_cluster_operator_pods_and_events.go:280] Error: "log buffer is empty" I0512 17:39:23.365698 1 gather_cluster_operator_pods_and_events.go:365] Fetching logs for dns-node-resolver container node-resolver-bjq46 pod in namespace openshift-dns (previous: false). I0512 17:39:23.529323 1 gather_cluster_operator_pods_and_events.go:280] Error: "log buffer is empty" I0512 17:39:23.529342 1 gather_cluster_operator_pods_and_events.go:365] Fetching logs for registry container image-registry-6f546f9c95-69djc pod in namespace openshift-image-registry (previous: false). I0512 17:39:23.551587 1 gather_cluster_operators.go:184] Unable to get configs.samples.operator.openshift.io resource due to: configs.samples.operator.openshift.io "cluster" not found I0512 17:39:23.717295 1 gather_cluster_operator_pods_and_events.go:280] Error: "log buffer is empty" I0512 17:39:23.717316 1 gather_cluster_operator_pods_and_events.go:365] Fetching logs for registry container image-registry-7fbc9b798d-cxxvn pod in namespace openshift-image-registry (previous: false). I0512 17:39:23.922000 1 gather_cluster_operator_pods_and_events.go:408] Failed to fetch log for image-registry-7fbc9b798d-cxxvn pod in namespace openshift-image-registry for failing operator registry (previous: false): "container \"registry\" in pod \"image-registry-7fbc9b798d-cxxvn\" is waiting to start: ContainerCreating" I0512 17:39:23.922024 1 gather_cluster_operator_pods_and_events.go:280] Error: "container \"registry\" in pod \"image-registry-7fbc9b798d-cxxvn\" is waiting to start: ContainerCreating" I0512 17:39:23.922036 1 gather_cluster_operator_pods_and_events.go:365] Fetching logs for registry container image-registry-7fbc9b798d-n86ht pod in namespace openshift-image-registry (previous: false). I0512 17:39:23.952403 1 tasks_processing.go:74] worker 52 stopped. I0512 17:39:23.952476 1 recorder.go:75] Recording config/clusteroperator/console with fingerprint=5c401cddbe4e5f1d1bbf292563e2865972988c09d69a162a2f655bdd5a14b8f1 I0512 17:39:23.952542 1 recorder.go:75] Recording config/clusteroperator/csi-snapshot-controller with fingerprint=21f0fc9de0635c0fd19e291825cbb852ed93d42be2ce98e77d1817ce1c5e4e3a I0512 17:39:23.952583 1 recorder.go:75] Recording config/clusteroperator/operator.openshift.io/csisnapshotcontroller/cluster with fingerprint=5adc514f4b63e2f1ecc68bf6f9c0af70c5eea04522a49524e102721b1c41f80e I0512 17:39:23.952623 1 recorder.go:75] Recording config/clusteroperator/dns with fingerprint=5f60fe585954b63b5348e13e453834688aa0960f8646ee33e783db9075334e2c I0512 17:39:23.952648 1 recorder.go:75] Recording config/clusteroperator/operator.openshift.io/dns/default with fingerprint=9e7b4ce029030d3d8c3b49af92c556acdcc415000b40d3f969dbdc42c432b47f I0512 17:39:23.952684 1 recorder.go:75] Recording config/clusteroperator/image-registry with fingerprint=849176b266d6d611393e171f7cb84c9da9c416cf814d4413907022abbefc833a I0512 17:39:23.952756 1 recorder.go:75] Recording config/clusteroperator/ingress with fingerprint=e4ddb0a7fb8c0070816d83f51fb090de5dea0b1aac58060760f7462b4ebf3d38 I0512 17:39:23.952796 1 recorder.go:75] Recording config/clusteroperator/operator.openshift.io/ingresscontroller/openshift-ingress-operator/default with fingerprint=ef3fc9e477a8645c11114bb3e4995c2618a8bc75a5fcf665afd230a279a45777 I0512 17:39:23.952819 1 recorder.go:75] Recording config/clusteroperator/insights with fingerprint=0e591f2a2d87743378f83ed2476421a6d1a1326a8497458fcf73bacf02ff09fc I0512 17:39:23.952859 1 recorder.go:75] Recording config/clusteroperator/kube-apiserver with fingerprint=c95cd7d602e1b156167f30e9855b94c1dcd181c5ef5d32c395eb79c0a237f5b1 I0512 17:39:23.952873 1 recorder.go:75] Recording config/clusteroperator/operator.openshift.io/kubeapiserver/cluster with fingerprint=51503bf0b784fcf65ea46bcaf1f72ac1a5c4d5dc211934f18f27871efed05762 I0512 17:39:23.952901 1 recorder.go:75] Recording config/clusteroperator/kube-controller-manager with fingerprint=7d00c7afd9e6fc2d165a401a6c18bbacbc9792490c5e474a09c7ef7bd9f30024 I0512 17:39:23.952916 1 recorder.go:75] Recording config/clusteroperator/operator.openshift.io/kubecontrollermanager/cluster with fingerprint=ce90c0d4f367d7da085074268031798382ae7c54fdcb0a21f15a4818fe308c11 I0512 17:39:23.952941 1 recorder.go:75] Recording config/clusteroperator/kube-scheduler with fingerprint=377893962d90d08307da04bb31b22905c04f45ba4d239e6d6e4db8168227d59e I0512 17:39:23.952953 1 recorder.go:75] Recording config/clusteroperator/operator.openshift.io/kubescheduler/cluster with fingerprint=f2940fb9fd20c19951dfc295eb363b7fba0c505f5ae61f01967a063099e6b60a I0512 17:39:23.952966 1 recorder.go:75] Recording config/clusteroperator/kube-storage-version-migrator with fingerprint=bf35272c7e49d43f1ed5f97409b7596a9d0813804bd4c812aff08838984e12b5 I0512 17:39:23.952975 1 recorder.go:75] Recording config/clusteroperator/operator.openshift.io/kubestorageversionmigrator/cluster with fingerprint=9351181aa7e6ada41ef581ab31e13516c6b934cc95710154bafb2eb222cb58db I0512 17:39:23.952990 1 recorder.go:75] Recording config/clusteroperator/monitoring with fingerprint=662cb7db915e393526e3b140bfa58c6626588b2baf883e518d8065255c4431e5 I0512 17:39:23.953145 1 recorder.go:75] Recording config/clusteroperator/network with fingerprint=512dd2f6f2b34d3b9dffe2211bb90db1c163640e0932ff105c2dc45a6d0f2661 I0512 17:39:23.953157 1 recorder.go:75] Recording config/clusteroperator/network.operator.openshift.io/operatorpki/openshift-ovn-kubernetes/ovn with fingerprint=626a89d20e0deaed5b6dfb533acfe65f4bb1618bd200a703b62e60c5d16d94ab I0512 17:39:23.953165 1 recorder.go:75] Recording config/clusteroperator/network.operator.openshift.io/operatorpki/openshift-ovn-kubernetes/signer with fingerprint=90410b16914712b85b3c4578716ad8c0ae072e688f4cd1e022bf76f20da3506d I0512 17:39:23.953186 1 recorder.go:75] Recording config/clusteroperator/operator.openshift.io/network/cluster with fingerprint=a93d15eaecb455a0e40ecb2826eeecc1533899204ddd3c3921d15ab70af7ae75 I0512 17:39:23.953212 1 recorder.go:75] Recording config/clusteroperator/node-tuning with fingerprint=86b85a616764b770576866d671465ed96239aa5c83046e21c17c716bd61832d0 I0512 17:39:23.953235 1 recorder.go:75] Recording config/clusteroperator/openshift-apiserver with fingerprint=2c2482cd669aa323174dd27b1adbc385150e0e60028e69ff115f011787bdca83 I0512 17:39:23.953244 1 recorder.go:75] Recording config/clusteroperator/operator.openshift.io/openshiftapiserver/cluster with fingerprint=e712e6cf27339b441e4ed1f4cde91dbde7e952698ba93407e4457db63a4a4c76 I0512 17:39:23.953261 1 recorder.go:75] Recording config/clusteroperator/openshift-controller-manager with fingerprint=6c31da71acd9e081f51ce96e2bea3bbd962a4862fdade065d30088c9b30b484e I0512 17:39:23.953271 1 recorder.go:75] Recording config/clusteroperator/operator.openshift.io/openshiftcontrollermanager/cluster with fingerprint=d71a0f4672f9b45d9fc8293bf1687afc650fd28d32e2e30de27523fe7b4eadf7 I0512 17:39:23.953283 1 recorder.go:75] Recording config/clusteroperator/openshift-samples with fingerprint=52ff8212a728f16573beee7823b67be0f68790a62bcf420c33769003b28e17b3 I0512 17:39:23.953298 1 recorder.go:75] Recording config/clusteroperator/operator-lifecycle-manager with fingerprint=960adfb7e2ee6a0070536921a786bc227c07eb910979f96a8f516b529a099611 I0512 17:39:23.953312 1 recorder.go:75] Recording config/clusteroperator/operator-lifecycle-manager-catalog with fingerprint=3d8d0b1ce139663fa604c9da93977704b507c955283c63dd74c579c7f0a68202 I0512 17:39:23.953327 1 recorder.go:75] Recording config/clusteroperator/operator-lifecycle-manager-packageserver with fingerprint=9a0e4abbf611d8968473d587f78e80237523f1b3365102cdd0f23aff983a10ca I0512 17:39:23.953340 1 recorder.go:75] Recording config/clusteroperator/service-ca with fingerprint=e3dd02810b771bf7998425ec011e8ff3ad5915a059b8bc3abe0e2a395af3a78e I0512 17:39:23.953369 1 recorder.go:75] Recording config/clusteroperator/storage with fingerprint=7ac24f83bfb6ce52a2db0004379201b7a4285fdcc8c567165b4e7460993f8251 I0512 17:39:23.953385 1 recorder.go:75] Recording config/clusteroperator/operator.openshift.io/clustercsidriver/ebs.csi.aws.com with fingerprint=510064d6f6bcced87ab5bd2ddaff3d0edd7f93f4a4f7af2641f29fc53ffab21e I0512 17:39:23.953393 1 recorder.go:75] Recording config/clusteroperator/operator.openshift.io/storage/cluster with fingerprint=8e480f8c1ce1b39baac42d8ec780c57c2592929ae0c801b61ffad49ba13f33ad I0512 17:39:23.953401 1 gather.go:177] gatherer "clusterconfig" function "operators" took 1.888283115s to process 35 records W0512 17:39:24.104673 1 gather_dvo_metrics.go:210] Failed to read the DVO metrics. Trying again. I0512 17:39:24.123177 1 gather_cluster_operator_pods_and_events.go:408] Failed to fetch log for image-registry-7fbc9b798d-n86ht pod in namespace openshift-image-registry for failing operator registry (previous: false): "container \"registry\" in pod \"image-registry-7fbc9b798d-n86ht\" is waiting to start: ContainerCreating" I0512 17:39:24.123197 1 gather_cluster_operator_pods_and_events.go:280] Error: "container \"registry\" in pod \"image-registry-7fbc9b798d-n86ht\" is waiting to start: ContainerCreating" I0512 17:39:24.123209 1 gather_cluster_operator_pods_and_events.go:365] Fetching logs for node-ca container node-ca-5dfqf pod in namespace openshift-image-registry (previous: false). I0512 17:39:24.322784 1 gather_cluster_operator_pods_and_events.go:280] Error: "log buffer is empty" I0512 17:39:24.322804 1 gather_cluster_operator_pods_and_events.go:365] Fetching logs for node-ca container node-ca-cdqv2 pod in namespace openshift-image-registry (previous: false). I0512 17:39:24.522550 1 gather_cluster_operator_pods_and_events.go:280] Error: "log buffer is empty" I0512 17:39:24.522571 1 gather_cluster_operator_pods_and_events.go:365] Fetching logs for node-ca container node-ca-tngw7 pod in namespace openshift-image-registry (previous: false). I0512 17:39:24.723773 1 gather_cluster_operator_pods_and_events.go:280] Error: "log buffer is empty" I0512 17:39:24.723794 1 gather_cluster_operator_pods_and_events.go:365] Fetching logs for router container router-default-64b8f776df-w2f69 pod in namespace openshift-ingress (previous: false). I0512 17:39:24.922770 1 gather_cluster_operator_pods_and_events.go:408] Failed to fetch log for router-default-64b8f776df-w2f69 pod in namespace openshift-ingress for failing operator router (previous: false): "container \"router\" in pod \"router-default-64b8f776df-w2f69\" is waiting to start: ContainerCreating" I0512 17:39:24.922789 1 gather_cluster_operator_pods_and_events.go:280] Error: "container \"router\" in pod \"router-default-64b8f776df-w2f69\" is waiting to start: ContainerCreating" I0512 17:39:24.922800 1 gather_cluster_operator_pods_and_events.go:365] Fetching logs for router container router-default-6d99c7b758-jft4v pod in namespace openshift-ingress (previous: false). W0512 17:39:25.104508 1 gather_dvo_metrics.go:210] Failed to read the DVO metrics. Trying again. I0512 17:39:25.128720 1 gather_cluster_operator_pods_and_events.go:408] Failed to fetch log for router-default-6d99c7b758-jft4v pod in namespace openshift-ingress for failing operator router (previous: false): "container \"router\" in pod \"router-default-6d99c7b758-jft4v\" is waiting to start: ContainerCreating" I0512 17:39:25.128740 1 gather_cluster_operator_pods_and_events.go:280] Error: "container \"router\" in pod \"router-default-6d99c7b758-jft4v\" is waiting to start: ContainerCreating" I0512 17:39:25.128751 1 gather_cluster_operator_pods_and_events.go:365] Fetching logs for router container router-default-6d99c7b758-xt6jk pod in namespace openshift-ingress (previous: false). I0512 17:39:25.325938 1 gather_cluster_operator_pods_and_events.go:408] Failed to fetch log for router-default-6d99c7b758-xt6jk pod in namespace openshift-ingress for failing operator router (previous: false): "container \"router\" in pod \"router-default-6d99c7b758-xt6jk\" is waiting to start: ContainerCreating" I0512 17:39:25.325956 1 gather_cluster_operator_pods_and_events.go:280] Error: "container \"router\" in pod \"router-default-6d99c7b758-xt6jk\" is waiting to start: ContainerCreating" I0512 17:39:25.325967 1 gather_cluster_operator_pods_and_events.go:365] Fetching logs for serve-healthcheck-canary container ingress-canary-sgltz pod in namespace openshift-ingress-canary (previous: false). I0512 17:39:25.523224 1 tasks_processing.go:74] worker 3 stopped. I0512 17:39:25.523309 1 recorder.go:75] Recording events/openshift-dns-operator with fingerprint=bed55f9d097d1b866def8cd9c8fec7bad6f6bb170d071aedbcfbeff7e0822caa I0512 17:39:25.523351 1 recorder.go:75] Recording events/openshift-dns with fingerprint=bb7477f3f5cad7027f434926fa46e86a38f1e494678f8f4f0d942756bca3e1de I0512 17:39:25.523430 1 recorder.go:75] Recording events/openshift-image-registry with fingerprint=a3db3833022e5670ed058422dadd32a63b9bdef3b64de9a2cb88caba419d774b I0512 17:39:25.523460 1 recorder.go:75] Recording events/openshift-ingress-operator with fingerprint=4e61acfdc49b4ab46479752cdb03da1c3fd5ba867620a663b768e013de4dc5ba I0512 17:39:25.523503 1 recorder.go:75] Recording events/openshift-ingress with fingerprint=2a6d3ad2fb9eb2ec52e5488b8ba71c8a9649ed82c3ea27274ed1dbe41579d8d5 I0512 17:39:25.523522 1 recorder.go:75] Recording events/openshift-ingress-canary with fingerprint=eabc9fa336448452d076070c4417ca47552c354e6f8e43d326cc7d330092c04d I0512 17:39:25.523528 1 recorder.go:75] Recording config/pod/openshift-ingress-canary/logs/ingress-canary-sgltz/serve-healthcheck-canary_current.log with fingerprint=f3b7f367e35ed6ed583a36c49821b1be0dd665f35f8fe867d4d2cbc0cba21c1f I0512 17:39:25.523536 1 gather.go:177] gatherer "clusterconfig" function "operators_pods_and_events" took 3.454854593s to process 7 records W0512 17:39:26.104740 1 gather_dvo_metrics.go:210] Failed to read the DVO metrics. Trying again. W0512 17:39:27.100481 1 gather_dvo_metrics.go:117] Unable to read metrics from endpoint "http://deployment-validation-operator-metrics.openshift-deployment-validation-operator.svc:8383": DVO metrics service was not available within the 5s timeout: context deadline exceeded E0512 17:39:27.100519 1 gather.go:140] gatherer "clusterconfig" function "dvo_metrics" failed with the error: DVO metrics service was not available within the 5s timeout: context deadline exceeded I0512 17:39:27.100531 1 tasks_processing.go:74] worker 50 stopped. I0512 17:39:27.100549 1 recorder.go:75] Recording config/dvo_metrics with fingerprint=e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855 W0512 17:39:27.100564 1 gather.go:155] issue recording gatherer "clusterconfig" function "dvo_metrics" result "config/dvo_metrics" because of the warning: warning: the record with the same fingerprint "e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855" was already recorded at path "config/configmaps/openshift-config/openshift-service-ca.crt/service-ca.crt", recording another one with a different path "config/dvo_metrics" I0512 17:39:27.100578 1 gather.go:177] gatherer "clusterconfig" function "dvo_metrics" took 5.035980356s to process 1 records I0512 17:39:34.716195 1 tasks_processing.go:74] worker 12 stopped. I0512 17:39:34.716234 1 recorder.go:75] Recording config/installplans with fingerprint=7b887df561a3a9e6ef0dc672845aa5d56e348505006b7496d3a2f83892b0c95b I0512 17:39:34.716246 1 gather.go:177] gatherer "clusterconfig" function "install_plans" took 12.649141327s to process 1 records I0512 17:39:35.026885 1 configmapobserver.go:84] configmaps "insights-config" not found I0512 17:39:35.471587 1 tasks_processing.go:74] worker 32 stopped. I0512 17:39:35.471856 1 recorder.go:75] Recording config/serviceaccounts with fingerprint=2a6a98cd978f6a8f6d561f8f5025ec9397f180ae98d9caca3683e8186873f98f I0512 17:39:35.471873 1 gather.go:177] gatherer "clusterconfig" function "service_accounts" took 13.407490487s to process 1 records E0512 17:39:35.471929 1 periodic.go:254] "Unhandled Error" err="clusterconfig failed after 13.408s with: function \"machine_healthchecks\" failed with an error, function \"pod_network_connectivity_checks\" failed with an error, function \"machines\" failed with an error, function \"support_secret\" failed with an error, function \"machine_configs\" failed with an error, function \"config_maps\" failed with an error, function \"ingress_certificates\" failed with an error, function \"dvo_metrics\" failed with an error" I0512 17:39:35.473044 1 controllerstatus.go:89] name=periodic-clusterconfig healthy=false reason=PeriodicGatherFailed message=Source clusterconfig could not be retrieved: function "machine_healthchecks" failed with an error, function "pod_network_connectivity_checks" failed with an error, function "machines" failed with an error, function "support_secret" failed with an error, function "machine_configs" failed with an error, function "config_maps" failed with an error, function "ingress_certificates" failed with an error, function "dvo_metrics" failed with an error I0512 17:39:35.473062 1 periodic.go:216] Running workloads gatherer I0512 17:39:35.473081 1 tasks_processing.go:45] number of workers: 2 I0512 17:39:35.473108 1 tasks_processing.go:69] worker 1 listening for tasks. I0512 17:39:35.473113 1 tasks_processing.go:71] worker 1 working on workload_info task. I0512 17:39:35.473125 1 tasks_processing.go:69] worker 0 listening for tasks. I0512 17:39:35.473151 1 tasks_processing.go:71] worker 0 working on helmchart_info task. I0512 17:39:35.500055 1 tasks_processing.go:74] worker 0 stopped. I0512 17:39:35.500076 1 gather.go:177] gatherer "workloads" function "helmchart_info" took 26.879921ms to process 0 records I0512 17:39:35.556117 1 gather_workloads_info.go:278] Loaded pods in 0s, will wait 22s for image data I0512 17:39:35.564301 1 gather_workloads_info.go:387] No image sha256:a043239802b3eb8b323d285193d2527fad0ecec98ca91d188a3472a2fac8ae04 (9ms) I0512 17:39:35.572692 1 gather_workloads_info.go:387] No image sha256:3a1c7ec9bb0db343348170a70697ea3209ed81e22f1259ff694190e5d04ef250 (8ms) I0512 17:39:35.580398 1 gather_workloads_info.go:387] No image sha256:730d1b6988025bef0daa3a9a5d8467ec4a26b0382cc52f91c3375b4590d3518a (8ms) I0512 17:39:35.587980 1 gather_workloads_info.go:387] No image sha256:5808401268394502d335281ea1a294b07210461b986b58f91d7d1f29c0029c6d (8ms) I0512 17:39:35.595244 1 gather_workloads_info.go:387] No image sha256:0a4dfb8d4c1b3849319d45b4c54dff26a7238a2c08fcaa121f93073e95ab12e8 (7ms) I0512 17:39:35.602786 1 gather_workloads_info.go:387] No image sha256:084aa9b0f8a6d478549dc384d4e66da13ee9b25cc98531da861cc19dee2a9e8f (8ms) I0512 17:39:35.610645 1 gather_workloads_info.go:387] No image sha256:89277d8d4560d71db88c2dcc67c992a24544ca21810920b609c1d49d53b4a287 (8ms) I0512 17:39:35.618537 1 gather_workloads_info.go:387] No image sha256:30e597ec5d6bb96ff70a4f8688c748b659cd4fd5d73d222e8701821d236795c5 (8ms) I0512 17:39:35.626137 1 gather_workloads_info.go:387] No image sha256:3bba1358d4a0ae878ff491c0c2cbfffe60649e110b40342b878fe8fa332f8858 (8ms) I0512 17:39:35.633347 1 gather_workloads_info.go:387] No image sha256:e8b96d9318b3b8c9ed0afe4e6381f635c6b0c2f20772044ac68001ed49af2c87 (7ms) I0512 17:39:35.663908 1 gather_workloads_info.go:387] No image sha256:5c6d21c3f97366bc7ab57031cc027b67405a684bd804ce364ed5998b0685eaca (31ms) I0512 17:39:35.765023 1 gather_workloads_info.go:387] No image sha256:4733236617781e3469ffffb15e4daaa1f14ea8e1c52b426a3787a4f1f2945424 (101ms) I0512 17:39:35.865633 1 gather_workloads_info.go:387] No image sha256:5ac9c549d65fc1d8bc900773bebee43e9192bcec1bb5fa46afb4597230c16ac7 (101ms) I0512 17:39:35.965196 1 gather_workloads_info.go:387] No image sha256:ab60623bb32f7e75fca71ef65137731cae347a21c7a4091dfd583fa00732721c (100ms) I0512 17:39:36.065065 1 gather_workloads_info.go:387] No image sha256:637d41f067a5239096fc22b135181cda5113da833f1370e7a73965e83792e93a (100ms) I0512 17:39:36.165154 1 gather_workloads_info.go:387] No image sha256:5b6212b8f539f08e78417d8a4b7485ca0b4e7927cacd7b752742a28841bc8ccd (100ms) I0512 17:39:36.264965 1 gather_workloads_info.go:387] No image sha256:48e883a3932aea9457f55cd4628d21397d429d4828f42f95d1c903d9d5395bde (100ms) I0512 17:39:36.364452 1 gather_workloads_info.go:387] No image sha256:d6fbe0075cbb12bfd287c973704fadf97154c7f73e370733d976a40835e9155a (99ms) I0512 17:39:36.465355 1 gather_workloads_info.go:387] No image sha256:934d8e8c50f3c609b8eea80d1051111fe3d066fe8c65c79572072ae55fcb0a86 (101ms) I0512 17:39:36.564700 1 gather_workloads_info.go:387] No image sha256:673ebc8cc22c56c8f410e011b2fa950d28cf7b6420e17fdb6580d6cb10523384 (99ms) I0512 17:39:36.664898 1 gather_workloads_info.go:387] No image sha256:90a8ffd9643ebb16a6a8c04bb38cf9ed58903e9d3bf836c68f399193db5edaf6 (100ms) I0512 17:39:36.764789 1 gather_workloads_info.go:387] No image sha256:ce138e8cf5b96557b1864ef6f27c2608bdca59be2611804366cef7169c36291e (100ms) I0512 17:39:36.864966 1 gather_workloads_info.go:387] No image sha256:000105ef5150e7079b90a613fb9e6193e2a6ef9b1908d2dce44f2395d4fd070f (100ms) I0512 17:39:36.969145 1 gather_workloads_info.go:387] No image sha256:25c148fd380b1a9db3f6039d2e0eabc489a954921452391390ba9192b2325678 (104ms) I0512 17:39:37.064825 1 gather_workloads_info.go:387] No image sha256:875c77e5d144f03fb91d8cee0259f6966683ca88d1bf818dbf4652c16b70312c (96ms) I0512 17:39:37.165179 1 gather_workloads_info.go:387] No image sha256:6814f5fced0ef219d06374011c68a11a3da788a764a00a69ade435466d9ee240 (100ms) I0512 17:39:37.265272 1 gather_workloads_info.go:387] No image sha256:55b1db6038c5beaed54c626e3343b7a8589cc0be8dc41d1a66b4deab766ff520 (100ms) I0512 17:39:37.364905 1 gather_workloads_info.go:387] No image sha256:ae7d3453fd734ecc865e5f9bb16f29244ebffe6291b77e1d4e496f71eb053174 (100ms) I0512 17:39:37.465254 1 gather_workloads_info.go:387] No image sha256:e3732e356ae2324565c74cae57d5d016917314fa293ddaa3a68ee9ae030c6f07 (100ms) I0512 17:39:37.565125 1 gather_workloads_info.go:387] No image sha256:e1ba458cf6f0b3606c90880da72db8ab99cd11040bae84baebe3ff2e0d1ea075 (100ms) I0512 17:39:37.665077 1 gather_workloads_info.go:387] No image sha256:d64bea34bf3e1bb0b3a701c3ff14e66665afc1b050f28124ad7e6888eaec3a81 (100ms) I0512 17:39:37.765421 1 gather_workloads_info.go:387] No image sha256:5aaea0419169e55832cc27acfe0fe3b9513a343d6bdf71d3da1575ed322245d0 (100ms) I0512 17:39:37.864388 1 gather_workloads_info.go:387] No image sha256:08c5a78c8a5af04c549e2273aaf4bb452a75bf038d68aa9d01bb2aff66c30e90 (99ms) I0512 17:39:37.965237 1 gather_workloads_info.go:387] No image sha256:15677f0b70e6aa2dfaf088e45fc1a425c22bf6fda326b8116f87e88b6694dfab (101ms) I0512 17:39:37.965272 1 tasks_processing.go:74] worker 1 stopped. E0512 17:39:37.965284 1 gather.go:140] gatherer "workloads" function "workload_info" failed with the error: no running pods found for the insights-runtime-extractor statefulset I0512 17:39:37.965541 1 recorder.go:75] Recording config/workload_info with fingerprint=ddee7d9e45c1200c602a2998db495a31433b1730fabef66e7c9d120a07b0756c I0512 17:39:37.965559 1 gather.go:177] gatherer "workloads" function "workload_info" took 2.492150696s to process 1 records E0512 17:39:37.965589 1 periodic.go:254] "Unhandled Error" err="workloads failed after 2.492s with: function \"workload_info\" failed with an error" I0512 17:39:37.966698 1 controllerstatus.go:89] name=periodic-workloads healthy=false reason=PeriodicGatherFailed message=Source workloads could not be retrieved: function "workload_info" failed with an error I0512 17:39:37.966712 1 periodic.go:216] Running conditional gatherer I0512 17:39:37.972717 1 requests.go:294] Making HTTP GET request at: https://console.redhat.com/api/gathering/v2/4.21.9/gathering_rules I0512 17:39:37.980034 1 conditional_gatherer.go:107] Get "https://console.redhat.com/api/gathering/v2/4.21.9/gathering_rules": dial tcp: lookup console.redhat.com on 172.30.0.10:53: read udp 10.128.0.8:33680->172.30.0.10:53: read: connection refused E0512 17:39:37.980316 1 conditional_gatherer.go:322] unable to update alerts cache: open /var/run/configmaps/service-ca-bundle/service-ca.crt: no such file or directory I0512 17:39:37.980380 1 conditional_gatherer.go:384] updating version cache for conditional gatherer I0512 17:39:37.986287 1 conditional_gatherer.go:392] cluster version is '4.21.9' E0512 17:39:37.986309 1 conditional_gatherer.go:209] error checking conditions for a gathering rule: alerts cache is missing E0512 17:39:37.986317 1 conditional_gatherer.go:209] error checking conditions for a gathering rule: alerts cache is missing E0512 17:39:37.986320 1 conditional_gatherer.go:209] error checking conditions for a gathering rule: alerts cache is missing E0512 17:39:37.986323 1 conditional_gatherer.go:209] error checking conditions for a gathering rule: alerts cache is missing E0512 17:39:37.986327 1 conditional_gatherer.go:209] error checking conditions for a gathering rule: alerts cache is missing E0512 17:39:37.986330 1 conditional_gatherer.go:209] error checking conditions for a gathering rule: alerts cache is missing E0512 17:39:37.986333 1 conditional_gatherer.go:209] error checking conditions for a gathering rule: alerts cache is missing E0512 17:39:37.986336 1 conditional_gatherer.go:209] error checking conditions for a gathering rule: alerts cache is missing E0512 17:39:37.986339 1 conditional_gatherer.go:209] error checking conditions for a gathering rule: alerts cache is missing I0512 17:39:37.986357 1 tasks_processing.go:45] number of workers: 3 I0512 17:39:37.986371 1 tasks_processing.go:69] worker 2 listening for tasks. I0512 17:39:37.986375 1 tasks_processing.go:71] worker 2 working on conditional_gatherer_rules task. I0512 17:39:37.986380 1 tasks_processing.go:69] worker 0 listening for tasks. I0512 17:39:37.986392 1 tasks_processing.go:71] worker 0 working on remote_configuration task. I0512 17:39:37.986398 1 tasks_processing.go:71] worker 2 working on rapid_container_logs task. I0512 17:39:37.986397 1 tasks_processing.go:69] worker 1 listening for tasks. I0512 17:39:37.986407 1 tasks_processing.go:74] worker 1 stopped. I0512 17:39:37.986477 1 recorder.go:75] Recording insights-operator/conditional-gatherer-rules with fingerprint=7034af97e7e41c22e4b775abdd4b9066c8ebb19da33eb7f69f39bfd2eb5f6406 I0512 17:39:37.986491 1 gather.go:177] gatherer "conditional" function "conditional_gatherer_rules" took 1.128µs to process 1 records I0512 17:39:37.986523 1 recorder.go:75] Recording insights-operator/remote-configuration with fingerprint=0394430c431eec4d48bb1811a90918e95161d2282c59af26f2473613cc0959db I0512 17:39:37.986530 1 gather.go:177] gatherer "conditional" function "remote_configuration" took 1.307µs to process 1 records I0512 17:39:37.986537 1 tasks_processing.go:74] worker 0 stopped. I0512 17:39:37.986663 1 tasks_processing.go:74] worker 2 stopped. I0512 17:39:37.986675 1 gather.go:177] gatherer "conditional" function "rapid_container_logs" took 259.568µs to process 0 records I0512 17:39:37.986705 1 controllerstatus.go:89] name=periodic-conditional healthy=false reason=NotAvailable message=Get "https://console.redhat.com/api/gathering/v2/4.21.9/gathering_rules": dial tcp: lookup console.redhat.com on 172.30.0.10:53: read udp 10.128.0.8:33680->172.30.0.10:53: read: connection refused I0512 17:39:37.986725 1 recorder.go:75] Recording insights-operator/remote-configuration.json with fingerprint=359de9c990c741675cec72fda96b5c3682221efdb4799f5eaa6e9805bcd3b5c1 I0512 17:39:38.007628 1 recorder.go:75] Recording insights-operator/gathers with fingerprint=465adc313e4e41611d0d8a8fb761f74fcb1ecefc309b2b2331cb07d5dd3c2c35 I0512 17:39:38.007776 1 diskrecorder.go:70] Writing 100 records to /var/lib/insights-operator/insights-2026-05-12-173938.tar.gz I0512 17:39:38.015444 1 diskrecorder.go:51] Wrote 100 records to disk in 7ms I0512 17:39:38.015486 1 periodic.go:285] Gathering cluster info every 2h0m0s I0512 17:39:38.015503 1 periodic.go:286] Configuration is dataReporting: interval: 2h0m0s, uploadEndpoint: https://console.redhat.com/api/ingress/v1/upload, storagePath: /var/lib/insights-operator, downloadEndpoint: https://console.redhat.com/api/insights-results-aggregator/v2/cluster/%s/reports, conditionalGathererEndpoint: https://console.redhat.com/api/gathering/v2/%s/gathering_rules, obfuscation: [] sca: disabled: false, endpoint: https://api.openshift.com/api/accounts_mgmt/v1/entitlement_certificates, interval: 8h0m0s alerting: disabled: false clusterTransfer: endpoint: https://api.openshift.com/api/accounts_mgmt/v1/cluster_transfers/, interval: 12h0m0s proxy: httpProxy: , httpsProxy: , noProxy: I0512 17:39:39.458537 1 configmapobserver.go:84] configmaps "insights-config" not found I0512 17:39:39.661396 1 configmapobserver.go:84] configmaps "insights-config" not found I0512 17:39:48.271684 1 configmapobserver.go:84] configmaps "insights-config" not found I0512 17:40:41.408845 1 observer_polling.go:111] Observed file "/var/run/secrets/serving-cert/tls.crt" has been created (hash="46e23fa9592f0b6da9734bb0524cb72ab3fcd1a91b57f6a703d3e9a50ee7b39a") W0512 17:40:41.408879 1 builder.go:160] Restart triggered because of file /var/run/secrets/serving-cert/tls.crt was created I0512 17:40:41.408923 1 observer_polling.go:111] Observed file "/var/run/secrets/serving-cert/tls.key" has been created (hash="4a21d3c85ec0b695aa3b7e8487feec3e8dc6a54d804fb534a9d9df62e069e864") I0512 17:40:41.408937 1 simple_featuregate_reader.go:177] Shutting down feature-gate-detector I0512 17:40:41.408965 1 periodic.go:177] Shutting down I0512 17:40:41.408995 1 base_controller.go:181] Shutting down LoggingSyncer ... I0512 17:40:41.409003 1 genericapiserver.go:548] "[graceful-termination] shutdown event" name="ShutdownInitiated" I0512 17:40:41.409014 1 genericapiserver.go:693] "[graceful-termination] pre-shutdown hooks completed" name="PreShutdownHooksStopped" I0512 17:40:41.409021 1 observer_polling.go:111] Observed file "/var/run/configmaps/service-ca-bundle/service-ca.crt" has been created (hash="bd37c853fa9cf6ae76d53e628dce0b8beedc6a2e56dc68f4f98448b2ea9a1613") I0512 17:40:41.409040 1 genericapiserver.go:651] "[graceful-termination] not going to wait for active watch request(s) to drain"