I0505 09:35:38.117393 1 cmd.go:253] Using service-serving-cert provided certificates I0505 09:35:38.117846 1 observer_polling.go:159] Starting file observer I0505 09:35:38.480961 1 operator.go:76] Starting insights-operator v0.0.0-master+$Format:%H$ I0505 09:35:38.481149 1 legacy_config.go:327] Current config: {"report":false,"storagePath":"/var/lib/insights-operator","interval":"2h","endpoint":"https://console.redhat.com/api/ingress/v1/upload","conditionalGathererEndpoint":"https://console.redhat.com/api/gathering/v2/%s/gathering_rules","pull_report":{"endpoint":"https://console.redhat.com/api/insights-results-aggregator/v2/cluster/%s/reports","delay":"60s","timeout":"3000s","min_retry":"30s"},"impersonate":"system:serviceaccount:openshift-insights:gather","enableGlobalObfuscation":false,"ocm":{"scaEndpoint":"https://api.openshift.com/api/accounts_mgmt/v1/entitlement_certificates","scaInterval":"8h","scaDisabled":false,"clusterTransferEndpoint":"https://api.openshift.com/api/accounts_mgmt/v1/cluster_transfers/","clusterTransferInterval":"12h"},"disableInsightsAlerts":false,"processingStatusEndpoint":"https://console.redhat.com/api/insights-results-aggregator/v2/cluster/%s/request/%s/status","reportEndpointTechPreview":"https://console.redhat.com/api/insights-results-aggregator/v2/cluster/%s/request/%s/report"} I0505 09:35:38.481724 1 secure_serving.go:57] Forcing use of http/1.1 only W0505 09:35:38.481746 1 secure_serving.go:69] Use of insecure cipher 'TLS_ECDHE_ECDSA_WITH_AES_128_CBC_SHA256' detected. W0505 09:35:38.481751 1 secure_serving.go:69] Use of insecure cipher 'TLS_ECDHE_RSA_WITH_AES_128_CBC_SHA256' detected. W0505 09:35:38.481754 1 secure_serving.go:69] Use of insecure cipher 'TLS_RSA_WITH_AES_128_GCM_SHA256' detected. W0505 09:35:38.481757 1 secure_serving.go:69] Use of insecure cipher 'TLS_RSA_WITH_AES_256_GCM_SHA384' detected. W0505 09:35:38.481759 1 secure_serving.go:69] Use of insecure cipher 'TLS_RSA_WITH_AES_128_CBC_SHA' detected. W0505 09:35:38.481761 1 secure_serving.go:69] Use of insecure cipher 'TLS_RSA_WITH_AES_256_CBC_SHA' detected. I0505 09:35:38.481790 1 simple_featuregate_reader.go:171] Starting feature-gate-detector I0505 09:35:38.487990 1 requestheader_controller.go:180] Starting RequestHeaderAuthRequestController I0505 09:35:38.488013 1 shared_informer.go:313] Waiting for caches to sync for RequestHeaderAuthRequestController I0505 09:35:38.488029 1 configmap_cafile_content.go:205] "Starting controller" name="client-ca::kube-system::extension-apiserver-authentication::client-ca-file" I0505 09:35:38.488045 1 shared_informer.go:313] Waiting for caches to sync for client-ca::kube-system::extension-apiserver-authentication::client-ca-file I0505 09:35:38.488043 1 configmap_cafile_content.go:205] "Starting controller" name="client-ca::kube-system::extension-apiserver-authentication::requestheader-client-ca-file" I0505 09:35:38.488054 1 shared_informer.go:313] Waiting for caches to sync for client-ca::kube-system::extension-apiserver-authentication::requestheader-client-ca-file I0505 09:35:38.488044 1 operator.go:141] FeatureGates initialized: knownFeatureGates=[AdditionalRoutingCapabilities AdminNetworkPolicy AlibabaPlatform AzureWorkloadIdentity BuildCSIVolumes CPMSMachineNamePrefix ConsolePluginContentSecurityPolicy ExternalOIDC ExternalOIDCWithUIDAndExtraClaimMappings GCPClusterHostedDNSInstall GatewayAPI GatewayAPIController HighlyAvailableArbiter HyperShiftOnlyDynamicResourceAllocation ImageStreamImportMode ImageVolume KMSv1 MachineConfigNodes ManagedBootImages ManagedBootImagesAWS ManagedBootImagesAzure ManagedBootImagesvSphere MetricsCollectionProfiles NetworkDiagnosticsConfig NetworkLiveMigration NetworkSegmentation PinnedImages PreconfiguredUDNAddresses ProcMountType RouteAdvertisements RouteExternalCertificate ServiceAccountTokenNodeBinding SigstoreImageVerification SigstoreImageVerificationPKI StoragePerformantSecurityPolicy UpgradeStatus UserNamespacesPodSecurityStandards UserNamespacesSupport VSphereMultiDisk VSphereMultiNetworks VolumeAttributesClass AWSClusterHostedDNS AWSClusterHostedDNSInstall AWSDedicatedHosts AWSDualStackInstall AWSServiceLBNetworkSecurityGroup AutomatedEtcdBackup AzureClusterHostedDNSInstall AzureDedicatedHosts AzureDualStackInstall AzureMultiDisk BootImageSkewEnforcement BootcNodeManagement CBORServingAndStorage CRDCompatibilityRequirementOperator ClientsAllowCBOR ClientsPreferCBOR ClusterAPIInstall ClusterAPIInstallIBMCloud ClusterAPIMachineManagement ClusterAPIMachineManagementVSphere ClusterMonitoringConfig ClusterVersionOperatorConfiguration DNSNameResolver DualReplica DyanmicServiceEndpointIBMCloud EtcdBackendQuota EventTTL EventedPLEG Example Example2 ExternalSnapshotMetadata GCPClusterHostedDNS GCPCustomAPIEndpoints GCPCustomAPIEndpointsInstall GCPDualStackInstall ImageModeStatusReporting IngressControllerDynamicConfigurationManager InsightsConfig InsightsOnDemandDataGather IrreconcilableMachineConfig KMSEncryption KMSEncryptionProvider MachineAPIMigration MachineAPIOperatorDisableMachineHealthCheckController ManagedBootImagesCPMS MaxUnavailableStatefulSet MinimumKubeletVersion MixedCPUsAllocation MultiArchInstallAzure MultiDiskSetup MutableCSINodeAllocatableCount MutatingAdmissionPolicy NewOLM NewOLMBoxCutterRuntime NewOLMCatalogdAPIV1Metas NewOLMOwnSingleNamespace NewOLMPreflightPermissionChecks NewOLMWebhookProviderOpenshiftServiceCA NoRegistryClusterInstall NutanixMultiSubnets OSStreams OVNObservability OnPremDNSRecords OpenShiftPodSecurityAdmission ProvisioningRequestAvailable SELinuxMount ShortCertRotation SignatureStores TranslateStreamCloseWebsocketRequests VSphereConfigurableMaxAllowedBlockVolumesPerNode VSphereHostVMGroupZonal VSphereMixedNodeEnv VolumeGroupSnapshot] I0505 09:35:38.488986 1 event.go:377] Event(v1.ObjectReference{Kind:"Deployment", Namespace:"openshift-insights", Name:"insights-operator", UID:"ca054602-651a-45c8-902f-e5e43b2a0d0f", APIVersion:"apps/v1", ResourceVersion:"", FieldPath:""}): type: 'Normal' reason: 'FeatureGatesInitialized' FeatureGates updated to featuregates.Features{Enabled:[]v1.FeatureGateName{"AdditionalRoutingCapabilities", "AdminNetworkPolicy", "AlibabaPlatform", "AzureWorkloadIdentity", "BuildCSIVolumes", "CPMSMachineNamePrefix", "ConsolePluginContentSecurityPolicy", "ExternalOIDC", "ExternalOIDCWithUIDAndExtraClaimMappings", "GCPClusterHostedDNSInstall", "GatewayAPI", "GatewayAPIController", "HighlyAvailableArbiter", "HyperShiftOnlyDynamicResourceAllocation", "ImageStreamImportMode", "ImageVolume", "KMSv1", "MachineConfigNodes", "ManagedBootImages", "ManagedBootImagesAWS", "ManagedBootImagesAzure", "ManagedBootImagesvSphere", "MetricsCollectionProfiles", "NetworkDiagnosticsConfig", "NetworkLiveMigration", "NetworkSegmentation", "PinnedImages", "PreconfiguredUDNAddresses", "ProcMountType", "RouteAdvertisements", "RouteExternalCertificate", "ServiceAccountTokenNodeBinding", "SigstoreImageVerification", "SigstoreImageVerificationPKI", "StoragePerformantSecurityPolicy", "UpgradeStatus", "UserNamespacesPodSecurityStandards", "UserNamespacesSupport", "VSphereMultiDisk", "VSphereMultiNetworks", "VolumeAttributesClass"}, Disabled:[]v1.FeatureGateName{"AWSClusterHostedDNS", "AWSClusterHostedDNSInstall", "AWSDedicatedHosts", "AWSDualStackInstall", "AWSServiceLBNetworkSecurityGroup", "AutomatedEtcdBackup", "AzureClusterHostedDNSInstall", "AzureDedicatedHosts", "AzureDualStackInstall", "AzureMultiDisk", "BootImageSkewEnforcement", "BootcNodeManagement", "CBORServingAndStorage", "CRDCompatibilityRequirementOperator", "ClientsAllowCBOR", "ClientsPreferCBOR", "ClusterAPIInstall", "ClusterAPIInstallIBMCloud", "ClusterAPIMachineManagement", "ClusterAPIMachineManagementVSphere", "ClusterMonitoringConfig", "ClusterVersionOperatorConfiguration", "DNSNameResolver", "DualReplica", "DyanmicServiceEndpointIBMCloud", "EtcdBackendQuota", "EventTTL", "EventedPLEG", "Example", "Example2", "ExternalSnapshotMetadata", "GCPClusterHostedDNS", "GCPCustomAPIEndpoints", "GCPCustomAPIEndpointsInstall", "GCPDualStackInstall", "ImageModeStatusReporting", "IngressControllerDynamicConfigurationManager", "InsightsConfig", "InsightsOnDemandDataGather", "IrreconcilableMachineConfig", "KMSEncryption", "KMSEncryptionProvider", "MachineAPIMigration", "MachineAPIOperatorDisableMachineHealthCheckController", "ManagedBootImagesCPMS", "MaxUnavailableStatefulSet", "MinimumKubeletVersion", "MixedCPUsAllocation", "MultiArchInstallAzure", "MultiDiskSetup", "MutableCSINodeAllocatableCount", "MutatingAdmissionPolicy", "NewOLM", "NewOLMBoxCutterRuntime", "NewOLMCatalogdAPIV1Metas", "NewOLMOwnSingleNamespace", "NewOLMPreflightPermissionChecks", "NewOLMWebhookProviderOpenshiftServiceCA", "NoRegistryClusterInstall", "NutanixMultiSubnets", "OSStreams", "OVNObservability", "OnPremDNSRecords", "OpenShiftPodSecurityAdmission", "ProvisioningRequestAvailable", "SELinuxMount", "ShortCertRotation", "SignatureStores", "TranslateStreamCloseWebsocketRequests", "VSphereConfigurableMaxAllowedBlockVolumesPerNode", "VSphereHostVMGroupZonal", "VSphereMixedNodeEnv", "VolumeGroupSnapshot"}} I0505 09:35:38.489713 1 dynamic_serving_content.go:135] "Starting controller" name="serving-cert::/var/run/secrets/serving-cert/tls.crt::/var/run/secrets/serving-cert/tls.key" I0505 09:35:38.490790 1 secure_serving.go:213] Serving securely on [::]:8443 I0505 09:35:38.490826 1 tlsconfig.go:243] "Starting DynamicServingCertificateController" W0505 09:35:38.502121 1 configmapobserver.go:64] Cannot get the configuration config map: configmaps "insights-config" not found. Default configuration is used. I0505 09:35:38.502150 1 secretconfigobserver.go:216] Legacy configuration set: enabled=false endpoint=https://console.redhat.com/api/ingress/v1/upload conditional_gatherer_endpoint=https://console.redhat.com/api/gathering/v2/%s/gathering_rules interval=2h0m0s token=false reportEndpoint=https://console.redhat.com/api/insights-results-aggregator/v2/cluster/%s/reports initialPollingDelay=1m0s minRetryTime=30s pollingTimeout=50m0s processingStatusEndpoint=https://console.redhat.com/api/insights-results-aggregator/v2/cluster/%s/request/%s/status I0505 09:35:38.502272 1 base_controller.go:76] Waiting for caches to sync for ConfigController I0505 09:35:38.507788 1 secretconfigobserver.go:249] Found cloud.openshift.com token I0505 09:35:38.507810 1 secretconfigobserver.go:204] Legacy configuration updated: enabled=true endpoint=https://console.redhat.com/api/ingress/v1/upload conditional_gatherer_endpoint=https://console.redhat.com/api/gathering/v2/%s/gathering_rules interval=2h0m0s token=true reportEndpoint=https://console.redhat.com/api/insights-results-aggregator/v2/cluster/%s/reports initialPollingDelay=1m0s minRetryTime=30s pollingTimeout=50m0s processingStatusEndpoint=https://console.redhat.com/api/insights-results-aggregator/v2/cluster/%s/request/%s/status I0505 09:35:38.513176 1 secretconfigobserver.go:119] support secret does not exist I0505 09:35:38.522381 1 secretconfigobserver.go:249] Found cloud.openshift.com token I0505 09:35:38.527709 1 secretconfigobserver.go:119] support secret does not exist I0505 09:35:38.533209 1 recorder.go:176] Pruning old reports every 5h31m36s, max age is 288h0m0s I0505 09:35:38.538793 1 periodic.go:216] Running clusterconfig gatherer I0505 09:35:38.538799 1 controllerstatus.go:80] name=insightsuploader healthy=true reason= message= I0505 09:35:38.538816 1 insightsuploader.go:86] Reporting status periodically to https://console.redhat.com/api/ingress/v1/upload every 2h0m0s, starting in 1m30s I0505 09:35:38.538831 1 tasks_processing.go:45] number of workers: 64 I0505 09:35:38.538835 1 controllerstatus.go:80] name=insightsreport healthy=true reason= message= I0505 09:35:38.538840 1 insightsreport.go:296] Starting report retriever I0505 09:35:38.538844 1 insightsreport.go:298] Insights analysis reports will be downloaded from the https://console.redhat.com/api/insights-results-aggregator/v2/cluster/%s/reports endpoint with a delay of 1m0s I0505 09:35:38.538853 1 tasks_processing.go:69] worker 3 listening for tasks. I0505 09:35:38.538859 1 tasks_processing.go:69] worker 0 listening for tasks. I0505 09:35:38.538860 1 tasks_processing.go:69] worker 7 listening for tasks. I0505 09:35:38.538865 1 tasks_processing.go:69] worker 1 listening for tasks. I0505 09:35:38.538869 1 tasks_processing.go:69] worker 2 listening for tasks. I0505 09:35:38.538871 1 tasks_processing.go:71] worker 7 working on image task. I0505 09:35:38.538871 1 tasks_processing.go:69] worker 29 listening for tasks. I0505 09:35:38.538877 1 tasks_processing.go:71] worker 2 working on jaegers task. I0505 09:35:38.538880 1 tasks_processing.go:69] worker 42 listening for tasks. I0505 09:35:38.538882 1 tasks_processing.go:71] worker 29 working on authentication task. I0505 09:35:38.538883 1 tasks_processing.go:69] worker 41 listening for tasks. I0505 09:35:38.538890 1 tasks_processing.go:69] worker 43 listening for tasks. I0505 09:35:38.538876 1 tasks_processing.go:69] worker 53 listening for tasks. I0505 09:35:38.538895 1 tasks_processing.go:69] worker 49 listening for tasks. I0505 09:35:38.538900 1 tasks_processing.go:69] worker 50 listening for tasks. I0505 09:35:38.538904 1 tasks_processing.go:69] worker 51 listening for tasks. I0505 09:35:38.538905 1 tasks_processing.go:69] worker 48 listening for tasks. I0505 09:35:38.538912 1 tasks_processing.go:69] worker 59 listening for tasks. I0505 09:35:38.538909 1 tasks_processing.go:69] worker 30 listening for tasks. I0505 09:35:38.538918 1 tasks_processing.go:69] worker 54 listening for tasks. I0505 09:35:38.538919 1 tasks_processing.go:69] worker 32 listening for tasks. I0505 09:35:38.538914 1 tasks_processing.go:69] worker 31 listening for tasks. I0505 09:35:38.538924 1 tasks_processing.go:69] worker 55 listening for tasks. I0505 09:35:38.538921 1 tasks_processing.go:69] worker 52 listening for tasks. I0505 09:35:38.538930 1 tasks_processing.go:69] worker 56 listening for tasks. I0505 09:35:38.538932 1 tasks_processing.go:69] worker 33 listening for tasks. I0505 09:35:38.538925 1 tasks_processing.go:69] worker 44 listening for tasks. I0505 09:35:38.538930 1 tasks_processing.go:69] worker 45 listening for tasks. I0505 09:35:38.538938 1 tasks_processing.go:69] worker 57 listening for tasks. I0505 09:35:38.538940 1 tasks_processing.go:69] worker 34 listening for tasks. I0505 09:35:38.538943 1 tasks_processing.go:69] worker 47 listening for tasks. I0505 09:35:38.538945 1 tasks_processing.go:69] worker 58 listening for tasks. I0505 09:35:38.538935 1 tasks_processing.go:69] worker 46 listening for tasks. I0505 09:35:38.538946 1 tasks_processing.go:69] worker 37 listening for tasks. I0505 09:35:38.538950 1 tasks_processing.go:69] worker 38 listening for tasks. I0505 09:35:38.538952 1 tasks_processing.go:69] worker 4 listening for tasks. I0505 09:35:38.538951 1 tasks_processing.go:69] worker 35 listening for tasks. I0505 09:35:38.538957 1 tasks_processing.go:69] worker 36 listening for tasks. I0505 09:35:38.538946 1 tasks_processing.go:69] worker 62 listening for tasks. I0505 09:35:38.538962 1 tasks_processing.go:69] worker 63 listening for tasks. I0505 09:35:38.538963 1 tasks_processing.go:69] worker 40 listening for tasks. I0505 09:35:38.538964 1 tasks_processing.go:69] worker 60 listening for tasks. I0505 09:35:38.538971 1 tasks_processing.go:69] worker 61 listening for tasks. I0505 09:35:38.538974 1 tasks_processing.go:69] worker 15 listening for tasks. I0505 09:35:38.538975 1 tasks_processing.go:69] worker 9 listening for tasks. I0505 09:35:38.538978 1 tasks_processing.go:69] worker 16 listening for tasks. I0505 09:35:38.538956 1 tasks_processing.go:69] worker 39 listening for tasks. I0505 09:35:38.538957 1 tasks_processing.go:71] worker 3 working on metrics task. I0505 09:35:38.538982 1 tasks_processing.go:69] worker 11 listening for tasks. I0505 09:35:38.538984 1 tasks_processing.go:69] worker 13 listening for tasks. I0505 09:35:38.538987 1 tasks_processing.go:69] worker 8 listening for tasks. I0505 09:35:38.538989 1 tasks_processing.go:69] worker 14 listening for tasks. I0505 09:35:38.538988 1 tasks_processing.go:69] worker 18 listening for tasks. I0505 09:35:38.538998 1 tasks_processing.go:69] worker 19 listening for tasks. I0505 09:35:38.538996 1 tasks_processing.go:69] worker 17 listening for tasks. I0505 09:35:38.539001 1 tasks_processing.go:69] worker 25 listening for tasks. I0505 09:35:38.539002 1 tasks_processing.go:69] worker 23 listening for tasks. I0505 09:35:38.539007 1 tasks_processing.go:69] worker 28 listening for tasks. I0505 09:35:38.539010 1 tasks_processing.go:69] worker 20 listening for tasks. I0505 09:35:38.538963 1 tasks_processing.go:71] worker 0 working on storage_cluster task. I0505 09:35:38.539008 1 tasks_processing.go:69] worker 21 listening for tasks. I0505 09:35:38.538970 1 tasks_processing.go:69] worker 12 listening for tasks. I0505 09:35:38.538975 1 tasks_processing.go:69] worker 10 listening for tasks. I0505 09:35:38.538965 1 tasks_processing.go:69] worker 5 listening for tasks. I0505 09:35:38.538987 1 tasks_processing.go:69] worker 24 listening for tasks. I0505 09:35:38.538991 1 tasks_processing.go:69] worker 27 listening for tasks. I0505 09:35:38.538995 1 tasks_processing.go:69] worker 22 listening for tasks. I0505 09:35:38.538873 1 tasks_processing.go:71] worker 1 working on validating_webhook_configurations task. I0505 09:35:38.539006 1 tasks_processing.go:69] worker 6 listening for tasks. I0505 09:35:38.539092 1 tasks_processing.go:71] worker 22 working on container_images task. I0505 09:35:38.539100 1 tasks_processing.go:71] worker 41 working on networks task. I0505 09:35:38.539102 1 tasks_processing.go:71] worker 6 working on node_features task. I0505 09:35:38.539109 1 tasks_processing.go:71] worker 31 working on schedulers task. I0505 09:35:38.539008 1 tasks_processing.go:69] worker 26 listening for tasks. I0505 09:35:38.539090 1 tasks_processing.go:71] worker 37 working on openstack_controlplanes task. I0505 09:35:38.539158 1 tasks_processing.go:71] worker 50 working on qemu_kubevirt_launcher_logs task. I0505 09:35:38.539156 1 tasks_processing.go:71] worker 53 working on container_runtime_configs task. I0505 09:35:38.539166 1 tasks_processing.go:71] worker 38 working on image_pruners task. I0505 09:35:38.539174 1 tasks_processing.go:71] worker 15 working on openshift_machine_api_events task. I0505 09:35:38.539175 1 tasks_processing.go:71] worker 30 working on number_of_pods_and_netnamespaces_with_sdn_annotations task. I0505 09:35:38.539181 1 tasks_processing.go:71] worker 60 working on active_alerts task. I0505 09:35:38.539094 1 tasks_processing.go:71] worker 42 working on mutating_webhook_configurations task. I0505 09:35:38.539205 1 tasks_processing.go:71] worker 26 working on machine_configs task. I0505 09:35:38.539208 1 tasks_processing.go:71] worker 45 working on sap_pods task. I0505 09:35:38.539924 1 tasks_processing.go:71] worker 17 working on support_secret task. I0505 09:35:38.540309 1 tasks_processing.go:71] worker 48 working on version task. I0505 09:35:38.539951 1 tasks_processing.go:71] worker 24 working on pdbs task. I0505 09:35:38.540488 1 tasks_processing.go:71] worker 54 working on clusterroles task. I0505 09:35:38.539155 1 tasks_processing.go:71] worker 49 working on olm_operators task. I0505 09:35:38.540323 1 tasks_processing.go:71] worker 59 working on ingress task. I0505 09:35:38.540413 1 tasks_processing.go:71] worker 57 working on machines task. I0505 09:35:38.539162 1 tasks_processing.go:71] worker 40 working on node_logs task. I0505 09:35:38.539174 1 tasks_processing.go:71] worker 16 working on lokistack task. I0505 09:35:38.539166 1 tasks_processing.go:71] worker 61 working on openshift_logging task. I0505 09:35:38.541673 1 tasks_processing.go:71] worker 47 working on openstack_version task. I0505 09:35:38.539148 1 tasks_processing.go:71] worker 51 working on service_accounts task. I0505 09:35:38.539208 1 tasks_processing.go:71] worker 8 working on pod_network_connectivity_checks task. I0505 09:35:38.539150 1 tasks_processing.go:71] worker 43 working on machine_healthchecks task. I0505 09:35:38.539483 1 tasks_processing.go:71] worker 20 working on sap_datahubs task. I0505 09:35:38.539511 1 tasks_processing.go:71] worker 21 working on aggregated_monitoring_cr_names task. I0505 09:35:38.539513 1 tasks_processing.go:71] worker 14 working on machine_sets task. I0505 09:35:38.539597 1 tasks_processing.go:71] worker 12 working on operators_pods_and_events task. I0505 09:35:38.539645 1 tasks_processing.go:71] worker 18 working on infrastructures task. I0505 09:35:38.539677 1 tasks_processing.go:71] worker 10 working on cluster_apiserver task. I0505 09:35:38.539765 1 tasks_processing.go:71] worker 5 working on crds task. I0505 09:35:38.539785 1 tasks_processing.go:71] worker 19 working on cost_management_metrics_configs task. I0505 09:35:38.539957 1 tasks_processing.go:71] worker 27 working on feature_gates task. I0505 09:35:38.540342 1 tasks_processing.go:71] worker 58 working on openstack_dataplanenodesets task. I0505 09:35:38.540352 1 tasks_processing.go:71] worker 11 working on install_plans task. I0505 09:35:38.540410 1 tasks_processing.go:71] worker 39 working on silenced_alerts task. I0505 09:35:38.540426 1 tasks_processing.go:71] worker 9 working on machine_config_pools task. I0505 09:35:38.540517 1 tasks_processing.go:71] worker 36 working on ceph_cluster task. I0505 09:35:38.540536 1 tasks_processing.go:71] worker 56 working on storage_classes task. I0505 09:35:38.540538 1 tasks_processing.go:71] worker 46 working on operators task. I0505 09:35:38.540548 1 tasks_processing.go:71] worker 4 working on tsdb_status task. I0505 09:35:38.540553 1 tasks_processing.go:71] worker 55 working on dvo_metrics task. I0505 09:35:38.540600 1 tasks_processing.go:71] worker 35 working on nodenetworkstates task. I0505 09:35:38.539160 1 tasks_processing.go:71] worker 13 working on sap_config task. I0505 09:35:38.540613 1 tasks_processing.go:71] worker 33 working on image_registries task. I0505 09:35:38.540629 1 tasks_processing.go:71] worker 52 working on openstack_dataplanedeployments task. I0505 09:35:38.540637 1 tasks_processing.go:71] worker 44 working on machine_autoscalers task. I0505 09:35:38.540644 1 tasks_processing.go:71] worker 34 working on proxies task. I0505 09:35:38.540681 1 tasks_processing.go:71] worker 63 working on monitoring_persistent_volumes task. I0505 09:35:38.540685 1 tasks_processing.go:71] worker 25 working on nodenetworkconfigurationpolicies task. I0505 09:35:38.540690 1 tasks_processing.go:71] worker 23 working on config_maps task. I0505 09:35:38.540698 1 tasks_processing.go:71] worker 32 working on overlapping_namespace_uids task. I0505 09:35:38.540722 1 tasks_processing.go:71] worker 62 working on certificate_signing_requests task. I0505 09:35:38.541343 1 tasks_processing.go:71] worker 28 working on ingress_certificates task. I0505 09:35:38.544479 1 tasks_processing.go:71] worker 2 working on nodes task. I0505 09:35:38.544496 1 gather.go:177] gatherer "clusterconfig" function "jaegers" took 5.593834ms to process 0 records E0505 09:35:38.545700 1 gather_most_recent_metrics.go:87] Unable to retrieve most recent metrics: Get "https://prometheus-k8s.openshift-monitoring.svc:9091/federate?match%5B%5D=cluster_installer&match%5B%5D=namespace%3Acontainer_cpu_usage%3Asum&match%5B%5D=namespace%3Acontainer_memory_usage_bytes%3Asum&match%5B%5D=vsphere_node_hw_version_total&match%5B%5D=virt_platform&match%5B%5D=console_helm_installs_total&match%5B%5D=console_helm_upgrades_total&match%5B%5D=console_helm_uninstalls_total&match%5B%5D=openshift_apps_deploymentconfigs_strategy_total&match%5B%5D=etcd_server_slow_apply_total&match%5B%5D=etcd_server_slow_read_indexes_total&match%5B%5D=haproxy_exporter_server_threshold": dial tcp 172.30.119.64:9091: connect: connection refused I0505 09:35:38.545731 1 tasks_processing.go:71] worker 3 working on oauths task. E0505 09:35:38.545729 1 gather_prometheus_tsdb_status.go:49] Unable to tsdb status: Get "https://prometheus-k8s.openshift-monitoring.svc:9091/api/v1/status/tsdb": dial tcp 172.30.119.64:9091: connect: connection refused E0505 09:35:38.545751 1 gather.go:140] gatherer "clusterconfig" function "metrics" failed with the error: Get "https://prometheus-k8s.openshift-monitoring.svc:9091/federate?match%5B%5D=cluster_installer&match%5B%5D=namespace%3Acontainer_cpu_usage%3Asum&match%5B%5D=namespace%3Acontainer_memory_usage_bytes%3Asum&match%5B%5D=vsphere_node_hw_version_total&match%5B%5D=virt_platform&match%5B%5D=console_helm_installs_total&match%5B%5D=console_helm_upgrades_total&match%5B%5D=console_helm_uninstalls_total&match%5B%5D=openshift_apps_deploymentconfigs_strategy_total&match%5B%5D=etcd_server_slow_apply_total&match%5B%5D=etcd_server_slow_read_indexes_total&match%5B%5D=haproxy_exporter_server_threshold": dial tcp 172.30.119.64:9091: connect: connection refused I0505 09:35:38.545789 1 gather.go:177] gatherer "clusterconfig" function "metrics" took 6.736692ms to process 0 records E0505 09:35:38.545812 1 gather.go:140] gatherer "clusterconfig" function "tsdb_status" failed with the error: Get "https://prometheus-k8s.openshift-monitoring.svc:9091/api/v1/status/tsdb": dial tcp 172.30.119.64:9091: connect: connection refused I0505 09:35:38.545827 1 gather.go:177] gatherer "clusterconfig" function "tsdb_status" took 2.316857ms to process 0 records I0505 09:35:38.545840 1 tasks_processing.go:74] worker 4 stopped. I0505 09:35:38.558362 1 tasks_processing.go:74] worker 45 stopped. I0505 09:35:38.558375 1 gather.go:177] gatherer "clusterconfig" function "sap_pods" took 18.51644ms to process 0 records I0505 09:35:38.562314 1 tasks_processing.go:74] worker 8 stopped. E0505 09:35:38.562331 1 gather.go:140] gatherer "clusterconfig" function "pod_network_connectivity_checks" failed with the error: the server could not find the requested resource (get podnetworkconnectivitychecks.controlplane.operator.openshift.io) I0505 09:35:38.562340 1 gather.go:177] gatherer "clusterconfig" function "pod_network_connectivity_checks" took 20.333192ms to process 0 records I0505 09:35:38.562554 1 tasks_processing.go:74] worker 7 stopped. I0505 09:35:38.562668 1 recorder.go:75] Recording config/image with fingerprint=1bace8cf843c4a78f7ab92db279cadc8be72358fe0390716523bb9981465f9a3 I0505 09:35:38.562683 1 gather.go:177] gatherer "clusterconfig" function "image" took 23.670171ms to process 1 records I0505 09:35:38.562767 1 tasks_processing.go:74] worker 29 stopped. I0505 09:35:38.562864 1 recorder.go:75] Recording config/authentication with fingerprint=71b9ae1a9b1c929e222b8d59e718b93f3cb25322bb34a008964fb69b881a8baf I0505 09:35:38.562876 1 gather.go:177] gatherer "clusterconfig" function "authentication" took 23.761335ms to process 1 records E0505 09:35:38.571999 1 gather_node_features.go:86] GatherNodeFeatures: NodeFeatures resource not found in openshift-nfd namespace (may not be installed) I0505 09:35:38.572013 1 tasks_processing.go:74] worker 6 stopped. I0505 09:35:38.572022 1 gather.go:177] gatherer "clusterconfig" function "node_features" took 32.902841ms to process 0 records I0505 09:35:38.572316 1 controller.go:129] Initializing last reported time to 0001-01-01T00:00:00Z I0505 09:35:38.572334 1 controller.go:368] The initial operator extension status is healthy I0505 09:35:38.572343 1 controller.go:254] Source periodic-clusterconfig *controllerstatus.Simple is not ready I0505 09:35:38.572347 1 controller.go:254] Source periodic-conditional *controllerstatus.Simple is not ready I0505 09:35:38.572349 1 controller.go:254] Source periodic-workloads *controllerstatus.Simple is not ready I0505 09:35:38.572364 1 controller.go:531] The operator is still being initialized I0505 09:35:38.572372 1 controller.go:554] The operator is healthy I0505 09:35:38.572430 1 sca.go:136] Pulling SCA certificates from https://api.openshift.com/api/accounts_mgmt/v1/entitlement_certificates. Next check is in 8h0m0s I0505 09:35:38.572441 1 cluster_transfer.go:83] checking the availability of cluster transfer. Next check is in 12h0m0s W0505 09:35:38.572642 1 operator.go:328] started I0505 09:35:38.572693 1 base_controller.go:76] Waiting for caches to sync for LoggingSyncer I0505 09:35:38.577666 1 tasks_processing.go:74] worker 53 stopped. I0505 09:35:38.577683 1 gather.go:177] gatherer "clusterconfig" function "container_runtime_configs" took 38.492933ms to process 0 records I0505 09:35:38.577693 1 gather.go:177] gatherer "clusterconfig" function "machine_config_pools" took 34.489014ms to process 0 records I0505 09:35:38.577702 1 tasks_processing.go:74] worker 9 stopped. I0505 09:35:38.577730 1 tasks_processing.go:74] worker 35 stopped. I0505 09:35:38.577745 1 gather.go:177] gatherer "clusterconfig" function "nodenetworkstates" took 34.248021ms to process 0 records I0505 09:35:38.577778 1 tasks_processing.go:74] worker 36 stopped. I0505 09:35:38.577785 1 gather.go:177] gatherer "clusterconfig" function "ceph_cluster" took 34.491017ms to process 0 records I0505 09:35:38.577793 1 gather.go:177] gatherer "clusterconfig" function "storage_cluster" took 38.761791ms to process 0 records I0505 09:35:38.577798 1 tasks_processing.go:74] worker 0 stopped. I0505 09:35:38.577829 1 tasks_processing.go:74] worker 43 stopped. E0505 09:35:38.577841 1 gather.go:140] gatherer "clusterconfig" function "machine_healthchecks" failed with the error: machinehealthchecks.machine.openshift.io is forbidden: User "system:serviceaccount:openshift-insights:gather" cannot list resource "machinehealthchecks" in API group "machine.openshift.io" at the cluster scope I0505 09:35:38.577849 1 gather.go:177] gatherer "clusterconfig" function "machine_healthchecks" took 35.790304ms to process 0 records E0505 09:35:38.577856 1 gather.go:140] gatherer "clusterconfig" function "machines" failed with the error: machines.machine.openshift.io is forbidden: User "system:serviceaccount:openshift-insights:gather" cannot list resource "machines" in API group "machine.openshift.io" at the cluster scope I0505 09:35:38.577861 1 gather.go:177] gatherer "clusterconfig" function "machines" took 36.475277ms to process 0 records I0505 09:35:38.577866 1 tasks_processing.go:74] worker 57 stopped. I0505 09:35:38.578063 1 tasks_processing.go:74] worker 52 stopped. I0505 09:35:38.578076 1 gather.go:177] gatherer "clusterconfig" function "openstack_dataplanedeployments" took 34.433346ms to process 0 records I0505 09:35:38.578965 1 tasks_processing.go:74] worker 59 stopped. I0505 09:35:38.579128 1 recorder.go:75] Recording config/ingress with fingerprint=fb36c65214e7da274d1a2895b4187f6a90183f4cc45ae6eac1148f164945c97a I0505 09:35:38.579150 1 gather.go:177] gatherer "clusterconfig" function "ingress" took 37.819916ms to process 1 records I0505 09:35:38.586006 1 tasks_processing.go:74] worker 24 stopped. I0505 09:35:38.586137 1 recorder.go:75] Recording config/pdbs/openshift-console/console with fingerprint=7b067910b893882731e3d5286c400de08ab690876c37e6febfd76bcedd9ce1b7 I0505 09:35:38.586167 1 recorder.go:75] Recording config/pdbs/openshift-console/downloads with fingerprint=4c7416ce6de11217247d3d23069fec6584b39f222fd42f923da775eb5063a9af I0505 09:35:38.586198 1 recorder.go:75] Recording config/pdbs/openshift-image-registry/image-registry with fingerprint=1a8624f63be0dadd274b653c494921649a8cf2bef569b15603aa7432865c8fd1 I0505 09:35:38.586227 1 recorder.go:75] Recording config/pdbs/openshift-ingress/router-default with fingerprint=83ddb15f993c2bbb31476c995fc885d47ea7f13fdbff2437ff0c44c3462a6a8a I0505 09:35:38.586258 1 recorder.go:75] Recording config/pdbs/openshift-monitoring/alertmanager-main with fingerprint=51376272bd6d6d46f4a5903984957b63a82893f32791aebf2c358ff60b66bf16 I0505 09:35:38.586275 1 recorder.go:75] Recording config/pdbs/openshift-monitoring/metrics-server with fingerprint=2e9b5504522b1096b35e6ad158656c0201571407fad914e77edc4e4f0bd93fa1 I0505 09:35:38.586301 1 recorder.go:75] Recording config/pdbs/openshift-monitoring/monitoring-plugin with fingerprint=4a54f708f7a92976b0a1181fd42c721bc51950415aef5d0d7809f332135f9e9e I0505 09:35:38.586324 1 recorder.go:75] Recording config/pdbs/openshift-monitoring/prometheus-k8s with fingerprint=88c2c255a8f9d1af3316612aed45f3a4e0ce6bbcd4f6a43bb7e6873080a94bff I0505 09:35:38.586343 1 recorder.go:75] Recording config/pdbs/openshift-monitoring/prometheus-operator-admission-webhook with fingerprint=f89c414c452f99a3804dcbd11cc4948e66eaa914a9293c974797b0d3e28c24ae I0505 09:35:38.586364 1 recorder.go:75] Recording config/pdbs/openshift-monitoring/thanos-querier-pdb with fingerprint=4489df4fda0c3e00509448f56f6cabaac361bfbc35a49774dd4755f95ccfdf68 I0505 09:35:38.586394 1 recorder.go:75] Recording config/pdbs/openshift-operator-lifecycle-manager/packageserver-pdb with fingerprint=da57a69410370cf9af738fc928fcb16a3eba0cf4d367866a28213ab0f4ee2ff0 I0505 09:35:38.586408 1 gather.go:177] gatherer "clusterconfig" function "pdbs" took 45.520636ms to process 11 records I0505 09:35:38.586599 1 tasks_processing.go:74] worker 41 stopped. I0505 09:35:38.586646 1 recorder.go:75] Recording config/network with fingerprint=897bfb81e9608e9ca30ffc7f1fd4e3744903bf3e619ee2fa890f9148ad51528b I0505 09:35:38.586666 1 gather.go:177] gatherer "clusterconfig" function "networks" took 47.115845ms to process 1 records I0505 09:35:38.586748 1 recorder.go:75] Recording config/schedulers/cluster with fingerprint=14081e8632086cd22ce5c443dd8d625d8698c48d7d65515d1ac871d46a16e407 I0505 09:35:38.586763 1 gather.go:177] gatherer "clusterconfig" function "schedulers" took 47.433225ms to process 1 records I0505 09:35:38.586772 1 gather.go:177] gatherer "clusterconfig" function "openstack_controlplanes" took 47.545266ms to process 0 records I0505 09:35:38.586783 1 tasks_processing.go:74] worker 37 stopped. I0505 09:35:38.586792 1 tasks_processing.go:74] worker 31 stopped. I0505 09:35:38.586833 1 tasks_processing.go:74] worker 47 stopped. I0505 09:35:38.586847 1 gather.go:177] gatherer "clusterconfig" function "openstack_version" took 45.139671ms to process 0 records I0505 09:35:38.587326 1 gather.go:177] gatherer "clusterconfig" function "machine_autoscalers" took 43.189904ms to process 0 records I0505 09:35:38.587397 1 tasks_processing.go:74] worker 44 stopped. I0505 09:35:38.587480 1 tasks_processing.go:74] worker 1 stopped. I0505 09:35:38.587660 1 recorder.go:75] Recording config/validatingwebhookconfigurations/alertmanagerconfigs.openshift.io with fingerprint=81b47fb7f0c5add4e442aeba67aecfdb5d89a637e12d25d18856ae1d46c289e5 I0505 09:35:38.587790 1 recorder.go:75] Recording config/validatingwebhookconfigurations/multus.openshift.io with fingerprint=b1fae831b6b24da41fad44b4fcea3a01d382457cab406c80927092bec6361778 I0505 09:35:38.587927 1 recorder.go:75] Recording config/validatingwebhookconfigurations/network-node-identity.openshift.io with fingerprint=fd8bf0743cb4901482ebab784aabb3a023133886386a94575a57294b3dc8562b I0505 09:35:38.587985 1 recorder.go:75] Recording config/validatingwebhookconfigurations/performance-addon-operator with fingerprint=8b9946b12bf6fc587e796fe1c3656f84ca50278d258d3398bd3de4f57b5cae92 I0505 09:35:38.588026 1 recorder.go:75] Recording config/validatingwebhookconfigurations/prometheusrules.openshift.io with fingerprint=7a1dcd15822386492619b570b8d46cd3cc5b98c5a2512c76ac930f2f51068480 I0505 09:35:38.588071 1 recorder.go:75] Recording config/validatingwebhookconfigurations/sre-clusterrolebindings-validation with fingerprint=c4a38319d84a6f30cf60c431f3fef51368e2c0361f270331a4dd8634149289b3 I0505 09:35:38.588150 1 recorder.go:75] Recording config/validatingwebhookconfigurations/sre-clusterroles-validation with fingerprint=f3b295e2ec1ef875ab9e737b02f8b5024af5a12e40025276db66c004fdea4908 I0505 09:35:38.588199 1 recorder.go:75] Recording config/validatingwebhookconfigurations/sre-ingress-config-validation with fingerprint=eeeafbdec5252ba8ce02d729ae647b4305df3be3bcd00542cafd954454844c67 I0505 09:35:38.588249 1 shared_informer.go:320] Caches are synced for RequestHeaderAuthRequestController I0505 09:35:38.588240 1 shared_informer.go:320] Caches are synced for client-ca::kube-system::extension-apiserver-authentication::requestheader-client-ca-file I0505 09:35:38.588282 1 recorder.go:75] Recording config/validatingwebhookconfigurations/sre-network-operator-validation with fingerprint=90e3d3d88b39a17e3f274af2193fb208d95dfda8df4d90ceb82520ffa76f7ce9 I0505 09:35:38.588318 1 shared_informer.go:320] Caches are synced for client-ca::kube-system::extension-apiserver-authentication::client-ca-file I0505 09:35:38.588344 1 recorder.go:75] Recording config/validatingwebhookconfigurations/sre-regular-user-validation with fingerprint=7a5027e40698ffe6ee18943f39eeb3dfb9ff96a4fa5c52c03d3ea190356951da I0505 09:35:38.588401 1 recorder.go:75] Recording config/validatingwebhookconfigurations/sre-scc-validation with fingerprint=523dbf3671fe1558957bbdf97fbe63d38f3f8e37680f674413db7f745d7794dc I0505 09:35:38.588473 1 recorder.go:75] Recording config/validatingwebhookconfigurations/sre-serviceaccount-validation with fingerprint=21059ccf18c7ce2dde81f4d8d81d9505b61b5bbea9162e2c11390f7d13a65690 I0505 09:35:38.588577 1 recorder.go:75] Recording config/validatingwebhookconfigurations/sre-techpreviewnoupgrade-validation with fingerprint=2fa539536e32f5adafba12a3f70af2113153ab7ddab80ccdc21b3dce32515d20 I0505 09:35:38.589135 1 gather.go:177] gatherer "clusterconfig" function "validating_webhook_configurations" took 48.247317ms to process 13 records I0505 09:35:38.589775 1 gather.go:177] gatherer "clusterconfig" function "sap_config" took 43.734796ms to process 0 records I0505 09:35:38.589800 1 tasks_processing.go:74] worker 13 stopped. I0505 09:35:38.589881 1 tasks_processing.go:74] worker 42 stopped. I0505 09:35:38.589916 1 recorder.go:75] Recording config/mutatingwebhookconfigurations/aws-pod-identity with fingerprint=46480230e198b3b0e8b018ac8f688f3b103935d0e5a3e7a52bf7b6edccc6e471 I0505 09:35:38.589966 1 recorder.go:75] Recording config/mutatingwebhookconfigurations/sre-podimagespec-mutation with fingerprint=af05a01ade80ac1e117d3befd83d100c71d991f75de4dd4029ef93d347285a31 I0505 09:35:38.590001 1 recorder.go:75] Recording config/mutatingwebhookconfigurations/sre-service-mutation with fingerprint=596f8d000fb1df923bdbc96f9dc7e0fde1876cf2e411abc75437392e63c280b1 I0505 09:35:38.590009 1 gather.go:177] gatherer "clusterconfig" function "mutating_webhook_configurations" took 49.164678ms to process 3 records I0505 09:35:38.594889 1 tasks_processing.go:74] worker 25 stopped. I0505 09:35:38.594902 1 gather.go:177] gatherer "clusterconfig" function "nodenetworkconfigurationpolicies" took 50.948318ms to process 0 records I0505 09:35:38.595978 1 tasks_processing.go:74] worker 34 stopped. I0505 09:35:38.596064 1 recorder.go:75] Recording config/proxy with fingerprint=bb4b47e95d1862f001c0eafb295ed393f216eb2df514df5fba570bcbf977cfa9 I0505 09:35:38.596080 1 gather.go:177] gatherer "clusterconfig" function "proxies" took 52.238589ms to process 1 records I0505 09:35:38.596298 1 tasks_processing.go:74] worker 15 stopped. I0505 09:35:38.596313 1 gather.go:177] gatherer "clusterconfig" function "openshift_machine_api_events" took 57.111575ms to process 0 records I0505 09:35:38.599077 1 tasks_processing.go:74] worker 38 stopped. I0505 09:35:38.599387 1 recorder.go:75] Recording config/clusteroperator/imageregistry.operator.openshift.io/imagepruner/cluster with fingerprint=9456439b06ffe32a650a2915134c45a57a667c5e1fa2fb523446c2eefaac8b7b I0505 09:35:38.599405 1 gather.go:177] gatherer "clusterconfig" function "image_pruners" took 59.896263ms to process 1 records I0505 09:35:38.599479 1 tasks_processing.go:74] worker 33 stopped. I0505 09:35:38.599784 1 recorder.go:75] Recording config/clusteroperator/imageregistry.operator.openshift.io/config/cluster with fingerprint=528e04eaabab7155fb349a7b6af4de0523a6c7042d4df319a2b0703222a629f9 I0505 09:35:38.599797 1 gather.go:177] gatherer "clusterconfig" function "image_registries" took 55.863821ms to process 1 records I0505 09:35:38.601522 1 tasks_processing.go:74] worker 14 stopped. I0505 09:35:38.601537 1 gather.go:177] gatherer "clusterconfig" function "machine_sets" took 59.398986ms to process 0 records I0505 09:35:38.602331 1 base_controller.go:82] Caches are synced for ConfigController I0505 09:35:38.602349 1 base_controller.go:119] Starting #1 worker of ConfigController controller ... I0505 09:35:38.602875 1 tasks_processing.go:74] worker 16 stopped. I0505 09:35:38.602887 1 gather.go:177] gatherer "clusterconfig" function "lokistack" took 61.449168ms to process 0 records W0505 09:35:38.603342 1 gather_dvo_metrics.go:210] Failed to read the DVO metrics. Trying again. I0505 09:35:38.615332 1 tasks_processing.go:74] worker 20 stopped. I0505 09:35:38.615352 1 gather.go:177] gatherer "clusterconfig" function "sap_datahubs" took 73.279831ms to process 0 records I0505 09:35:38.615962 1 tasks_processing.go:74] worker 58 stopped. I0505 09:35:38.615973 1 gather.go:177] gatherer "clusterconfig" function "openstack_dataplanenodesets" took 72.967838ms to process 0 records I0505 09:35:38.616013 1 tasks_processing.go:74] worker 18 stopped. I0505 09:35:38.616672 1 recorder.go:75] Recording config/infrastructure with fingerprint=8fe077387f750783fd287d28c42549ccc3efc3c8a67c3b7e9cee87806a5e419b I0505 09:35:38.616688 1 gather.go:177] gatherer "clusterconfig" function "infrastructures" took 73.826996ms to process 1 records I0505 09:35:38.616770 1 tasks_processing.go:74] worker 10 stopped. I0505 09:35:38.616793 1 recorder.go:75] Recording config/apiserver with fingerprint=05c47f2b1685374d2e5af96f3f818472a11fa6e2d5788516db2c4e8578228c92 I0505 09:35:38.616801 1 gather.go:177] gatherer "clusterconfig" function "cluster_apiserver" took 73.764149ms to process 1 records I0505 09:35:38.620019 1 tasks_processing.go:74] worker 3 stopped. I0505 09:35:38.620219 1 recorder.go:75] Recording config/oauth with fingerprint=dcb6cc8d40633d40b377ac3da21fb6aaa18e55d3cbee0f3be1bcd230ecb0364a I0505 09:35:38.620232 1 gather.go:177] gatherer "clusterconfig" function "oauths" took 74.274351ms to process 1 records I0505 09:35:38.620241 1 gather.go:177] gatherer "clusterconfig" function "cost_management_metrics_configs" took 77.590822ms to process 0 records I0505 09:35:38.620247 1 tasks_processing.go:74] worker 19 stopped. I0505 09:35:38.620572 1 controller.go:254] Source periodic-clusterconfig *controllerstatus.Simple is not ready I0505 09:35:38.620582 1 controller.go:254] Source periodic-conditional *controllerstatus.Simple is not ready I0505 09:35:38.620585 1 controller.go:254] Source periodic-workloads *controllerstatus.Simple is not ready I0505 09:35:38.620591 1 controller.go:254] Source scaController *sca.Controller is not ready I0505 09:35:38.620594 1 controller.go:254] Source clusterTransferController *clustertransfer.Controller is not ready I0505 09:35:38.620628 1 controller.go:531] The operator is still being initialized I0505 09:35:38.620636 1 controller.go:554] The operator is healthy I0505 09:35:38.621629 1 tasks_processing.go:74] worker 56 stopped. I0505 09:35:38.621714 1 recorder.go:75] Recording config/storage/storageclasses/gp2-csi with fingerprint=fa55986c3e6219dcef5bac081b4433311a77b016d87564e2f7b799fbda49abcf I0505 09:35:38.621735 1 recorder.go:75] Recording config/storage/storageclasses/gp3-csi with fingerprint=a9bbe04424f98841d899925e7b7c23474473b49f466659350509254eb09ea109 I0505 09:35:38.621742 1 gather.go:177] gatherer "clusterconfig" function "storage_classes" took 78.248572ms to process 2 records I0505 09:35:38.633194 1 tasks_processing.go:74] worker 63 stopped. I0505 09:35:38.633209 1 gather.go:177] gatherer "clusterconfig" function "monitoring_persistent_volumes" took 89.351215ms to process 0 records I0505 09:35:38.633624 1 tasks_processing.go:74] worker 61 stopped. I0505 09:35:38.633644 1 gather.go:177] gatherer "clusterconfig" function "openshift_logging" took 91.976828ms to process 0 records E0505 09:35:38.633662 1 gather.go:140] gatherer "clusterconfig" function "support_secret" failed with the error: secrets "support" not found I0505 09:35:38.633674 1 gather.go:177] gatherer "clusterconfig" function "support_secret" took 93.635296ms to process 0 records E0505 09:35:38.633701 1 gather.go:140] gatherer "clusterconfig" function "machine_configs" failed with the error: getting MachineConfigPools failed: the server could not find the requested resource (get machineconfigpools.machineconfiguration.openshift.io) I0505 09:35:38.633740 1 recorder.go:75] Recording aggregated/unused_machine_configs_count with fingerprint=4bfc9fa984e5dfcd45848faaf05269de7619bf42edf9f781751af5ee05c1a499 I0505 09:35:38.633754 1 gather.go:177] gatherer "clusterconfig" function "machine_configs" took 94.446867ms to process 1 records I0505 09:35:38.633759 1 gather.go:177] gatherer "clusterconfig" function "node_logs" took 92.361071ms to process 0 records I0505 09:35:38.633765 1 tasks_processing.go:74] worker 40 stopped. I0505 09:35:38.633766 1 tasks_processing.go:74] worker 17 stopped. I0505 09:35:38.633769 1 tasks_processing.go:74] worker 26 stopped. I0505 09:35:38.634111 1 gather_logs.go:145] no pods in namespace were found I0505 09:35:38.634136 1 tasks_processing.go:74] worker 50 stopped. I0505 09:35:38.634148 1 gather.go:177] gatherer "clusterconfig" function "qemu_kubevirt_launcher_logs" took 94.967666ms to process 0 records I0505 09:35:38.634539 1 tasks_processing.go:74] worker 2 stopped. I0505 09:35:38.634544 1 configmapobserver.go:84] configmaps "insights-config" not found I0505 09:35:38.635290 1 recorder.go:75] Recording config/node/ip-10-0-0-57.ec2.internal with fingerprint=0781fdd6414db565a54163a300d67a59713a5005b116c5b9e4738239c0ad3566 I0505 09:35:38.635511 1 recorder.go:75] Recording config/node/ip-10-0-1-134.ec2.internal with fingerprint=b44e79c2ef0709cad1a70914eaf3c770b9143af9932a5914d35abb7791564fe1 I0505 09:35:38.635720 1 recorder.go:75] Recording config/node/ip-10-0-2-147.ec2.internal with fingerprint=b2a23ebf9c9163c78a30b506fed7fe9a71b2a7c583c8722e42bb04312b54d3b0 I0505 09:35:38.635740 1 gather.go:177] gatherer "clusterconfig" function "nodes" took 90.044928ms to process 3 records I0505 09:35:38.637043 1 tasks_processing.go:74] worker 49 stopped. I0505 09:35:38.637096 1 recorder.go:75] Recording config/olm_operators with fingerprint=f39ff972a84bf61475b73d20e705cf4dbc1f852cb98611904e835d56836342e2 I0505 09:35:38.637115 1 gather.go:177] gatherer "clusterconfig" function "olm_operators" took 96.196856ms to process 1 records I0505 09:35:38.637350 1 tasks_processing.go:74] worker 54 stopped. I0505 09:35:38.637526 1 recorder.go:75] Recording cluster-scoped-resources/rbac.authorization.k8s.io/clusterroles/admin with fingerprint=6ac1b3c397cd2f6e33afa11527c35963af80c4c898c52a566b387060d2a67a84 I0505 09:35:38.637610 1 recorder.go:75] Recording cluster-scoped-resources/rbac.authorization.k8s.io/clusterroles/edit with fingerprint=772da0d0eca4dc0e46cf3a33f192fec948015ac39224c0eea9da3f5495440749 I0505 09:35:38.637618 1 gather.go:177] gatherer "clusterconfig" function "clusterroles" took 96.786866ms to process 2 records I0505 09:35:38.637773 1 tasks_processing.go:74] worker 62 stopped. I0505 09:35:38.637790 1 gather.go:177] gatherer "clusterconfig" function "certificate_signing_requests" took 93.753058ms to process 0 records I0505 09:35:38.663795 1 tasks_processing.go:74] worker 48 stopped. I0505 09:35:38.664118 1 recorder.go:75] Recording config/version with fingerprint=36cc8d4ddafe59051bf4f18b89e0b53a0a6e85a27854d99082d461ca9ff02f94 I0505 09:35:38.664138 1 recorder.go:75] Recording config/id with fingerprint=67168ded8c75846ad78a8ce17bfd048cc7b9f7869c61fb47d7c06df572b53fbe I0505 09:35:38.664147 1 gather.go:177] gatherer "clusterconfig" function "version" took 123.427073ms to process 2 records I0505 09:35:38.664642 1 tasks_processing.go:74] worker 32 stopped. I0505 09:35:38.664705 1 recorder.go:75] Recording config/namespaces_with_overlapping_uids with fingerprint=4f53cda18c2baa0c0354bb5f9a3ecbe5ed12ab4d8e11ba873c2f11161202b945 I0505 09:35:38.664718 1 gather.go:177] gatherer "clusterconfig" function "overlapping_namespace_uids" took 120.640387ms to process 1 records I0505 09:35:38.673444 1 base_controller.go:82] Caches are synced for LoggingSyncer I0505 09:35:38.673474 1 base_controller.go:119] Starting #1 worker of LoggingSyncer controller ... I0505 09:35:38.675878 1 tasks_processing.go:74] worker 5 stopped. I0505 09:35:38.676669 1 recorder.go:75] Recording config/crd/volumesnapshots.snapshot.storage.k8s.io with fingerprint=9f38cab1bc20ebe44ff20c8d1a38a8931b26d4136ee22c8e5e78cfaa403619df I0505 09:35:38.677043 1 recorder.go:75] Recording config/crd/volumesnapshotcontents.snapshot.storage.k8s.io with fingerprint=1eef91b918fcd3969e52ae357472c5fbdd936e81519f890df810b889e388d434 I0505 09:35:38.677060 1 gather.go:177] gatherer "clusterconfig" function "crds" took 133.331726ms to process 2 records I0505 09:35:38.677285 1 gather.go:177] gatherer "clusterconfig" function "aggregated_monitoring_cr_names" took 134.74141ms to process 0 records I0505 09:35:38.677371 1 tasks_processing.go:74] worker 21 stopped. I0505 09:35:38.677863 1 tasks_processing.go:74] worker 27 stopped. I0505 09:35:38.678131 1 recorder.go:75] Recording config/featuregate with fingerprint=7969857ac8f6b9c9303fca47b53b52353bc93a21be03c9548288826e53e5275a I0505 09:35:38.678173 1 gather.go:177] gatherer "clusterconfig" function "feature_gates" took 134.915574ms to process 1 records I0505 09:35:38.682114 1 requests.go:205] Asking for SCA certificate with "{"arch": ["x86_64"]}" payload I0505 09:35:38.696241 1 tasks_processing.go:74] worker 28 stopped. E0505 09:35:38.696279 1 gather.go:140] gatherer "clusterconfig" function "ingress_certificates" failed with the error: failed to get secret 'router-certs-default' in namespace 'openshift-ingress': secrets "router-certs-default" not found E0505 09:35:38.696420 1 gather.go:140] gatherer "clusterconfig" function "ingress_certificates" failed with the error: failed to get secret '2q3mf58ftm55ihbc68po7fqbj5oplorq-primary-cert-bundle-secret' in namespace 'openshift-ingress-operator': secrets "2q3mf58ftm55ihbc68po7fqbj5oplorq-primary-cert-bundle-secret" not found I0505 09:35:38.696578 1 recorder.go:75] Recording aggregated/ingress_controllers_certs with fingerprint=6bd72503e048da04802db32b0e3b66a57b9f86527f810097bf739e13cae4d34c I0505 09:35:38.696646 1 gather.go:177] gatherer "clusterconfig" function "ingress_certificates" took 152.121918ms to process 1 records I0505 09:35:38.702638 1 tasks_processing.go:74] worker 22 stopped. I0505 09:35:38.703945 1 recorder.go:75] Recording config/pod/openshift-console-operator/console-operator-66dbbb6468-s6shz with fingerprint=e1eba1571e48d6c5e8535348cc471a3241885df2361656ac417561328c79c940 I0505 09:35:38.704024 1 recorder.go:75] Recording config/running_containers with fingerprint=20f021a52992dee4cc14b4976edc2af96f66b1a7ae76995084a30be672de7d4a I0505 09:35:38.704035 1 gather.go:177] gatherer "clusterconfig" function "container_images" took 163.523446ms to process 2 records I0505 09:35:38.704628 1 tasks_processing.go:74] worker 23 stopped. E0505 09:35:38.704644 1 gather.go:140] gatherer "clusterconfig" function "config_maps" failed with the error: configmaps "cluster-monitoring-config" not found E0505 09:35:38.704650 1 gather.go:140] gatherer "clusterconfig" function "config_maps" failed with the error: configmaps "gateway-mode-config" not found E0505 09:35:38.704654 1 gather.go:140] gatherer "clusterconfig" function "config_maps" failed with the error: configmaps "insights-config" not found I0505 09:35:38.704663 1 recorder.go:75] Recording config/configmaps/openshift-config/installer-images/images.json with fingerprint=ad286723f58bdcfc37aeba3ec5b4110c08e1af59cd34d14b4bfaab02d18e4856 I0505 09:35:38.704715 1 recorder.go:75] Recording config/configmaps/openshift-config/kube-root-ca.crt/ca.crt with fingerprint=d476c7d3f5b104863f08f481b1264dcc68cc272ecefb0ecb709b18a6afab034d I0505 09:35:38.704723 1 recorder.go:75] Recording config/configmaps/openshift-config/openshift-install/invoker with fingerprint=76b482f683cd3ef9da02debac5b26080a5aeb06ff768ee5c21117514dff29d8a I0505 09:35:38.704727 1 recorder.go:75] Recording config/configmaps/openshift-config/openshift-install/version with fingerprint=2ab5ab7b1b10d7fcf1197bb24dea7c90f400e4effc18c7356873209d54fdf84b I0505 09:35:38.704738 1 recorder.go:75] Recording config/configmaps/openshift-config/openshift-service-ca.crt/service-ca.crt with fingerprint=34cdf5d36d45a9e3ee7c2be31546e3d937ae4ebf144ac07b0210c06a1610a94d I0505 09:35:38.704788 1 recorder.go:75] Recording config/configmaps/openshift-config/rosa-brand-logo/rosa-brand-logo.svg with fingerprint=6ed8ca4dd7a8eee7249182bc006e9649ce84d76c551ddfaaa33e55d8c4cc1ed0 I0505 09:35:38.704795 1 recorder.go:75] Recording config/configmaps/kube-system/cluster-config-v1/install-config with fingerprint=ab3811c6b83fd7b8e920094cfa3080d1b4ee3c35ec4c8379437b21d27bd6608d I0505 09:35:38.704801 1 gather.go:177] gatherer "clusterconfig" function "config_maps" took 160.654424ms to process 7 records I0505 09:35:38.709480 1 tasks_processing.go:74] worker 30 stopped. I0505 09:35:38.709498 1 gather.go:177] gatherer "clusterconfig" function "number_of_pods_and_netnamespaces_with_sdn_annotations" took 170.28425ms to process 0 records I0505 09:35:38.729480 1 gather_cluster_operator_pods_and_events.go:121] Found 6 pods with 9 containers I0505 09:35:38.729493 1 gather_cluster_operator_pods_and_events.go:235] Maximum buffer size: 2796202 bytes I0505 09:35:38.730439 1 gather_cluster_operator_pods_and_events.go:365] Fetching logs for dns container dns-default-d86dk pod in namespace openshift-dns (previous: false). I0505 09:35:38.767334 1 gather_cluster_operators.go:184] Unable to get dnsrecords.ingress.operator.openshift.io resource due to: dnsrecords.ingress.operator.openshift.io "default" not found I0505 09:35:38.771851 1 gather_cluster_operators.go:184] Unable to get dnsrecords.ingress.operator.openshift.io resource due to: dnsrecords.ingress.operator.openshift.io "default" not found I0505 09:35:38.785934 1 cluster_transfer.go:102] no available accepted cluster transfer I0505 09:35:38.785948 1 controllerstatus.go:80] name=clusterTransferController healthy=true reason=NoClusterTransfer message=no available cluster transfer I0505 09:35:38.808606 1 gather_cluster_operator_pods_and_events.go:365] Fetching logs for kube-rbac-proxy container dns-default-d86dk pod in namespace openshift-dns (previous: false). I0505 09:35:38.822235 1 gather_cluster_operator_pods_and_events.go:365] Fetching logs for dns container dns-default-l26wm pod in namespace openshift-dns (previous: false). I0505 09:35:38.874333 1 gather_cluster_operator_pods_and_events.go:408] Failed to fetch log for dns-default-l26wm pod in namespace openshift-dns for failing operator dns (previous: false): "container \"dns\" in pod \"dns-default-l26wm\" is waiting to start: ContainerCreating" I0505 09:35:38.874350 1 gather_cluster_operator_pods_and_events.go:280] Error: "container \"dns\" in pod \"dns-default-l26wm\" is waiting to start: ContainerCreating" I0505 09:35:38.874359 1 gather_cluster_operator_pods_and_events.go:365] Fetching logs for kube-rbac-proxy container dns-default-l26wm pod in namespace openshift-dns (previous: false). I0505 09:35:38.887047 1 gather_cluster_operator_pods_and_events.go:408] Failed to fetch log for dns-default-l26wm pod in namespace openshift-dns for failing operator kube-rbac-proxy (previous: false): "container \"kube-rbac-proxy\" in pod \"dns-default-l26wm\" is waiting to start: ContainerCreating" I0505 09:35:38.887065 1 gather_cluster_operator_pods_and_events.go:280] Error: "container \"kube-rbac-proxy\" in pod \"dns-default-l26wm\" is waiting to start: ContainerCreating" I0505 09:35:38.887092 1 gather_cluster_operator_pods_and_events.go:365] Fetching logs for dns container dns-default-xrmnk pod in namespace openshift-dns (previous: false). I0505 09:35:38.897348 1 gather_cluster_operator_pods_and_events.go:408] Failed to fetch log for dns-default-xrmnk pod in namespace openshift-dns for failing operator dns (previous: false): "container \"dns\" in pod \"dns-default-xrmnk\" is waiting to start: ContainerCreating" I0505 09:35:38.897361 1 gather_cluster_operator_pods_and_events.go:280] Error: "container \"dns\" in pod \"dns-default-xrmnk\" is waiting to start: ContainerCreating" I0505 09:35:38.897368 1 gather_cluster_operator_pods_and_events.go:365] Fetching logs for kube-rbac-proxy container dns-default-xrmnk pod in namespace openshift-dns (previous: false). I0505 09:35:38.907146 1 gather_cluster_operator_pods_and_events.go:408] Failed to fetch log for dns-default-xrmnk pod in namespace openshift-dns for failing operator kube-rbac-proxy (previous: false): "container \"kube-rbac-proxy\" in pod \"dns-default-xrmnk\" is waiting to start: ContainerCreating" I0505 09:35:38.907160 1 gather_cluster_operator_pods_and_events.go:280] Error: "container \"kube-rbac-proxy\" in pod \"dns-default-xrmnk\" is waiting to start: ContainerCreating" I0505 09:35:38.907168 1 gather_cluster_operator_pods_and_events.go:365] Fetching logs for dns-node-resolver container node-resolver-5rx64 pod in namespace openshift-dns (previous: false). I0505 09:35:38.920797 1 gather_cluster_operator_pods_and_events.go:365] Fetching logs for dns-node-resolver container node-resolver-mwz9m pod in namespace openshift-dns (previous: false). I0505 09:35:38.923399 1 sca.go:266] etc-pki-entitlement secret successfully created I0505 09:35:38.923413 1 controllerstatus.go:80] name=scaController healthy=true reason=Updated message=SCA certs successfully updated I0505 09:35:39.092860 1 gather_cluster_operator_pods_and_events.go:280] Error: "log buffer is empty" I0505 09:35:39.092876 1 gather_cluster_operator_pods_and_events.go:365] Fetching logs for dns-node-resolver container node-resolver-nh9xq pod in namespace openshift-dns (previous: false). I0505 09:35:39.295927 1 tasks_processing.go:74] worker 12 stopped. I0505 09:35:39.296016 1 recorder.go:75] Recording events/openshift-dns-operator with fingerprint=1a5f8f3b6a180278429fa1c24ca9ced518ddb1d865213953a7791df427cba5b6 I0505 09:35:39.296080 1 recorder.go:75] Recording events/openshift-dns with fingerprint=328ab82677824810aefc5b618febba11f30505659e315a3e424053d2ff09c858 I0505 09:35:39.296090 1 recorder.go:75] Recording config/pod/openshift-dns/logs/dns-default-d86dk/dns_current.log with fingerprint=ebeaef036ab596234949eb1fa99da6e2dca9c38919023c7b1aa1621df6da4830 I0505 09:35:39.296098 1 recorder.go:75] Recording config/pod/openshift-dns/logs/dns-default-d86dk/kube-rbac-proxy_current.log with fingerprint=6a9b27635d03332a88cb83ffbe6f7f080f45b9f625f0d222efead21533eb05ab I0505 09:35:39.296230 1 recorder.go:75] Recording config/pod/openshift-dns/dns-default-l26wm with fingerprint=0541be259be72a555b3cc6bf7efbf03d63b44900bd95af4a05a75d2d31ea057d I0505 09:35:39.296315 1 recorder.go:75] Recording config/pod/openshift-dns/dns-default-xrmnk with fingerprint=58f0595db883517d2d520d696ad5ed9979e8c54ef6b12c6f73afd576f5acfce8 I0505 09:35:39.296322 1 recorder.go:75] Recording config/pod/openshift-dns/logs/node-resolver-5rx64/dns-node-resolver_current.log with fingerprint=84ba6551769a5fc24004b265f9c8223f3341168c4ab6ea623e3679d2763ec449 I0505 09:35:39.296326 1 recorder.go:75] Recording config/pod/openshift-dns/logs/node-resolver-nh9xq/dns-node-resolver_current.log with fingerprint=e193a9318636fbff2dbc35d745a7ef555ecb723ce2e6ac77cd6b7a2e85210c8d I0505 09:35:39.296333 1 gather.go:177] gatherer "clusterconfig" function "operators_pods_and_events" took 753.766675ms to process 8 records I0505 09:35:39.578296 1 tasks_processing.go:74] worker 39 stopped. I0505 09:35:39.578322 1 recorder.go:75] Recording config/silenced_alerts.json with fingerprint=37517e5f3dc66819f61f5a7bb8ace1921282415f10551d2defa5c3eb0985b570 I0505 09:35:39.578340 1 gather.go:177] gatherer "clusterconfig" function "silenced_alerts" took 1.03511102s to process 1 records I0505 09:35:39.579376 1 tasks_processing.go:74] worker 60 stopped. I0505 09:35:39.579423 1 recorder.go:75] Recording config/alerts with fingerprint=2435f8d0c12c2b84da7b61cc98605839f3f064904e85d1faf020b8665b07e2b1 I0505 09:35:39.579440 1 gather.go:177] gatherer "clusterconfig" function "active_alerts" took 1.040175932s to process 1 records W0505 09:35:39.603961 1 gather_dvo_metrics.go:210] Failed to read the DVO metrics. Trying again. W0505 09:35:40.603674 1 gather_dvo_metrics.go:210] Failed to read the DVO metrics. Trying again. I0505 09:35:41.090118 1 tasks_processing.go:74] worker 46 stopped. I0505 09:35:41.090249 1 recorder.go:75] Recording config/clusteroperator/console with fingerprint=e6c338594984e47fd375df66aaffe0b59700941b0ef151c21442b308ea98d69a I0505 09:35:41.090299 1 recorder.go:75] Recording config/clusteroperator/operator.openshift.io/console/cluster with fingerprint=be84fc75c87766f0f4d4e9fb1f9db618d008a21cc463e8239af81a1c296612d1 I0505 09:35:41.090334 1 recorder.go:75] Recording config/clusteroperator/csi-snapshot-controller with fingerprint=beb94659b24cffe9abb75eb33768467a8c0ae46f6a1f07aafb70b92a19667d55 I0505 09:35:41.090344 1 recorder.go:75] Recording config/clusteroperator/operator.openshift.io/csisnapshotcontroller/cluster with fingerprint=5adc514f4b63e2f1ecc68bf6f9c0af70c5eea04522a49524e102721b1c41f80e I0505 09:35:41.090365 1 recorder.go:75] Recording config/clusteroperator/dns with fingerprint=0d8e7f954ac9de6fb5ca5385cab74666330cec4aef2ac35c35a1dffb74ae7ec0 I0505 09:35:41.090381 1 recorder.go:75] Recording config/clusteroperator/operator.openshift.io/dns/default with fingerprint=9e7b4ce029030d3d8c3b49af92c556acdcc415000b40d3f969dbdc42c432b47f I0505 09:35:41.090401 1 recorder.go:75] Recording config/clusteroperator/image-registry with fingerprint=15d6f579a7ea9351e802996885c27568f90b4758624cdf35239b127b23ffee5d I0505 09:35:41.090426 1 recorder.go:75] Recording config/clusteroperator/ingress with fingerprint=bd0ef1461aff2a0813b0bb32fa3fb61267da221353795fb351fdece127325a0f I0505 09:35:41.090469 1 recorder.go:75] Recording config/clusteroperator/operator.openshift.io/ingresscontroller/openshift-ingress-operator/default with fingerprint=65a73b585fd548ea4619679f6a1fec66ebd4124144a4c991fe488820eb65d78d I0505 09:35:41.090499 1 recorder.go:75] Recording config/clusteroperator/insights with fingerprint=21e18e0aeebc8a66fc49a86f517748747dcb9d530791d50a5943f7824e6ec6ef I0505 09:35:41.090820 1 recorder.go:75] Recording config/clusteroperator/operator.openshift.io/insightsoperator/cluster with fingerprint=6dc81c6aa2286e78535c98226bc718f8b93513346f5343d57633b6775dfeca7f I0505 09:35:41.090842 1 recorder.go:75] Recording config/clusteroperator/kube-apiserver with fingerprint=3b2dfc28b2a984995b66202c5fea60b539669b5a2f0d61e3ff0fa8042aa85b09 I0505 09:35:41.090851 1 recorder.go:75] Recording config/clusteroperator/operator.openshift.io/kubeapiserver/cluster with fingerprint=51503bf0b784fcf65ea46bcaf1f72ac1a5c4d5dc211934f18f27871efed05762 I0505 09:35:41.090867 1 recorder.go:75] Recording config/clusteroperator/kube-controller-manager with fingerprint=81db969f7e03517b1c054b8962cd038bdd940a0de5eb5873cf7517c180b4c105 I0505 09:35:41.090877 1 recorder.go:75] Recording config/clusteroperator/operator.openshift.io/kubecontrollermanager/cluster with fingerprint=ce90c0d4f367d7da085074268031798382ae7c54fdcb0a21f15a4818fe308c11 I0505 09:35:41.090894 1 recorder.go:75] Recording config/clusteroperator/kube-scheduler with fingerprint=a6aaf0b728648f936d617cc564c2e76549cd3d8faf79c5c492c99e4c786ef5be I0505 09:35:41.090901 1 recorder.go:75] Recording config/clusteroperator/operator.openshift.io/kubescheduler/cluster with fingerprint=f2940fb9fd20c19951dfc295eb363b7fba0c505f5ae61f01967a063099e6b60a I0505 09:35:41.090920 1 recorder.go:75] Recording config/clusteroperator/kube-storage-version-migrator with fingerprint=75f56d8dac51e53dcdf818a1b981603f689974cf04b14c8fe9d1cea027e09d21 I0505 09:35:41.090929 1 recorder.go:75] Recording config/clusteroperator/operator.openshift.io/kubestorageversionmigrator/cluster with fingerprint=9351181aa7e6ada41ef581ab31e13516c6b934cc95710154bafb2eb222cb58db I0505 09:35:41.090950 1 recorder.go:75] Recording config/clusteroperator/monitoring with fingerprint=1bc317e4907c21215a0c40d3c2fe889d55484a1680e8320f0a67d01a600a9fdd I0505 09:35:41.091052 1 recorder.go:75] Recording config/clusteroperator/network with fingerprint=4cfab5ecb7d85917325f218c18c5242bfd401f5e74762eb8fa2772dcd9e0a675 I0505 09:35:41.091063 1 recorder.go:75] Recording config/clusteroperator/network.operator.openshift.io/operatorpki/openshift-ovn-kubernetes/ovn with fingerprint=626a89d20e0deaed5b6dfb533acfe65f4bb1618bd200a703b62e60c5d16d94ab I0505 09:35:41.091071 1 recorder.go:75] Recording config/clusteroperator/network.operator.openshift.io/operatorpki/openshift-ovn-kubernetes/signer with fingerprint=90410b16914712b85b3c4578716ad8c0ae072e688f4cd1e022bf76f20da3506d I0505 09:35:41.091094 1 recorder.go:75] Recording config/clusteroperator/operator.openshift.io/network/cluster with fingerprint=a93d15eaecb455a0e40ecb2826eeecc1533899204ddd3c3921d15ab70af7ae75 I0505 09:35:41.091118 1 recorder.go:75] Recording config/clusteroperator/node-tuning with fingerprint=cab3d4f3c6a5700fc1f4c169cf4676c99544150928e032552a48350a4c1e0162 I0505 09:35:41.091141 1 recorder.go:75] Recording config/clusteroperator/openshift-apiserver with fingerprint=f87c41f5e3c04c12832efd8a78cf18f4a099d743fe0e71d7037e2fb2d735c6b9 I0505 09:35:41.091150 1 recorder.go:75] Recording config/clusteroperator/operator.openshift.io/openshiftapiserver/cluster with fingerprint=e712e6cf27339b441e4ed1f4cde91dbde7e952698ba93407e4457db63a4a4c76 I0505 09:35:41.091167 1 recorder.go:75] Recording config/clusteroperator/openshift-controller-manager with fingerprint=00e84d4297abcf3b7039c2ff156f7294b9cc93b9d823de0a0b15f104f7f3f14d I0505 09:35:41.091177 1 recorder.go:75] Recording config/clusteroperator/operator.openshift.io/openshiftcontrollermanager/cluster with fingerprint=d71a0f4672f9b45d9fc8293bf1687afc650fd28d32e2e30de27523fe7b4eadf7 I0505 09:35:41.091194 1 recorder.go:75] Recording config/clusteroperator/openshift-samples with fingerprint=894b2b8600806ac0613f2b601bce92f4e2252108840ad4066425ab90c7e1a4b5 I0505 09:35:41.091203 1 recorder.go:75] Recording config/clusteroperator/samples.operator.openshift.io/config/cluster with fingerprint=439a4284281b8dcef0621bb14ba23e2175a28a50613f86b33a171c49689474fd I0505 09:35:41.091215 1 recorder.go:75] Recording config/clusteroperator/operator-lifecycle-manager with fingerprint=4744a7bc010b1206c3fcd577bba4f4496a1c42d90a2a7b63e022fd340ff82468 I0505 09:35:41.091229 1 recorder.go:75] Recording config/clusteroperator/operator-lifecycle-manager-catalog with fingerprint=34120682ee46aeb9a9768e26c6def3fb9a6231e6db56fece72ea0e79f9330942 I0505 09:35:41.091242 1 recorder.go:75] Recording config/clusteroperator/operator-lifecycle-manager-packageserver with fingerprint=e1d46dcaf7c459ca75627e5b72aa72a03c4190143b2b38e4ced7e10db77be096 I0505 09:35:41.091263 1 recorder.go:75] Recording config/clusteroperator/service-ca with fingerprint=20e64e581e7920f048b558f4afdd86159274ad8267cf93d64e27f0a4bfc641fb I0505 09:35:41.091273 1 recorder.go:75] Recording config/clusteroperator/operator.openshift.io/serviceca/cluster with fingerprint=812f7edc2cdb30e61e7f2b29454357a40b1a507a4b0c2b7729193b67f0e3b4aa I0505 09:35:41.091300 1 recorder.go:75] Recording config/clusteroperator/storage with fingerprint=3491296b384aac228ea021df58d9eb9e9b33c9f6e06ac8cd646186e2113a6ada I0505 09:35:41.091317 1 recorder.go:75] Recording config/clusteroperator/operator.openshift.io/clustercsidriver/ebs.csi.aws.com with fingerprint=510064d6f6bcced87ab5bd2ddaff3d0edd7f93f4a4f7af2641f29fc53ffab21e I0505 09:35:41.091323 1 recorder.go:75] Recording config/clusteroperator/operator.openshift.io/storage/cluster with fingerprint=8e480f8c1ce1b39baac42d8ec780c57c2592929ae0c801b61ffad49ba13f33ad I0505 09:35:41.091330 1 gather.go:177] gatherer "clusterconfig" function "operators" took 2.546695515s to process 39 records W0505 09:35:41.603030 1 gather_dvo_metrics.go:210] Failed to read the DVO metrics. Trying again. W0505 09:35:42.602780 1 gather_dvo_metrics.go:210] Failed to read the DVO metrics. Trying again. W0505 09:35:43.603691 1 gather_dvo_metrics.go:210] Failed to read the DVO metrics. Trying again. W0505 09:35:43.603715 1 gather_dvo_metrics.go:117] Unable to read metrics from endpoint "http://deployment-validation-operator-metrics.openshift-deployment-validation-operator.svc:8383": DVO metrics service was not available within the 5s timeout: context deadline exceeded I0505 09:35:43.603731 1 tasks_processing.go:74] worker 55 stopped. E0505 09:35:43.603742 1 gather.go:140] gatherer "clusterconfig" function "dvo_metrics" failed with the error: DVO metrics service was not available within the 5s timeout: context deadline exceeded I0505 09:35:43.603753 1 recorder.go:75] Recording config/dvo_metrics with fingerprint=e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855 I0505 09:35:43.603794 1 gather.go:177] gatherer "clusterconfig" function "dvo_metrics" took 5.0602579s to process 1 records I0505 09:35:51.396387 1 tasks_processing.go:74] worker 11 stopped. I0505 09:35:51.396433 1 recorder.go:75] Recording config/installplans with fingerprint=95dfd5f33a9a46199239158a0ec0183b629818eca24ae41f4d706d7a3d604aec I0505 09:35:51.396447 1 gather.go:177] gatherer "clusterconfig" function "install_plans" took 12.85323797s to process 1 records I0505 09:35:52.149410 1 tasks_processing.go:74] worker 51 stopped. I0505 09:35:52.149704 1 recorder.go:75] Recording config/serviceaccounts with fingerprint=796bd9f20e825fd32af435d21e60c10c3ec4910fa1211c5a6338f5b81c3f8071 I0505 09:35:52.149722 1 gather.go:177] gatherer "clusterconfig" function "service_accounts" took 13.607697457s to process 1 records E0505 09:35:52.149783 1 periodic.go:254] "Unhandled Error" err="clusterconfig failed after 13.61s with: function \"metrics\" failed with an error, function \"tsdb_status\" failed with an error, function \"pod_network_connectivity_checks\" failed with an error, function \"machine_healthchecks\" failed with an error, function \"machines\" failed with an error, function \"support_secret\" failed with an error, function \"machine_configs\" failed with an error, function \"ingress_certificates\" failed with an error, function \"config_maps\" failed with an error, function \"dvo_metrics\" failed with an error" I0505 09:35:52.150893 1 controllerstatus.go:89] name=periodic-clusterconfig healthy=false reason=PeriodicGatherFailed message=Source clusterconfig could not be retrieved: function "metrics" failed with an error, function "tsdb_status" failed with an error, function "pod_network_connectivity_checks" failed with an error, function "machine_healthchecks" failed with an error, function "machines" failed with an error, function "support_secret" failed with an error, function "machine_configs" failed with an error, function "ingress_certificates" failed with an error, function "config_maps" failed with an error, function "dvo_metrics" failed with an error I0505 09:35:52.150910 1 periodic.go:216] Running workloads gatherer I0505 09:35:52.150926 1 tasks_processing.go:45] number of workers: 2 I0505 09:35:52.150935 1 tasks_processing.go:69] worker 1 listening for tasks. I0505 09:35:52.150939 1 tasks_processing.go:71] worker 1 working on workload_info task. I0505 09:35:52.150949 1 tasks_processing.go:69] worker 0 listening for tasks. I0505 09:35:52.150963 1 tasks_processing.go:71] worker 0 working on helmchart_info task. I0505 09:35:52.157902 1 gather_workloads_runtime_infos.go:63] Gathering workload runtime info for node ip-10-0-0-57.ec2.internal... I0505 09:35:52.157912 1 gather_workloads_runtime_infos.go:63] Gathering workload runtime info for node ip-10-0-1-134.ec2.internal... I0505 09:35:52.157928 1 gather_workloads_runtime_infos.go:63] Gathering workload runtime info for node ip-10-0-2-147.ec2.internal... I0505 09:35:52.181168 1 tasks_processing.go:74] worker 0 stopped. I0505 09:35:52.181185 1 gather.go:177] gatherer "workloads" function "helmchart_info" took 30.192086ms to process 0 records I0505 09:35:56.069646 1 gather_workloads_runtime_infos.go:84] Gathered workload runtime infos in 4s I0505 09:35:56.096423 1 gather_workloads_info.go:278] Loaded pods in 0s, will wait 24s for image data I0505 09:35:56.106264 1 gather_workloads_info.go:387] No image sha256:a043239802b3eb8b323d285193d2527fad0ecec98ca91d188a3472a2fac8ae04 (11ms) I0505 09:35:56.117417 1 gather_workloads_info.go:387] No image sha256:730d1b6988025bef0daa3a9a5d8467ec4a26b0382cc52f91c3375b4590d3518a (11ms) I0505 09:35:56.128270 1 gather_workloads_info.go:387] No image sha256:0acfe677ed58031f8381cb8a4f2d8917ec4cfa95485e2a46529dffae54c75ace (11ms) I0505 09:35:56.138431 1 gather_workloads_info.go:387] No image sha256:084aa9b0f8a6d478549dc384d4e66da13ee9b25cc98531da861cc19dee2a9e8f (10ms) I0505 09:35:56.149312 1 gather_workloads_info.go:387] No image sha256:0a4dfb8d4c1b3849319d45b4c54dff26a7238a2c08fcaa121f93073e95ab12e8 (11ms) I0505 09:35:56.159335 1 gather_workloads_info.go:387] No image sha256:6c6d412ccf0c3d6a14645a79df6c4658c0f8725d7954cb83c3d2f0116499345a (10ms) I0505 09:35:56.169444 1 gather_workloads_info.go:387] No image sha256:ab60623bb32f7e75fca71ef65137731cae347a21c7a4091dfd583fa00732721c (10ms) I0505 09:35:56.180561 1 gather_workloads_info.go:387] No image sha256:e1ba458cf6f0b3606c90880da72db8ab99cd11040bae84baebe3ff2e0d1ea075 (11ms) I0505 09:35:56.191201 1 gather_workloads_info.go:387] No image sha256:637d41f067a5239096fc22b135181cda5113da833f1370e7a73965e83792e93a (11ms) I0505 09:35:56.201081 1 gather_workloads_info.go:387] No image sha256:1eb6a6ee3c66ac15be6bc21bb6015c595c11c15b9dfd21bd9f5ce7f7b990ebaa (10ms) I0505 09:35:56.210953 1 gather_workloads_info.go:387] No image sha256:9ef6dc311b880a6442b7a3a19006ddd955c7da36850bf000f5c9b68fd0862dd4 (10ms) I0505 09:35:56.307968 1 gather_workloads_info.go:387] No image sha256:8f881f6f0498eaba860d711dc8e9dc066db8728f22abedb82e1dc9a8e1c30046 (97ms) I0505 09:35:56.406642 1 gather_workloads_info.go:387] No image sha256:5b6212b8f539f08e78417d8a4b7485ca0b4e7927cacd7b752742a28841bc8ccd (99ms) I0505 09:35:56.507384 1 gather_workloads_info.go:398] Found image sha256:673ebc8cc22c56c8f410e011b2fa950d28cf7b6420e17fdb6580d6cb10523384 (101ms) I0505 09:35:56.606741 1 gather_workloads_info.go:387] No image sha256:875c77e5d144f03fb91d8cee0259f6966683ca88d1bf818dbf4652c16b70312c (99ms) I0505 09:35:56.706571 1 gather_workloads_info.go:387] No image sha256:48f5559439bea030a6423652da448c25fff0c62ba293eb521843c1a8eb89ddb0 (100ms) I0505 09:35:56.806154 1 gather_workloads_info.go:387] No image sha256:89277d8d4560d71db88c2dcc67c992a24544ca21810920b609c1d49d53b4a287 (100ms) I0505 09:35:56.906311 1 gather_workloads_info.go:387] No image sha256:934d8e8c50f3c609b8eea80d1051111fe3d066fe8c65c79572072ae55fcb0a86 (100ms) I0505 09:35:57.007219 1 gather_workloads_info.go:387] No image sha256:bdeb1c455cffccdb6c90db238e9d371de06340b1d6105d47f66d87293f878ac1 (101ms) I0505 09:35:57.105668 1 gather_workloads_info.go:387] No image sha256:d6fbe0075cbb12bfd287c973704fadf97154c7f73e370733d976a40835e9155a (98ms) I0505 09:35:57.206403 1 gather_workloads_info.go:387] No image sha256:30e597ec5d6bb96ff70a4f8688c748b659cd4fd5d73d222e8701821d236795c5 (101ms) I0505 09:35:57.306055 1 gather_workloads_info.go:387] No image sha256:08c5a78c8a5af04c549e2273aaf4bb452a75bf038d68aa9d01bb2aff66c30e90 (100ms) I0505 09:35:57.407115 1 gather_workloads_info.go:387] No image sha256:52db05084f2a2e7d7bf4040e4865df88ec4074efbbe59a78d861be2c5ab2aa40 (101ms) I0505 09:35:57.507028 1 gather_workloads_info.go:387] No image sha256:15928a5b803fa00781b408dbc4c3bfa7c5287b18584ba31f817def8a4430c725 (100ms) I0505 09:35:57.606798 1 gather_workloads_info.go:387] No image sha256:5aaea0419169e55832cc27acfe0fe3b9513a343d6bdf71d3da1575ed322245d0 (100ms) I0505 09:35:57.708032 1 gather_workloads_info.go:387] No image sha256:15677f0b70e6aa2dfaf088e45fc1a425c22bf6fda326b8116f87e88b6694dfab (101ms) I0505 09:35:57.808029 1 gather_workloads_info.go:387] No image sha256:ae7d3453fd734ecc865e5f9bb16f29244ebffe6291b77e1d4e496f71eb053174 (100ms) I0505 09:35:57.907637 1 gather_workloads_info.go:387] No image sha256:5940cc34b322325618929b1e0eb3864dae73d54c8f5b474e218dfb28123a61c7 (100ms) I0505 09:35:58.006977 1 gather_workloads_info.go:387] No image sha256:17aefdc2f4752c23dc8f0947bb01fc34635822eeb18f2d8a5eddf8211b84d450 (99ms) I0505 09:35:58.107256 1 gather_workloads_info.go:387] No image sha256:3380fecebaeee139c1607fbcd9346cebc92806679568e2244a8905ef0f52aad2 (100ms) I0505 09:35:58.207854 1 gather_workloads_info.go:387] No image sha256:53ff8efdd8f563421ebc1859ab021a3bbc1306f82d4ff5d9c21bc60ec5f57898 (101ms) I0505 09:35:58.307651 1 gather_workloads_info.go:387] No image sha256:d64bea34bf3e1bb0b3a701c3ff14e66665afc1b050f28124ad7e6888eaec3a81 (100ms) I0505 09:35:58.409721 1 gather_workloads_info.go:387] No image sha256:5c6d21c3f97366bc7ab57031cc027b67405a684bd804ce364ed5998b0685eaca (102ms) I0505 09:35:58.507023 1 gather_workloads_info.go:398] Found image sha256:23204877f166455ce8f9de9b91a467eeda576f172d5f96124a68aa986c4775d6 (97ms) I0505 09:35:58.607331 1 gather_workloads_info.go:387] No image sha256:3bba1358d4a0ae878ff491c0c2cbfffe60649e110b40342b878fe8fa332f8858 (100ms) I0505 09:35:58.706571 1 gather_workloads_info.go:387] No image sha256:000105ef5150e7079b90a613fb9e6193e2a6ef9b1908d2dce44f2395d4fd070f (99ms) I0505 09:35:58.807619 1 gather_workloads_info.go:387] No image sha256:5d6a2b8d2508c7da71192661e7c0dcf1335f48b6336a20189abffbe79eeff962 (101ms) I0505 09:35:58.907315 1 gather_workloads_info.go:387] No image sha256:9d6d63dce784df5863e6a0ba4c11317a26c9d756d4e62d47759c97190a71cce5 (100ms) I0505 09:35:59.006975 1 gather_workloads_info.go:387] No image sha256:572ebf7c7fab1a518582f3ca46c866730423cce6e8a19f614dc15fd61bf0c65a (100ms) I0505 09:35:59.111811 1 gather_workloads_info.go:387] No image sha256:15e1b5f763553e86d0230a751b0783e3b990438fd07cf96eed164bccb72ed564 (105ms) I0505 09:35:59.206896 1 gather_workloads_info.go:387] No image sha256:55b1db6038c5beaed54c626e3343b7a8589cc0be8dc41d1a66b4deab766ff520 (95ms) I0505 09:35:59.307233 1 gather_workloads_info.go:387] No image sha256:5ff204630794311b0b37fa7b197b933ad85d76a481bf7cdb3bcbada08f0cdcbf (100ms) I0505 09:35:59.406307 1 gather_workloads_info.go:387] No image sha256:1f118a4ec0b92cb88c8544ea31d2b9adfc7006eeddb7a8d260150ecbce4adb8c (99ms) I0505 09:35:59.506732 1 gather_workloads_info.go:387] No image sha256:5808401268394502d335281ea1a294b07210461b986b58f91d7d1f29c0029c6d (100ms) I0505 09:35:59.607811 1 gather_workloads_info.go:387] No image sha256:6814f5fced0ef219d06374011c68a11a3da788a764a00a69ade435466d9ee240 (101ms) I0505 09:35:59.707740 1 gather_workloads_info.go:387] No image sha256:48e883a3932aea9457f55cd4628d21397d429d4828f42f95d1c903d9d5395bde (100ms) I0505 09:35:59.808025 1 gather_workloads_info.go:387] No image sha256:0b2e6536a0ca3afc5e64a218087a6287c74b77a359075a385f1949ca5bd51b22 (100ms) I0505 09:35:59.907300 1 gather_workloads_info.go:387] No image sha256:25c148fd380b1a9db3f6039d2e0eabc489a954921452391390ba9192b2325678 (99ms) I0505 09:36:00.006778 1 gather_workloads_info.go:387] No image sha256:5ac9c549d65fc1d8bc900773bebee43e9192bcec1bb5fa46afb4597230c16ac7 (99ms) I0505 09:36:00.006816 1 tasks_processing.go:74] worker 1 stopped. E0505 09:36:00.006825 1 gather.go:140] gatherer "workloads" function "workload_info" failed with the error: received unexpected status code 500 Internal Server Error from https://10.128.0.14:8443/gather_runtime_info I0505 09:36:00.007242 1 recorder.go:75] Recording config/workload_info with fingerprint=21323b90e323bd5dd77096433d7e2527937f83b560ea70dc186ac767b2c635d2 I0505 09:36:00.007259 1 gather.go:177] gatherer "workloads" function "workload_info" took 7.855869257s to process 1 records E0505 09:36:00.007283 1 periodic.go:254] "Unhandled Error" err="workloads failed after 7.856s with: function \"workload_info\" failed with an error" I0505 09:36:00.008387 1 controllerstatus.go:89] name=periodic-workloads healthy=false reason=PeriodicGatherFailed message=Source workloads could not be retrieved: function "workload_info" failed with an error I0505 09:36:00.008399 1 periodic.go:216] Running conditional gatherer I0505 09:36:00.014418 1 requests.go:294] Making HTTP GET request at: https://console.redhat.com/api/gathering/v2/4.21.9/gathering_rules I0505 09:36:00.185863 1 conditional_gatherer.go:338] updating alerts cache for conditional gatherer E0505 09:36:00.188409 1 conditional_gatherer.go:324] unable to update alerts cache: Get "https://prometheus-k8s.openshift-monitoring.svc:9091/api/v1/query?match%5B%5D=ALERTS%7Balertstate%3D%22firing%22%7D&query=ALERTS": dial tcp 172.30.119.64:9091: connect: connection refused I0505 09:36:00.188503 1 conditional_gatherer.go:384] updating version cache for conditional gatherer I0505 09:36:00.194953 1 conditional_gatherer.go:392] cluster version is '4.21.9' E0505 09:36:00.194975 1 conditional_gatherer.go:209] error checking conditions for a gathering rule: alerts cache is missing E0505 09:36:00.194983 1 conditional_gatherer.go:209] error checking conditions for a gathering rule: alerts cache is missing E0505 09:36:00.194989 1 conditional_gatherer.go:209] error checking conditions for a gathering rule: alerts cache is missing E0505 09:36:00.194993 1 conditional_gatherer.go:209] error checking conditions for a gathering rule: alerts cache is missing E0505 09:36:00.194997 1 conditional_gatherer.go:209] error checking conditions for a gathering rule: alerts cache is missing E0505 09:36:00.195001 1 conditional_gatherer.go:209] error checking conditions for a gathering rule: alerts cache is missing E0505 09:36:00.195005 1 conditional_gatherer.go:209] error checking conditions for a gathering rule: alerts cache is missing E0505 09:36:00.195010 1 conditional_gatherer.go:209] error checking conditions for a gathering rule: alerts cache is missing E0505 09:36:00.195014 1 conditional_gatherer.go:209] error checking conditions for a gathering rule: alerts cache is missing E0505 09:36:00.195019 1 conditional_gatherer.go:209] error checking conditions for a gathering rule: alerts cache is missing I0505 09:36:00.195038 1 tasks_processing.go:45] number of workers: 3 I0505 09:36:00.195055 1 tasks_processing.go:69] worker 2 listening for tasks. I0505 09:36:00.195063 1 tasks_processing.go:71] worker 2 working on remote_configuration task. I0505 09:36:00.195062 1 tasks_processing.go:69] worker 0 listening for tasks. I0505 09:36:00.195074 1 tasks_processing.go:69] worker 1 listening for tasks. I0505 09:36:00.195077 1 tasks_processing.go:71] worker 0 working on rapid_container_logs task. I0505 09:36:00.195082 1 tasks_processing.go:74] worker 1 stopped. I0505 09:36:00.195097 1 tasks_processing.go:71] worker 2 working on conditional_gatherer_rules task. I0505 09:36:00.195134 1 recorder.go:75] Recording insights-operator/remote-configuration with fingerprint=5f681cc1442447fd2c1534f876836dff54caf3c545b582a5a8aae894f9203c5d I0505 09:36:00.195151 1 gather.go:177] gatherer "conditional" function "remote_configuration" took 1.08µs to process 1 records I0505 09:36:00.195225 1 recorder.go:75] Recording insights-operator/conditional-gatherer-rules with fingerprint=9fafdcbf373f1f367f1d69551ea59114592cce1681189be0f3dde774e0530661 I0505 09:36:00.195237 1 tasks_processing.go:74] worker 2 stopped. I0505 09:36:00.195237 1 gather.go:177] gatherer "conditional" function "conditional_gatherer_rules" took 1.322µs to process 1 records I0505 09:36:00.195324 1 tasks_processing.go:74] worker 0 stopped. I0505 09:36:00.195336 1 gather.go:177] gatherer "conditional" function "rapid_container_logs" took 240.475µs to process 0 records I0505 09:36:00.195350 1 controllerstatus.go:80] name=periodic-conditional healthy=true reason=Succeeded message= I0505 09:36:00.195366 1 recorder.go:75] Recording insights-operator/remote-configuration.json with fingerprint=363fa5c8fa2573b18bc685faeca9ca9f021a8f80760895b7701f54e3dcd5c85f I0505 09:36:00.227035 1 recorder.go:75] Recording insights-operator/gathers with fingerprint=db16bab9b9c1c6d5cdef087aa8152b6241ef1865bce5c72b56c20ec242aaec58 I0505 09:36:00.227192 1 diskrecorder.go:70] Writing 119 records to /var/lib/insights-operator/insights-2026-05-05-093600.tar.gz I0505 09:36:00.234284 1 diskrecorder.go:51] Wrote 119 records to disk in 7ms I0505 09:36:00.234331 1 periodic.go:285] Gathering cluster info every 2h0m0s I0505 09:36:00.234347 1 periodic.go:286] Configuration is dataReporting: interval: 2h0m0s, uploadEndpoint: https://console.redhat.com/api/ingress/v1/upload, storagePath: /var/lib/insights-operator, downloadEndpoint: https://console.redhat.com/api/insights-results-aggregator/v2/cluster/%s/reports, conditionalGathererEndpoint: https://console.redhat.com/api/gathering/v2/%s/gathering_rules, obfuscation: [] sca: disabled: false, endpoint: https://api.openshift.com/api/accounts_mgmt/v1/entitlement_certificates, interval: 8h0m0s alerting: disabled: false clusterTransfer: endpoint: https://api.openshift.com/api/accounts_mgmt/v1/cluster_transfers/, interval: 12h0m0s proxy: httpProxy: , httpsProxy: , noProxy: I0505 09:37:08.539192 1 diskrecorder.go:223] Found files to send: insights-2026-05-05-093600.tar.gz I0505 09:37:08.539225 1 insightsuploader.go:150] Checking archives to upload periodically every 15m14.537110777s I0505 09:37:08.539234 1 insightsuploader.go:165] Uploading latest report since 0001-01-01T00:00:00Z I0505 09:37:08.554358 1 requests.go:46] Uploading application/vnd.redhat.openshift.periodic to https://console.redhat.com/api/ingress/v1/upload I0505 09:37:08.771886 1 requests.go:87] Successfully reported id=2026-05-05T09:37:08Z x-rh-insights-request-id=afca9921e46e40368d74903f8c26dfa4, wrote=53745 I0505 09:37:08.771950 1 insightsuploader.go:187] Uploaded report successfully in 232.706212ms I0505 09:37:08.771973 1 controller.go:129] Initializing last reported time to 2026-05-05T09:37:08Z I0505 09:37:08.771995 1 insightsreport.go:304] Archive uploaded, starting pulling report... I0505 09:37:08.772008 1 insightsreport.go:215] Starting retrieving report from Smart Proxy I0505 09:37:08.772018 1 insightsreport.go:221] Initial delay for pulling: 1m0s I0505 09:37:08.778061 1 controller.go:554] The operator is healthy I0505 09:37:38.580981 1 controller.go:554] The operator is healthy I0505 09:38:13.835043 1 insightsreport.go:137] Pulling report from smart-proxy I0505 09:38:13.835079 1 insightsreport.go:149] Retrieving report I0505 09:38:13.842326 1 requests.go:110] Retrieving report for cluster: 1592e6e4-e90d-4dc3-ac97-1dd9d01514de I0505 09:38:13.842337 1 requests.go:111] Endpoint: https://console.redhat.com/api/insights-results-aggregator/v2/cluster/1592e6e4-e90d-4dc3-ac97-1dd9d01514de/reports I0505 09:38:13.845422 1 requests.go:121] Retrieving report from https://console.redhat.com/api/insights-results-aggregator/v2/cluster/1592e6e4-e90d-4dc3-ac97-1dd9d01514de/reports I0505 09:38:14.023335 1 insightsreport.go:184] Report retrieved I0505 09:38:14.048517 1 insightsreport.go:239] Report retrieved correctly I0505 09:39:38.579085 1 controller.go:554] The operator is healthy I0505 09:40:38.528138 1 secretconfigobserver.go:136] Refreshing configuration from cluster pull secret I0505 09:40:38.533968 1 secretconfigobserver.go:249] Found cloud.openshift.com token I0505 09:40:38.533985 1 secretconfigobserver.go:162] Refreshing configuration from cluster support secret I0505 09:40:38.538358 1 secretconfigobserver.go:119] support secret does not exist I0505 09:41:38.580070 1 controller.go:554] The operator is healthy I0505 09:43:38.579892 1 controller.go:554] The operator is healthy I0505 09:45:38.520097 1 configmapobserver.go:84] configmaps "insights-config" not found I0505 09:45:38.524654 1 configmapobserver.go:84] configmaps "insights-config" not found I0505 09:45:38.538767 1 secretconfigobserver.go:136] Refreshing configuration from cluster pull secret I0505 09:45:38.543709 1 secretconfigobserver.go:249] Found cloud.openshift.com token I0505 09:45:38.543726 1 secretconfigobserver.go:162] Refreshing configuration from cluster support secret I0505 09:45:38.548114 1 secretconfigobserver.go:119] support secret does not exist I0505 09:45:38.579247 1 controller.go:554] The operator is healthy I0505 09:47:38.580183 1 controller.go:554] The operator is healthy I0505 09:49:38.579073 1 controller.go:554] The operator is healthy I0505 09:50:38.548157 1 secretconfigobserver.go:136] Refreshing configuration from cluster pull secret I0505 09:50:38.553894 1 secretconfigobserver.go:249] Found cloud.openshift.com token I0505 09:50:38.553914 1 secretconfigobserver.go:162] Refreshing configuration from cluster support secret I0505 09:50:38.557977 1 secretconfigobserver.go:119] support secret does not exist I0505 09:51:38.579181 1 controller.go:554] The operator is healthy I0505 09:52:28.310918 1 insightsuploader.go:146] Nothing to report since 2026-05-05T09:37:08Z I0505 09:53:38.580012 1 controller.go:554] The operator is healthy I0505 09:55:38.520589 1 configmapobserver.go:84] configmaps "insights-config" not found I0505 09:55:38.526564 1 configmapobserver.go:84] configmaps "insights-config" not found I0505 09:55:38.558671 1 secretconfigobserver.go:136] Refreshing configuration from cluster pull secret I0505 09:55:38.563705 1 secretconfigobserver.go:249] Found cloud.openshift.com token I0505 09:55:38.563724 1 secretconfigobserver.go:162] Refreshing configuration from cluster support secret I0505 09:55:38.567947 1 secretconfigobserver.go:119] support secret does not exist I0505 09:55:38.578578 1 controller.go:554] The operator is healthy I0505 09:57:38.581877 1 controller.go:554] The operator is healthy I0505 09:59:38.581238 1 controller.go:554] The operator is healthy I0505 10:00:38.568410 1 secretconfigobserver.go:136] Refreshing configuration from cluster pull secret I0505 10:00:38.575220 1 secretconfigobserver.go:249] Found cloud.openshift.com token I0505 10:00:38.575240 1 secretconfigobserver.go:162] Refreshing configuration from cluster support secret I0505 10:00:38.579562 1 secretconfigobserver.go:119] support secret does not exist I0505 10:01:38.582997 1 controller.go:554] The operator is healthy I0505 10:03:38.581921 1 controller.go:554] The operator is healthy I0505 10:05:38.523376 1 configmapobserver.go:84] configmaps "insights-config" not found I0505 10:05:38.579605 1 secretconfigobserver.go:136] Refreshing configuration from cluster pull secret I0505 10:05:38.580018 1 controller.go:554] The operator is healthy I0505 10:05:38.584637 1 secretconfigobserver.go:249] Found cloud.openshift.com token I0505 10:05:38.584662 1 secretconfigobserver.go:162] Refreshing configuration from cluster support secret I0505 10:05:38.589225 1 secretconfigobserver.go:119] support secret does not exist I0505 10:07:38.582532 1 controller.go:554] The operator is healthy I0505 10:07:47.634487 1 insightsuploader.go:146] Nothing to report since 2026-05-05T09:37:08Z I0505 10:09:38.582858 1 controller.go:554] The operator is healthy