Paper review · November 12, 2026
FERN: Cross-benchmark Text-to-SQL table retrieval with frozen retrievers and learned ranking
FERN: Cross-Benchmark Table Retrieval for Text-to-SQL with Frozen Retrievers and Learned Ranking
Using three frozen retrievers and a LightGBM ranker trained once on BIRD, the method achieved table recall comparable to LLM-based schema linking without a generative LLM.
Problem
To do Text-to-SQL over a unified catalog with hundreds of tables, one must first find the tables a question needs. Recent methods (such as RASL) use an externally hosted LLM like Claude 3.5 to extract keywords and select tables. This is costly, and because schemas and questions must be sent outside, it can be hard to use in enterprise settings.
Method
BM25 and embedding models (Qwen3-Embedding-0.6B, BGE-large) are combined with Reciprocal Rank Fusion to retrieve 100 candidate tables. Each candidate is represented with up to 120 features, including retrieval scores, agreement across retrievers, database-level support, lexical overlap, cell-value matches, column types and the foreign-key join graph. A LightGBM lambdarank ranker is trained once on 9,394 BIRD training questions and applied as-is to Spider, Fiben and BEAVER. Three variants, FERN-ev, FERN-q and FERN-schema, are used depending on the available information (evidence, cell values, types).
Results
- On Spider, which was not used for training, the candidate pool alone reaches RASL's recall@15 (98.0). The ranker raised complete@5 (the share of questions with all gold tables in the top 5) from 83.9% to 95.6%.
- Recall at 5 tables was within 1 point: Spider 96.7 (RASL 97.0) and BIRD with evidence 96.6 (RASL 97.5). BIRD without evidence was 89.3, 8.2 points lower.
- On 121 real BEAVER questions, recall@15 was 82.1, between two retrievers trained on BEAVER (80.8, 84.5). However, complete@15 was 49.6, lower than both (54.5, 62.0).
- On Fiben, recall@15 was 54.2, well below RASL (69.2) (the question sets differ).
- On BIRD dev, median latency was 208 ms per question on a single A100, with no external model calls.
Limitations
RASL was not rerun; comparisons are against its published numbers. The differences are therefore score differences rather than statistical tests, and exact alignment of questions and gold labels was not confirmed. For Fiben, the question sets themselves differ. RASL selects N or fewer tables while FERN returns exactly k, so actual output sizes may differ. BIRD dev and BEAVER were partly exposed during design decisions. Performance at finding complete table sets was insufficient on opaque financial schemas (Fiben) and enterprise data warehouses (BEAVER).
Why it matters
For enterprises that cannot send internal data to external LLMs, it offers a practical baseline for a table retrieval stage that runs quickly and cheaply on local hardware. It also shows the conditions (a small budget, an opaque schema) under which a ranker trained only on public data falls short.
Key terms
- Schema linking
- The early stage of text-to-SQL that finds the tables and columns a natural-language question needs in the database schema and links them to the question.
- Learning to Rank(LambdaRank)
- A method that trains a model to directly optimize the order of a list of candidates. Here it is implemented with LightGBM gradient boosting trees.
- Reciprocal Rank Fusion
- A simple fusion method that combines the rankings from several retrievers into a single candidate list by summing scores of the form 1/(60+rank).
- complete@k
- The share of questions for which every gold table the question needs appears in the top k. Missing even one table makes the SQL wrong, so this is stricter than recall.
Original
Authors: Arjun Rao, Nishant Gopinath · Posted 2026-11-10 · License CC BY 4.0 · Based on the full text
This review is an AI-written summary of the paper. Please treat the original paper as the reference.