Paper review · October 20, 2026
Recognition Is Not Grounding: A Multilingual Benchmark for Financial Evidence Extraction
Recognition Is Not Grounding: Benchmarking Multilingual Financial Evidence Extraction
The authors built a benchmark that links English, French and Greek financial reports through a shared accounting taxonomy, and showed that models recognize accounting concepts almost perfectly but fail to find every figure that belongs to them.
Problem
Comparing disclosures from companies in different countries requires recognizing the same accounting concept across languages and finding every figure that belongs to it (including prior-period comparative figures). Existing benchmarks did not evaluate these two abilities together on the same multilingual evidence.
Method
From the FY2025 ESEF reports of 30 English-, 30 French- and 30 Greek-language issuers, the authors extracted figures and their IFRS concept mappings using the iXBRL tags the issuers applied themselves. Actual disclosures from different companies were linked through 187 shared IFRS concepts. There are four tasks. T1 identifies financial figures and their attributes (type, period, debit/credit). T2a picks the concept shared by the highlighted figures. T2b selects every figure that belongs to a given concept. T2c picks out the excerpt with a different concept, without candidate labels. T2a and T2b are evaluated as pairs in the same context. Two financial experts reviewed a 10% stratified sample. Fifteen models were evaluated.
Results
- The authors report that eight API and open-weight reference models reached 97.3–99.4% accuracy on concept recognition (T2a), yet in 26.6–41.9% of the groups where they identified the concept correctly, they missed at least one target figure.
- For seven of the eight reference models, evidence recall was lowest for Greek. By contrast, concept recognition stayed high even when English evidence was removed.
- On concept discrimination without candidate labels (T2c), models differed widely: Gemini 3.8 Flash scored 95.1%, while GPT-5.6 Luna scored 49.4%.
- Under this protocol, the finance-specialized models evaluated showed only limited gains over their respective base models.
Limitations
The study covers only three languages, 90 issuers and one year of reports (FY2025). Expert review was applied only to a 10% sample; the remaining annotations use the issuers' tags as is. API models were evaluated with reasoning turned off, and compute was not matched across models. Because T2 shares issuers, no bootstrap confidence intervals were reported, and the statistical tests are exploratory.
Why it matters
The work shows that in financial document extraction, a metric for "got the concept right" does not guarantee "found all the required figures." It tells practitioners that omissions that break real-world analysis, such as missing prior-period comparative figures, need to be measured separately.
Key terms
- iXBRL
- A format that embeds accounting concept tags directly on figures in the text and tables of financial reports. It is used in European ESEF disclosures.
- IFRS Accounting Taxonomy
- A taxonomy that organizes accounting concepts from International Financial Reporting Standards (IFRS), such as revenue and profit, into standard identifiers.
- Exhaustive Fact Selection
- A task that requires selecting, from context, every figure that belongs to a given concept without missing any. It is evaluated with recall and exact set match.
Original
Authors: Lingfei Qian, Yan Wang, Xueqing Peng, Polydoros Giannouris, Nadhem Zmandar, Yi Han, Yanguang Liu, Víctor Gutiérrez-Basulto · Posted 2026-10-14 · License CC BY 4.0 · Based on the full text
This review is an AI-written summary of the paper. Please treat the original paper as the reference.