How much do hints matter? When Hindsight Self-Distillation works in financial reasoning

Problem

Financial institutions subject to data residency rules have to use small open-weight models they can run in-house, and these models are weaker than frontier models at multi-step quantitative reasoning. On-policy Distillation requires a strong external Teacher, and On-policy Self-Distillation (OPSD) requires a complete reference solution. The question is whether that much information is really needed.

Method

GPT-OSS-20B is used as the Student. For each incorrectly answered problem, a frozen self-copy of the model consults the correct answer and produces the first error step, a short critique (the hint) and an error type. Simple arithmetic mistakes contribute no gradient. The Trajectory is cut just before the error, and the Student is trained with reverse KL over a 100-token Rollout to follow the next-token distribution of a frozen Teacher (a self-copy) that sees the same prefix plus the hint. Failed problems are re-collected and the Teacher is re-frozen every epoch. Using the FinanceReasoning function library, the authors also built a corpus of multi-step financial problems in which input values must be derived by applying accounting conventions.

Results

• On FinanceReasoning Hard, Fin-SD raised accuracy by 3.03 percentage points over the base model. A control in which the hint was replaced with generic fixed feedback reached the same level, at +3.36 percentage points. The authors interpret this to mean that most of the gain comes from where supervision is applied, not from the content of the hint.
• Hints written by GPT-OSS-120B, which has about 6 times as many parameters, were not meaningfully better than self-generated hints.
• Compared with OPSD, which uses complete solutions, accuracy was similar (72.19% vs 70.42%), tokens generated at evaluation were 9.5% fewer, and the within-budget completion rate was directionally higher by +2.44 percentage points.
• After Fin-SD training, there was no meaningful performance drop on MMLU-Pro general reasoning or on the FinTrust compliance evaluation across 14 metrics.

Limitations

There is only one student model, GPT-OSS-20B, and the main evaluation set, FinanceReasoning Hard, is small at 238 questions, which limits the statistical reliability of differences of a few percentage points. The authors state that the results hold for a corpus in which non-arithmetic failures are mainly convention-selection errors. Validation of the error-type classifier consists solely of an informal check of 50 cases (96%).

Why it matters

Being able to improve the domain reasoning of small in-house models without a strong external Teacher or reference solutions is practical for regulated industries. The finding that 'where you correct' matters more than 'what you tell' leaves room to simplify Self-Distillation design.

Key terms

On-policy Distillation

A Distillation (knowledge distillation) method in which the Teacher provides a distribution at every token over Rollouts generated by the Student itself, and the Student is trained on them.

Self-Distillation

A method that Distills without a separate Teacher, using a copy of the model itself that is given extra information as the Teacher.

Reverse KL

KL divergence that measures the difference from the Teacher with respect to the Student's distribution. It tends to concentrate on modes to which the Teacher assigns high probability.

Original

• arXiv abstract
• PDF
• HTML full text
• Code
• Hugging Face

Authors: Johnny Yu, Cyril Gorlla · Posted 2026-10-26 · License arXiv 비독점 배포 라이선스 · Based on the full text

This review is an AI-written summary of the paper. Please treat the original paper as the reference.