Paper review · November 13, 2026
SteerDistill: Quantizing LLM agents with closed-loop Teacher steering
SteerDistill: Quantizing LLM Agents via Closed-Loop Teacher Steering
A full-precision Teacher guides corrections at the points where a quantized Student fails, and the results are distilled back into the Student, bringing 4-bit agent performance to within 1 point of BF16.
Problem
Low-bit quantization reduces the inference cost of LLM agents but lowers task success rates. The authors identify 'behavioral divergence' as the cause: quantized agents miss task requirements or environment conditions and produce environment changes different from those of the successful original model. This is hard to fix. Offline data may lack the situations where the Student actually fails, and the Student's own Rollouts rarely contain correct answers in those situations. In a BFCL V4 diagnosis, the Student got only 9.6% of divergence points right when retrying on its own.
Method
(1) Teacher-steered Trajectory correction: the Student's failed Rollout is aligned with a successful Teacher Trajectory based on environment state to find the first divergence point. The Teacher provides guidance containing the conditions the Student missed and correction rules, and the Student re-responds and re-executes from that point. The Teacher revises the guidance using execution feedback until the resulting environment state matches the reference. (2) Internalization: a Student trained without guidance follows, via forward KL, the predictions of a guided Student snapshot on corrected turns, and follows the full-precision Teacher via reverse KL on the remaining turns. As a result, neither the Teacher nor the guidance is needed at inference.
Results
- On Qwen3-8B W4A16, it averaged 53.18 across five agent benchmarks, 2.18 points above the strongest quantization baseline QeRL (51.00) and 0.36 points below BF16 (53.54).
- On Qwen3.5-4B and Gemma 4 12B-it (W4A16), it also beat QeRL by 1.71 and 1.67 points on average, and all three models were within 1 point of BF16.
- On Qwen3-8B W2A16, it averaged 42.97, 3.39 points above On-Policy QAD (39.58).
- Its τ²-Bench success rate was 1.84 percentage points higher than QeRL's, with 67% fewer training GPU hours. Average output tokens on successful tasks were 4.0K, close to BF16 (3.9K) (other quantization baselines: 4.1K–5.2K).
- In diagnostic experiments, Teacher steering raised Pass@4 at divergence turns from 9.6% to 82.3%.
Limitations
Fixing a failed Rollout requires a successful Teacher reference Trajectory for the same task. Failed Rollouts without a reference are used as-is without correction. Because verification compares environment states, an executable environment whose state can be checked is required. Experiments used only group-wise integer quantization on A100s, and the QeRL and NeMo-RL QAD baselines, built for NVFP4, were adapted to this setting for comparison. The Teacher was a task-specific GRPO post-trained checkpoint.
Why it matters
Agents call the model many times, so the cost savings from quantization can be large. Until now, however, the drop in success rate has been an obstacle. This paper shows a way to bring low-bit agents close to original performance while also cutting training cost, making it a useful reference for deploying agents at the edge or in low-cost serving.
Key terms
- QAD(Quantization-Aware Distillation)
- A technique that recovers performance lost to quantization by training a quantized Student model to follow the output distribution of a full-precision Teacher model.
- W4A16
- A quantization setting that represents weights as 4-bit integers and activations in BF16 (16-bit).
- Behavioral divergence
- The point at which a quantized agent misses a task condition and causes environment changes different from those of the successful original model.
- Forward KL / Reverse KL
- The difference in which direction KL divergence, a measure of how two probability distributions differ, is computed. It determines whether the training objective tries to cover the entire target distribution or focuses on its most likely region.
Original
Authors: Genming Cui, Xiangyu Li, Zixu Hao, Xin Ding, Yunxin Liu, Ting Cao · Posted 2026-11-10 · License CC BY 4.0 · Based on the full text
This review is an AI-written summary of the paper. Please treat the original paper as the reference.