Paper review · November 5, 2026
Route Anything: Model and Policy Routing for General-Purpose AI Systems
Route Anything: A Study of Model and Policy Routing for General-Purpose AI Systems
The authors built a trajectory matrix that runs 452 candidate configurations on the same inputs across 25 benchmarks, along with a theory of routing, and measured how much per-input routing improves performance and cost efficiency.
Problem
A general-purpose AI system has to pick one of several candidates for each input, such as language models, image, video and speech generators, and robot policies. Until now, routing research has mostly focused on language models only. As a result, it has not been established whether routing actually helps across modalities, or under what conditions it does.
Method
The authors developed a theory of optimal routing under a quality-cost trade-off. It includes a price-aware optimal router theorem, an "information ladder" stating that the achievable performance depends on the information the router observes, a proposition that routers should learn outcomes rather than winners, and a quality floor certification. For experiments, they built a matrix in which every candidate was run on the same inputs and scored (374,401 runs) across 25 benchmarks covering text and image QA, image, video and speech generation, code and work agents, and robot manipulation. On top of it, they fine-tuned Qwen3.5-4B to train and compare an outcome-regression router, a type router and a winner-classification router.
Results
- On 19,814 test inputs, per-input oracle routing scored 87.98, 5.4 points above the best fixed candidate per benchmark (82.60). The gap ranged from 0.7 points for speech generation to 18–19 points for video generation and robot manipulation, and choosing the best candidate per task type recovered only 8% of this gap.
- Applying a single capability-tree router to 16 benchmarks gave 79.98, 0.3 points ahead of the fixed candidate (79.68) (oracle: 84.80). The gains came from agents, video and image generation. In contrast, on GenEval2 and Qwen-Image-Bench, none of the 124 router configurations significantly beat the fixed generator.
- In RoboTwin robot manipulation, a router that selects a policy from camera frames achieved a 71.6% success rate. This was higher than the best single policy (70.5%), with 24% less policy execution time.
- With the same inputs and backbone, a router trained to regress each policy's success reached 71.6%, while a router trained to classify the winner reached only 42.8%. This is because 94% of the winner labels were decided by latency-based tie-breaking.
- The authors report that, compared with the reference model, routing improved efficiency by an average of 8.82x across four testbeds (25.5x for code, 7.1x for text QA), with a quality loss of less than 0.5 points.
Limitations
The margin by which the learned router beat the fixed candidate was small, at 0.3 points overall, and there was no significant improvement in image generation. Most of the matrix has no cost records, so the cost analysis focused mainly on the code and text testbeds where prices were recorded. Part of the oracle gap is outcome noise that no state can predict (on AA-LCR, the oracle advantage disappeared when scoring used the average of three runs). The candidate pools are small, with 3–7 per benchmark.
Why it matters
This work provides grounding for designing systems that combine several specialized models instead of scaling up a single model. It separates the upper bound on the gains routing can deliver from the gains that can actually be achieved. In practice, it gives guidance such as "learn per-candidate outcomes instead of winner labels" and "compare candidates on the same inputs." It also confirms that the cost savings are much larger than the quality gains.
Key terms
- Router
- A module that chooses, for each input, which of several candidate models or policies to hand it to. The goal is to reduce cost while maintaining or improving quality.
- Oracle routing
- An idealized baseline that picks, after the fact, the candidate that actually produced the best result for each input. It represents the upper bound on the gains routing can deliver.
- Information ladder
- A theoretical result stating that the coarser the information the router observes (benchmark level → task type → the input itself), the lower the achievable performance ceiling.
- Quality floor certification
- A procedure that uses sequential testing on held-out data to guarantee that the probability of the selected router's quality loss exceeding a tolerance stays below a set level.
Original
Authors: Pengfei Zhou, Yixing Ma, Hexin Wang, Weidong Tang, Zhenglin Wan, Wangbo Zhao, Yang You · Posted 2026-11-03 · License CC BY-NC-SA 4.0 · Based on the full text
This review is an AI-written summary of the paper. Please treat the original paper as the reference.