Paper review · November 18, 2026
Sage S2.1 Flash technical report: a 320B MoE model with scientific data perception and agent capabilities
Sage S2.1 Flash Technical Report
The authors combined dedicated encoders for molecules, sequences and spectra with large-scale training in verifiable environments to build a 320B MoE model that ranked first on document-based work and ML experiment benchmarks.
Problem
An agent that supports scientific research has to understand scientific data such as molecular structures, protein and DNA sequences, and spectra, not only text and images. It also has to carry out multi-step tasks itself, such as tool use, search, coding and operating scientific software.
Method
The authors attached Sage Sci-Perception to a 320B-parameter MoE backbone (a mix of Sparse and Linear Attention). Molecular graphs (MoleculeSTM) and protein, DNA and RNA sequences (ESM-2, DNABERT-2, RNA-FM) are processed by dedicated encoders, and spectra are represented as patches cut in order from ranges of measured values. A Projector for each modality maps these representations, along with positional information, into the backbone's embedding space. For post-training, the authors built a large number of environments where success can be checked through execution results, changes in environment state or submitted outputs. A key example is ToolSpace, which builds tool-composition graphs from real execution Trajectories and uses state-aware random walks to reverse-synthesize long-horizon tasks that span multiple environments, along with verification scripts. Search tasks were created by forward synthesis from knowledge graphs and backward synthesis from seed queries. Coding tasks were collected from SWE, terminal and web development tasks, with execution environments configured automatically.
Results
- In the main comparison against 5 baseline models, the authors reported first place on OfficeQA Pro (72.2), MLE-bench Lite (HumanRank 90.3), Harvey LAB and SpreadsheetBench 2.
- It ranked second on OSWorld-Verified (85.2), WebArena-Verified (67.6) and DeepResearch Bench II (54.2).
- The model was evaluated on 33 benchmarks across 7 domains. Of the 15 benchmarks in Figure 1, it ranked first on 4 and second on 7.
- ToolSpace built 77,760 tool-call pattern nodes and 804 environments from 2,042 tools. The generated SFT Trajectories average 18.86 tool calls (up to 209) and span an average of 2.29 environments (up to 22).
Limitations
The provided text is cut off partway, so the names of the baseline models, full scores per benchmark and the standalone performance of the scientific encoders (Sci-Perception) could not be confirmed. The reported rankings are limited to the comparison with 5 baseline models. The abstract names the model 'S1.5 Flash' while the title and body say 'S2.1 Flash', so the naming is inconsistent.
Why it matters
The paper shows both a design for attaching dedicated scientific-data encoders to a general-purpose LLM backbone and a way to build training environments at scale whose results can be verified automatically. The data synthesis pipeline is a useful reference for teams building agents for document and spreadsheet work or for automating ML experiments.
Key terms
- MoE(Mixture of Experts)
- An architecture that, for each token, selects and computes only some of several expert subnetworks. The total parameter count is large, but the computation per token can be reduced.
- Modality-specific Projector
- A small neural network that converts representations of non-text inputs, such as molecules, sequences and spectra, to the same dimension as the language model's embeddings.
- Verifiable environment
- A training environment where a program can automatically judge whether the agent finished the task, based on execution results or environment state.
- ToolSpace
- A framework that builds a graph of data and state dependencies between tools from real tool execution logs, then follows it to synthesize long-horizon tasks spanning multiple environments.
Original
Authors: Kun Cai, Haodong Chen, Guangran Cheng, Qingxiu Cheng, Xu Dai, Guanting Dong, Qiming Ge, Lixin Gu, et al. · Posted 2026-11-16 · License arXiv 비독점 배포 라이선스 · Based on the full text
This review is an AI-written summary of the paper. Please treat the original paper as the reference.