← Key papers

Paper review · November 18, 2026

Sage S2.1 Flash technical report: a 320B MoE model with scientific data perception and agent capabilities

Sage S2.1 Flash Technical Report

Language modelsAgentsScience & bio Intermediate Upvotes 85 cs.AI Intern Large Models

The authors combined dedicated encoders for molecules, sequences and spectra with large-scale training in verifiable environments to build a 320B MoE model that ranked first on document-based work and ML experiment benchmarks.

Problem

An agent that supports scientific research has to understand scientific data such as molecular structures, protein and DNA sequences, and spectra, not only text and images. It also has to carry out multi-step tasks itself, such as tool use, search, coding and operating scientific software.

Method

The authors attached Sage Sci-Perception to a 320B-parameter MoE backbone (a mix of Sparse and Linear Attention). Molecular graphs (MoleculeSTM) and protein, DNA and RNA sequences (ESM-2, DNABERT-2, RNA-FM) are processed by dedicated encoders, and spectra are represented as patches cut in order from ranges of measured values. A Projector for each modality maps these representations, along with positional information, into the backbone's embedding space. For post-training, the authors built a large number of environments where success can be checked through execution results, changes in environment state or submitted outputs. A key example is ToolSpace, which builds tool-composition graphs from real execution Trajectories and uses state-aware random walks to reverse-synthesize long-horizon tasks that span multiple environments, along with verification scripts. Search tasks were created by forward synthesis from knowledge graphs and backward synthesis from seed queries. Coding tasks were collected from SWE, terminal and web development tasks, with execution environments configured automatically.

Results

Limitations

The provided text is cut off partway, so the names of the baseline models, full scores per benchmark and the standalone performance of the scientific encoders (Sci-Perception) could not be confirmed. The reported rankings are limited to the comparison with 5 baseline models. The abstract names the model 'S1.5 Flash' while the title and body say 'S2.1 Flash', so the naming is inconsistent.

Why it matters

The paper shows both a design for attaching dedicated scientific-data encoders to a general-purpose LLM backbone and a way to build training environments at scale whose results can be verified automatically. The data synthesis pipeline is a useful reference for teams building agents for document and spreadsheet work or for automating ML experiments.

Key terms

MoE(Mixture of Experts)
An architecture that, for each token, selects and computes only some of several expert subnetworks. The total parameter count is large, but the computation per token can be reduced.
Modality-specific Projector
A small neural network that converts representations of non-text inputs, such as molecules, sequences and spectra, to the same dimension as the language model's embeddings.
Verifiable environment
A training environment where a program can automatically judge whether the agent finished the task, based on execution results or environment state.
ToolSpace
A framework that builds a graph of data and state dependencies between tools from real tool execution logs, then follows it to synthesize long-horizon tasks spanning multiple environments.

Original

Authors: Kun Cai, Haodong Chen, Guangran Cheng, Qingxiu Cheng, Xu Dai, Guanting Dong, Qiming Ge, Lixin Gu, et al. · Posted 2026-11-16 · License arXiv 비독점 배포 라이선스 · Based on the full text

This review is an AI-written summary of the paper. Please treat the original paper as the reference.