← Key papers

Paper review · November 19, 2026

From TeleChat to Xing4: Developing an agentic LLM on Chinese chips and frameworks (Ascend, MindSpore)

TeleChat to Xing4: Advancing Frontier Agentic Models on Domestic Chips and Frameworks

Language modelsEfficiency & infraReinforcement learning Advanced Upvotes 149 cs.AI XingChen-AGI

The authors co-designed the model architecture, pretraining infrastructure and an agentic reinforcement learning platform for Ascend NPUs and MindSpore, and trained a 285B MoE model on 8,192 chips.

Problem

Most open-source LLMs are trained on GPUs. Training and serving large MoE models on a Chinese chip (Ascend 910C) and framework (MindSpore) requires architecture and system design matched to the hardware's characteristics: strong matrix compute, relatively weak vector compute and HBM bandwidth. In agentic reinforcement learning, execution time varies widely from task to task, which also lowers training efficiency.

Method

On the architecture side, the authors combined MLA with DSA (sparse attention) and IndexShare, which reuses indices across layers. They used a wider-shallower MoE with fewer layers and wider experts, and added DSpark, a speculative decoding method based on a block-level draft model. For pretraining, they removed pipeline parallelism using topology-aware hybrid sharding (HSDP) on MindSpore HyperParallel, and applied distributed execution of the Muon optimizer, load balancing for long sequences, and CPU affinity and core isolation. For post-training, they built TeleGym, which connects agent execution, evaluation and trajectory construction to policy optimization, and added asynchronous decoupled execution, bounded oversampling and partial rollout.

Results

Limitations

The attention comparison is based on a Huawei simulator. The authors note that it does not capture all real serving overheads, so the results may be optimistic. Within the provided text, there is little comparison of model quality (accuracy), and most efficiency figures are normalized relative values. TeleGym's effect is also described mainly through design rather than quantitative results.

Why it matters

The paper shows, with concrete engineering choices, that frontier-scale MoE models can be trained on thousands of chips in a Chinese AI chip ecosystem rather than on GPUs. It is a useful example of choosing model architecture (depth, width, attention) to fit hardware characteristics.

Key terms

DSA(DeepSeek Sparse Attention)
A sparse attention method in which a lightweight indexer selects only the top-K past KV entries for each query to attend to. It reduces KV access in long contexts.
Speculative Decoding
A small draft model proposes several tokens first, and the main model verifies them in one pass and accepts the correct leading portion. It is a technique for speeding up decoding.
HSDP(Hybrid Sharded Data Parallelism)
An extension of FSDP that shards model state according to the cluster's bandwidth hierarchy. In this paper, it made training possible without pipeline stages.
Partial Rollout
A method that continues unfinished rollouts instead of discarding them when the policy is updated. It reduces waste in agent tasks whose execution times vary.

Original

Authors: Xingchen AGI Lab, :, Baotian Hu, Bingkai Yang, Boyang Wang, Changzai Pan, Chao Lyu, Chao Wang, et al. · Posted 2026-11-17 · License arXiv 비독점 배포 라이선스 · Based on the full text

This review is an AI-written summary of the paper. Please treat the original paper as the reference.