AI glossary · Advanced · Language models · Efficiency & infra
MoE
MoE (Mixture of Experts) is an architecture that places several small neural networks called "experts" inside a model and, for each input, picks only some of them to do the computation. This lets the model as a whole be large while reducing the amount of computation used at any one time.
About this term
- Level
- Advanced · Specialist terms that need the inner workings
- Category
- Models & generation
- Added
- 2026-10-08
How it works
An MoE model has a small directing component called a router and several expert networks. When a token comes in, the router scores which experts to send it to and picks only the few with the highest scores to compute. The outputs of the selected experts are mixed according to the router scores and passed to the next step. So even though the total number of parameters is very large, the parameters actually used to process a single token (active parameters) are only a fraction.
Why it matters
Making a model larger generally improves performance, but it usually raises computing costs as well. MoE stores a lot of "knowledge" but does not pull all of it out every time, so it can lower cost relative to performance. For this reason, many recently released large LLMs and open-weight models use an MoE architecture. This is also why model announcements list "total parameters" and "active parameters" separately.
Good to know
Even though computation drops, all the experts must be kept in memory, so the GPU memory required is still large. If the router sends work only to certain experts, training can become unstable, so a mechanism is needed to spread the work evenly. When experts are split across multiple GPUs, communication between devices can become a bottleneck. Training and post-processing such as quantization are also trickier than for standard models, and research on them is ongoing.
Example
Cloud companies offer large MoE models through APIs, so users can use large models without running them themselves. The news item on this site, 'AWS offers Z.ai's 753B MoE model GLM 5.3 on Amazon Bedrock,' is one example. The 753B in the title refers to the total parameter count; the part actually used in computation for each token is smaller.