Mixture of experts explained: total versus active parameters, routers and the trade-offs

A mixture-of-experts (MoE) model is a neural network split into many specialist sub-networks, where a small router picks only a few of them to process each token. The model can hold far more total parameters than it uses on any one token, which is why recent frontier-scale releases quote two numbers: total and active parameters.

That one idea explains the specs in this week's launches. Microsoft's MAI-Base-2 has 2.8 trillion total parameters and 80 billion active. Meta's Muse Spark Open has 816 billion total and 42 billion active. Nvidia's Nemotron 3.5 Super VL has 120 billion total and 12 billion active.

How it works

In an ordinary dense transformer, every parameter takes part in processing every token. In an MoE model, the feed-forward part of each layer is replaced by many parallel feed-forward blocks, called experts, plus a small router network. For each token, the router scores the experts and sends the token to the top few.

Mixtral 8x7B, from Mistral, is a clean example. Its paper describes each layer as having 8 feedforward blocks, with a router selecting two of them for each token at each layer. Each token can access 47 billion parameters in total but uses only 13 billion during inference. Mistral reported that it outperforms or matches Llama 2 70B and GPT-3.5 across all evaluated benchmarks, and released it under Apache 2.0.

The experts are not hand-assigned subjects like "law" or "chemistry." The router and experts are trained together, and what each expert specializes in emerges from training and is often hard to describe.

Where the idea came from

The sparsely gated MoE layer was introduced in a January 23, 2017 paper by Noam Shazeer, Azalia Mirhoseini, Krzysztof Maziarz, Andy Davis, Quoc Le, Geoffrey Hinton and Jeff Dean. The abstract describes a layer with "up to thousands of feed-forward sub-networks" and reports "greater than 1000x improvements in model capacity" with only minor losses in computational efficiency on GPU clusters. One of its models reached 137 billion parameters by placing the layer between stacked LSTM layers.

So the idea is nearly a decade old. What changed is scale, hardware and the tooling to keep many GPUs busy while tokens are routed between them.

What you gain

The central trade is compute for memory. A model with a small active share does far fewer calculations per token than a dense model of the same total size. By our arithmetic, MAI-Base-2 activates about 2.9 percent of its parameters per token, Muse Spark Open about 5.1 percent and Nemotron 3.5 Super VL 10 percent.

That lowers the cost of training a model to a given quality and of serving it once trained. Microsoft's claim that MAI-Base-2 is 4.2 to 12.3 times more pre-training FLOPs-efficient than DeepSeek-V4-Pro-Base rests on this effect, though that figure is Microsoft's own and untested outside.

What you pay

Sparse is not small. All the experts must sit in memory even though only a few run at a time. That is why Meta's demonstration of its 816 billion parameter model on two Mac Studios with up to 512 GB of unified memory each, pairing to about a terabyte, is notable: the weights are enormous even if the per-token work is not.

There are other costs. Routing creates communication between GPUs, which lowers hardware utilization. Microsoft reports 21 percent average model FLOPs utilization on its run, and says it raised utilization inside the expert layers from 24 to 35 percent. Experts can also be unevenly used, so training needs tricks to balance the load.

How to read the numbers on a launch

When a lab quotes parameters, ask which one. Total parameters tell you the memory footprint and, loosely, the capacity. Active parameters tell you roughly the compute per token. Comparing a 2.8 trillion parameter MoE model to a 70 billion dense one by total size is meaningless, and comparing by active size is only a rough guide.

Quality does not follow either number cleanly. A model's data, training recipe and post-training matter as much as its architecture. For how those claims get tested, see our explainer on what AI leaderboards rank and what a top score hides, and for the cost side, how much a training run costs in chips, power and capex.

Common misreadings

• "It is several small models." No. The experts are parts of one network trained together.
• "Only 80 billion parameters means it is an 80 billion model." No. The model must store and learn all 2.8 trillion.
• "Sparse means cheaper to run for everyone." Only if you can hold all the experts in memory.

A quick worked example

Take a model with 100 experts per layer where the router picks 2 for each token. Each token touches 2 percent of the expert parameters in every layer, while the attention layers and embeddings, which are shared, run for every token. That is why active parameters are not exactly the expert share times the total: shared components add to the active count. The numbers labs quote already include that, so 80 billion active for MAI-Base-2 is the full per-token working set, not just the expert slice. Memory, though, must hold every expert, which is the cost of the design.

Our take

Mixture-of-experts is now the default shape of large models, and it has earned the place: the compute savings are real and documented in papers going back to 2017. The caution is that the headline comparisons built on it, efficiency multiples and "most capable" labels, are mostly lab-run. Read both parameter numbers, ask who ran the test and treat the active count as the figure that predicts speed and cost.