What is distillation in AI? How a small model learns from a big one, and when it becomes an attack

Distillation in AI means training a smaller model to imitate a larger one by learning from the larger model's outputs. The larger model is the teacher, the smaller is the student, and the goal is a model that is cheaper and faster to run while keeping as much of the teacher's ability as possible. It is routine engineering. It has also become a flashpoint, because the same method can be used to copy a rival's model without permission.

Where the idea comes from

The technique was laid out in the 2015 paper "Distilling the Knowledge in a Neural Network" by Geoffrey Hinton, Oriol Vinyals and Jeff Dean, submitted to arXiv on March 9, 2015. The paper's central idea is to combine the outputs of many models, or one large one, and condense that knowledge into a single network that is easier to deploy.

The intuition is simple. A large model's answers carry more information than a bare right-or-wrong label: they show how confident it is and which wrong answers were nearly right. A student trained on those richer signals can learn more from fewer examples than one trained on raw data alone. That is why labs distill their own big models into smaller, cheaper product tiers.

Why it is legitimate, and when it is not

Anthropic's February 23, 2026 report on the subject opens with this framing: "Distillation is a widely used and legitimate training method." The line it draws is about permission and intent. Distilling your own model is normal. Distilling a licensed model within its terms is normal. Using fake accounts, resold access and proxy networks to pull a competitor's outputs at industrial volume is what Anthropic calls a distillation attack.

In that report, Anthropic said three Chinese labs ran such campaigns against Claude: DeepSeek with over 150,000 exchanges, Moonshot AI with over 3.4 million and MiniMax with over 13 million, a total of over 16 million through about 24,000 fraudulent accounts. Anthropic describes the accounts as arranged in "hydra cluster" networks, with one proxy network running over 20,000 accounts at once. These are Anthropic's allegations, published by a competitor of the labs named; the labs' responses are not in the documents we opened.

What the attackers want: reasoning, not just answers

The valuable output is the model's reasoning. Anthropic's October 12, 2026 post says a model trained on full Claude reasoning transcripts scored 80.4 percent on SWE-bench Verified, against 71.8 percent when trained on reasoning summaries only. That 8.6-point gap is a reason for labs to hide detailed reasoning traces, and a reason for rivals to try to extract them.

The February report listed what each lab targeted: reasoning, grading for reinforcement learning and politically safe answers for DeepSeek; agentic reasoning, tool use, coding and computer use for Moonshot; agentic coding and tool orchestration for MiniMax. It also said MiniMax redirected nearly half its traffic within 24 hours of a new Claude model's release.

How labs detect it

Anthropic's February report lists the detection signals: synchronized traffic, shared payment methods, repetitive prompt templates and very high volume concentrated on narrow tasks. It attributes campaigns using IP correlation, request metadata and infrastructure indicators, with partner corroboration. In some cases it says metadata tied activity to specific researchers.

The October post adds countermeasures: preserved thinking, which blocks editing of earlier context for API accounts created on or after August 31, 2026; a two-stage classifier with a false positive rate below 0.1 percent; and stricter limits, bans and charges for blocked requests since September 24. Anthropic says early evidence shows a 90 to 99 percent cut in harvested data, a company claim with no independent audit. We cover the detail in our report on the new defenses.

Why it is a policy issue

Anthropic argues distillation attacks undermine export controls, because a rival can recover capability from a US model without buying US chips, and create national security risks because distilled models may lack safeguards. That claim ties distillation to the chip-control debate; see our tracker of AI chip smuggling cases for the hardware side of the same argument.

The harder question is the grey zone between terms of service and law. The documents we opened describe commercial and policy responses, not court rulings, and we found no ruling on whether such distillation is unlawful.

How to read a distillation headline

Four questions sort most stories. Whose model was the teacher, and did the student's builder have permission? How large was the volume, and was it spread over fake accounts? What was extracted, answers or reasoning traces? And who measured the claim? A figure produced by the victim, such as Anthropic's, is a claim worth recording but not the last word.

Common misunderstandings

Distillation is not the same as stealing weights. A distilled student never sees the teacher's internal parameters; it only sees what the teacher says. That is why the defenses focus on outputs, accounts and traffic patterns rather than on locking down files.

It is also not the same as ordinary use. Anthropic's own wording separates the method from the misuse: the technique is legitimate, and the attack is the covert, large-scale version. And a distilled model is not a clone. It learns from a sample of behavior, so it generally captures a narrower slice of the teacher's ability, which is why attackers target specific skills such as coding, tool use or reasoning.

Our take

Distillation is neither scandal nor miracle. It is how cheap models get good, and it is also how an unlicensed copy gets made, and the difference is consent, scale and concealment. The interesting development is that labs are now treating reasoning traces like trade secrets and designing products around not leaking them. We would watch for the first independent test of those defenses and the first legal case that tests whether bulk extraction of outputs is actionable.