What is mechanistic interpretability? Features, circuits and the sparse autoencoder

Mechanistic interpretability is the attempt to work out what computations a neural network is actually performing inside itself, rather than inferring what it can do from what it outputs. The practical goal is to be able to say which internal structures produce a behavior, and then to test that claim by intervening on them. It is the main research program aimed at verifying a model's properties rather than measuring them.

Why not just test the model's behavior?

Because behavioral testing can only tell you what happened on the inputs you tried.

A model that answers a thousand safety questions correctly has told you that it answers those thousand questions correctly. It has not told you whether it would behave differently on inputs you did not think of, or whether it recognized that it was being tested. That second problem is not hypothetical: evaluations of frontier models have found chain-of-thought reasoning that shows awareness of being assessed.

Interpretability targets the gap directly. If you can identify the internal mechanism, you can in principle predict behavior on inputs you have not tried, which is what verification means in every other engineering discipline. This is also why benchmark scores stop being informative once every model passes.

What is a feature?

The central concept, and the thing the field is trying to find.

A neuron in a language model is usually not interpretable on its own: it fires for an unrelated-looking mixture of things. The dominant explanation is superposition, the idea that a model represents far more concepts than it has neurons by encoding them as directions in activation space rather than one concept per neuron.

A feature is one of those directions. If superposition is the right picture, then the interpretable units are not neurons but combinations of them, and the task is to find the combinations.

How do sparse autoencoders find features?

A sparse autoencoder is a small network trained on the model's internal activations. It compresses each activation into a much wider layer in which only a few units are allowed to be active at once, then reconstructs the original. The sparsity constraint forces the wide layer to use a small number of reusable parts, and those parts are candidate features.

The appeal is that it requires no labels. You do not tell it what to look for, and the features emerge from the activations.

Anthropic's 2024 work applied this at scale to Claude 3 Sonnet, training autoencoders of 1 million, 4 million and 34 million features on the model's middle-layer residual stream, using scaling laws to pick hyperparameters. The features it found were multilingual and in some cases responded to images despite the training being on text, and they fired both for concrete instances of a concept and for abstract discussion of it.

OpenAI published a parallel result on 6 June 2024, reporting 16 million features in GPT-4 using k-sparse autoencoders, with clean scaling laws in autoencoder size and sparsity and new metrics for feature quality.

Does finding a feature prove anything?

Only if you can intervene on it, which is the part that distinguishes this work from pattern-spotting.

The test is causal. Amplify or suppress a candidate feature and see whether the behavior you attributed to it changes in the predicted way. Anthropic's most-discussed demonstration was exactly this: amplifying a feature associated with a landmark produced a version of the model that steered almost every conversation toward it.

That is a real causal claim and it is checkable. It is also the reason feature names should be read carefully. A critic of the same work pointed out that the feature in question is only well described by its label at high activation levels, and fires for other things most of the time. The label is a summary of a distribution, not a definition.

What are circuits?

Features are the nouns. Circuits are the verbs.

A circuit is a connected path of features and computations that implements some behavior, such as recalling a fact, following an instruction, or copying a name from earlier in the context. Anthropic's 2025 work on attribution graphs traced these paths through a production model, publishing both the method and a set of case studies on how specific behaviors are implemented.

This is where the field is aiming. A feature list says what concepts a model represents. A circuit says what it does with them, which is closer to the kind of statement a safety case would need.

Where did this field come from?

It grew out of two older strands, and knowing them helps make sense of the vocabulary.

The first is computer-vision interpretability, which spent years identifying what individual units in convolutional networks responded to, from edges to textures to object parts, and then tracing how those composed into recognizable circuits. The language the field uses, features and circuits, comes from there.

The second is the superposition hypothesis. Once researchers accepted that a network can represent more concepts than it has neurons, the unit of analysis had to change from the neuron to the direction, and dictionary learning became the obvious tool for recovering directions.

Sparse autoencoders arrived as the practical implementation of that idea, and the 2023 and 2024 results were about whether the method would survive being scaled from small research models to production ones. The answer, with caveats, was yes.

Who does this work?

Mostly the labs, which is itself a finding worth noting.

Anthropic runs the largest dedicated effort and publishes it in the open, in its Transformer Circuits thread. OpenAI published its sparse autoencoder work with code and feature visualizations, though two of the paper's nine authors had left the company by the time it appeared. Google DeepMind has its own interpretability team, and there is a substantial independent and academic community, much of it working on smaller open-weight models because frontier activations are not public.

That last point is the structural constraint. The models that matter most for safety are the ones whose internals only their owners can inspect, so outside researchers either work on proxies or work on access granted by the lab.

What can this actually do for safety today?

Three things, modestly.

It can detect. Features associated with deception, sycophancy and power-seeking have been identified, which raises the possibility of monitoring for internal states rather than only for outputs.

It can steer. Interventions on features change behavior, which is a control surface that does not require retraining.

It can audit reasoning. Interpretability is one of the few routes to checking whether a model's stated reasoning corresponds to its actual computation, which matters because chain-of-thought monitorability is fragile and may not survive future training methods.

What it cannot yet do is certify. There is no published decomposition that accounts for everything a frontier model computes, and both labs say so.

What are the honest limitations?

The field states them more clearly than most, which is to its credit.

Coverage is partial. A 16 million feature autoencoder does not capture everything GPT-4 does, and OpenAI said covering all of it would be challenging even with the improved techniques.

Interpretation is uneven. Many recovered features are hard to understand, or activate on things unrelated to the concept they appear to represent.

Evaluation is unsolved. There is no gold-standard method for assessing how good a dictionary-learning run is, which means the field cannot straightforwardly say whether it is making progress.

And scale is the opponent. Autoencoders must get larger as models learn more concepts, so the cost of interpretation rises with the thing being interpreted.

How should a reader judge an interpretability result?

Four questions.

• How many features, on which model, at which layer? A result on a small open model is a different claim from one on a frontier model.
• Was the causal test run, or only the labeling? A feature that has not been intervened on is a correlation.
• What fraction of the model's behavior does the decomposition explain? This is the number most often left out.
• Is the method published in enough detail to be reproduced by someone outside the lab?

Most interpretability coverage answers the first question and skips the other three.

What to watch

Whether circuit-level results extend from case studies to systematic coverage of a model's behavior. Whether anyone proposes an evaluation metric for decomposition quality that the field accepts. And whether any regulator or safety framework starts asking for interpretability evidence, which would turn a research program into a compliance requirement.