Do Sparse Autoencoder Features Add Up? The SAE Additivity Residuals (SAR) Paper Explained

Sparse autoencoders (SAEs) are the workhorse of modern interpretability: they break a model's activations into a few "features" that people can name and, increasingly, steer. A quiet assumption sits underneath much of that work: that you can understand what a model does by adding up what each feature does on its own. A new paper, Behavioral Compositionality of Sparse Autoencoder Representations by Haotian Ma, Ziye Mei and Frederic Sala (Hugging Face lists the University of Wisconsin-Madison, Department of Computer Sciences), tests that assumption directly. It is listed on Hugging Face Papers, and a code repository has been released.

The answer, in plain terms: features compose better than a random baseline would, so the additive picture is mostly reasonable. But it is not complete. Adding pairwise interaction terms improved predictions in every one of the 18 settings the authors tested, which means explanations and interventions that treat features as independent leave something out.

TL;DR: the paper in questions

Background: why additivity matters

An SAE takes a hidden activation inside a language model and rewrites it as a sparse sum of learned directions. The pitch is that each direction is a human-readable concept. If that holds, you can switch features on or off to explain or steer the model. We covered the broader interpretability push in Goodfire's argument that neural networks are already clean and in Anthropic's natural language autoencoders, which take a different route to reading a model's internal state.

Linear decoding, though, is a statement about the representation, not about behavior. The paper's opening point is that "the downstream effects of those features need not compose additively." The rest of the network is nonlinear: removing feature A and feature B together can do something different from removing each alone. If the interactions are large, then a list of per-feature explanations can look tidy and still mislead.

What SAR measures

The authors define SAE Additivity Residuals as the share of behavioral variation that remains unexplained once you model feature effects up to a chosen order. In the setup described in the full text:

1. For each prompt, take the top 16 active SAE features and their decoded contributions.
2. Pick a coalition, meaning a subset of those features. Keep the contributions in the subset, remove the others, and hold the rest of the activation fixed.
3. Run the patched activation through the frozen upper layers and measure a scalar behavior, such as how strongly the model prefers the correct token.
4. Sample many coalitions (384 per prompt: 256 to fit the models, 128 held out for testing).
5. Fit a first-order model (one effect per feature) and a pairwise model (effects per feature and per pair). SAR is the held-out error divided by the held-out variance.

Under independent random sampling of subsets, the paper shows the population SAR equals the normalized energy of interactions above the chosen order, and that it bounds how well the best low-order approximation can rank coalitions. In other words, SAR is not just a heuristic score; it has a clean interpretation as "how much is left in the higher-order interactions."

SAE features steering a model behavior, illustrating sparse autoencoder additivity testing

The experiments: 3 models, 6 behaviors, 18 settings

The main experiments use three open models paired with published SAEs:

Each is tested on six behaviors: simple copying, subject-verb agreement, gendered pronoun preference, CounterFact, indirect object identification (IOI), and a greater-than comparison. Three models times six behaviors gives the 18 settings. An appendix extends the analysis to seven model-and-SAE pipelines, including GPT-2 Small, Pythia-70M, and smaller Gemma-3 variants, for depth and simple-copying studies.

Result 1: learned coordinates are more additive than random ones

To check that additivity is a property of the SAE and not an artifact, the authors compare learned SAE coordinates with sum-preserving random rotations of the same subspace. The learned coordinates had lower first-order residuals in all 18 settings. That is a point in favor of SAEs: their directions carry structure that a random basis for the same space does not.

Result 2: pairs still matter

Pairwise models improved held-out coalition ranking in all 18 settings. According to the full text, average rank fidelity moved from 0.9782 to 0.9959 and selection regret fell from 0.0272 to 0.0051. The pairwise models also beat parameter-matched models that used randomly chosen third-order terms, so the gain is not simply from adding more parameters. Interactions between specific pairs of features carry real signal.

Result 3: steering is where it shows

In a combined steering experiment on MMLU-Pro with Llama-3.1 8B (1,400 questions, 16 features, fixed published coefficients), pairwise modeling raised rank fidelity from 0.5874 to 0.7790 and cut ordering error from 0.1444 to 0.0606. The residual estimate fell from 0.6424 to 0.3891. When many features are pushed at once, which is what real steering does, the independent-effects assumption is much shakier than in the small copying and agreement tasks.

A steering wheel turning an arrow inside a box, showing combined SAE feature steering on MMLU-Pro

Result 4: SAR is not just reconstruction quality

In a matched comparison of six SAEs, the authors found that SAR and reconstruction quality captured different variation in behavioral fidelity. Two SAEs that rebuild activations equally well can differ in how compositional their features are. That suggests SAR could serve as an additional yardstick when comparing SAEs, beyond the usual reconstruction-versus-sparsity plot.

What the paper does not claim

The limits are worth reading closely, and several come from the paper's own framing:

• Results are conditional. They depend on the selected features, the behavioral readout, and the intervention distribution. They are not a global property of an SAE.
• The second-order residual is imprecise. Only the rank and ordering conclusions are stable across data splits; do not read the absolute value as exact.
• Narrow main setting. The main analysis uses 16 features per prompt. Sensitivity to that number is shown for one setting only.
• Steering is one model. The steering test uses Llama-3.1 8B with fixed published coefficients.
• Some held-out residuals are negative in certain steering domains. The authors report them without clipping.

The authors' own conclusion is balanced: SAE coordinates support "comparatively compositional" behavioral descriptions, but faithful joint explanations and interventions need interactions beyond individual feature effects.

How to try it

The code repository is Apache-2.0 licensed. Its estimator, sar.py, depends only on NumPy. The GPT-2 Small demo adds PyTorch, TransformerLens, SAE Lens and Matplotlib, and takes about a minute on a GPU or five minutes on a CPU. The README describes the steps:

git clone https://github.com/OpenSAE/SAE-Behavioral-Compositionality
cd SAE-Behavioral-Compositionality
pip install -r requirements.txt
python -m pytest
python demo/demo.py --layers 4 6 8 10

To use your own model, SAE or behavior, you replace the top_contributions and behavior functions in the demo. The README notes its clone URL points to a different GitHub account than the OpenSAE page, so check the source you use. The repo had only 3 stars when we checked, so expect rough edges and treat it as a research release.

Why this matters for people who build with models

Most readers will never fit a pairwise model over SAE coalitions, but the finding changes how you should read interpretability claims:

1. Per-feature stories are a first approximation. If a post says "turning off feature X removes behavior Y," that is evidence about X in context, and the effect of removing two features together may not be the sum.
2. Steering many features at once needs testing. The steering numbers show the largest gap between additive and pairwise predictions. If you steer with a bundle of features, validate the bundle.
3. Safety monitors that use features inherit the assumption. Approaches like Goodfire's activation probes for rogue agents and Anthropic's J-lens work rely on internal signals being readable. Knowing where additivity breaks tells you where a monitor might miss a combination.
4. A cheap diagnostic now exists. SAR is a way to ask, for your own model and task, how much of the behavior is explained by independent features.

For a grounding in why teams invest in this at all, see our piece on interpretability and monitoring for teams. The term itself is defined in the explainx.ai dictionary entry for sparse autoencoder.

Comparing it with other interpretability work

SAR sits in a different slot from the big lab releases. Anthropic's natural language autoencoders aim to translate activations into readable text. Goodfire's work argues that individual units are already interpretable. SAR does neither: it is a measurement tool that asks how far the shared assumption behind many of these methods holds. That makes it complementary and easy to apply to the others' outputs, though the paper itself only evaluates SAEs on the six listed behaviors.

What to watch next

• Whether the authors or others test larger models and open-ended behaviors beyond the six tasks.
• Whether SAE training methods start to optimize for low SAR.
• Whether interpretability groups report pairwise corrections in their steering and monitoring results.
• Independent replication: the code and the published SAEs (Gemma Scope 2, Llama Scope) make this feasible for any lab with a GPU.

Related reading

• Goodfire: neural networks are already clean
• Goodfire inside-out monitors and activation probes
• Anthropic natural language autoencoders
• What are NLAs?
• What is the J-lens?
• Interpretability and monitoring for teams

Paper details are from the arXiv listing and full text as of October 17, 2026. Numbers are the authors' reported results and have not been independently replicated.