Drew Breunig has released nesformer, a project that bakes a working Nintendo Entertainment System CPU into the weights of a stock Qwen3.5 4B model. According to the GitHub repository, nobody trained those weights: a compiler computed them, and the model then predicts the processor's execution log one token at a time. It can run cartridges such as Super Mario Bros., very slowly. Simon Willison flagged the project on October 30, 2026.
It is a stunt, and the author describes it as built for fun. But the trick underneath it, using a language model's own attention state as exact computer memory, is a genuinely interesting fact about the hybrid architectures now shipping in open models. This post explains what was built, how it works, how to run it, where it breaks and what to make of it.
Nesformer at a glance
| Item | What the repository says |
|---|---|
| What it is | An NES CPU compiled into a stock Qwen3.5 model |
| Training | None. A compiler sets every weight |
| Machine emulated | The 6502-family core of the NES chip, 2 KB of work RAM, a cartridge program ROM |
| Memory | The model's Gated DeltaNet state; each row of a state matrix acts as a memory cell |
| Host program | Draws the picture, reads the controller, fires the vblank interrupt |
| Games | NROM (mapper 0), e.g. Super Mario Bros. and Donkey Kong |
| Speed | 1 to 6 seconds per frame; about 1,650 tokens per second on one CPU core |
| Real console need | About 700,000 tokens per second |
| License | MIT |
| Builds on | Qwenputer by Divyansh Agrawal |
What does nesformer actually do?
A normal language model predicts the next word. Nesformer predicts the next line of a CPU trace. Each instruction the emulated processor executes becomes a few generated tokens that say which memory row to write and what value to put there. Because the weights were built to compute exactly that function, the output is not a guess: it is the deterministic result of running the program.
A tiny host program handles what the model does not. It draws the screen from the memory contents, reads the controller state and triggers the vertical blank interrupt that the NES uses to time each frame. The repository lists a terminal display option, so you can watch frames render as text. The picture processor is approximate, so do not expect pixel-perfect output, and there is no sound.
The important point is what this is not. It is not a language model that learned to play Mario from pixels, the way a reinforcement learning agent would. It is also not a model calling an emulator as a tool. The CPU logic lives inside the forward pass, in the matrices of the network.
How does a language model become a CPU?
Nesformer stands on Qwenputer, published on October 22, 2026 by Divyansh Agrawal. Qwenputer compiled first a CHIP-8 machine and then a 32-bit RISC-V core into Qwen3.5 4B. Its write-up explains the key insight.
Qwen3-Next and Qwen3.5 use a hybrid design in which three out of every four attention layers use Gated DeltaNet, a linear recurrent network, instead of ordinary attention. Each head keeps a state matrix and updates it with the delta rule. Agrawal found that if you set the gate to 1 (nothing fades), the write strength to 1 (new values fully replace old ones) and use one-hot keys, every write overwrites exactly one row and every query reads one row back exactly. In other words, the layer becomes RAM.
That has a practical advantage over using ordinary attention as memory. A Gated DeltaNet head's state stays the same size no matter how long the run is, while an attention KV cache grows with every token. The idea that the delta rule acts as an associative memory is not new; Agrawal credits the 2021 "Linear Transformers Are Secretly Fast Weight Programmers" paper. The new part is making it exact and addressable inside a production model.
With RAM available, a compiler can take a CPU described as a circuit and turn each step into part of a transformer layer: attention heads that look things up, feed-forward units that do arithmetic, and DeltaNet heads that hold memory. In Qwenputer's case, the RISC-V core takes 11 layers in front of Qwen's 32, passes the official riscv-tests, matches the qemu emulator, and ran 18 million instructions free-running without an error, according to the author.
Nesformer applies the same recipe to a different processor and a different memory map, and adds the host program needed for a game console. According to Willison's post, Breunig used Claude Opus 5.5 to explore and implement the idea, and Opus checked the emulated processor against nestest, a CPU conformance suite that Willison attributes to Christopher Pow (the repository's credits list the nestest ROM under Kevin Horton).
How do I run it?
The repository's instructions are short:
- Unzip the code and the weights archive into the same folder and check the weights against the included SHA-256 file.
- Install Python 3.11 or newer with numpy, scipy, numba and pillow.
- Run a game with
python -m nesformer run ROM. Add--framesto limit the run,--buttonsfor a scripted controller sequence and--showfor terminal display.
You bring your own ROM files. Three test ROMs ship in the repo, which is also the sensible place to start: run one for a handful of frames before committing 25 minutes to a long Mario session.
Notice what you do not need: a GPU. The default engine is a custom CPU implementation at about 1,650 tokens per second on a single core. That is about eight times faster than the numpy path (roughly 210 tokens per second) and far faster than stock transformers (about 10). The stock path exists mostly to prove that the compiled weights really behave like a normal Qwen3.5 checkpoint.
How slow is it, really?
A frame takes one to six seconds. A real NES draws 60 frames a second and needs, by the repository's estimate, about 700,000 tokens per second to keep pace. Nesformer manages roughly 1,650, putting it between 150 and 290 times slower than the console depending on the scene. Running 600 frames, ten seconds of gameplay, can take around 25 minutes.
Nobody should read that as a failure. A 4-billion-parameter model that executes an entire 8-bit console exactly, without a single learned parameter, is the point. It also shows the cost of doing computation through generation. Each emulated instruction costs several tokens, so even a very modest 1980s processor swamps a modern language model's decode speed.
What are the limits?
The repository is explicit about what is missing:
- Mappers. Only NROM (mapper 0) works. Bank-switched cartridges using MMC1 or MMC3 are out, which excludes many well-known games.
- Opcodes. Unofficial 6502 opcodes are not implemented.
- Interrupts and sound. IRQs and audio are missing.
- Graphics. The picture processor is approximate, so some effects will look wrong.
- Speed. As above, minutes per second of gameplay.
Also remember the stats: the repository had about 130 stars and 5 forks when we looked, so this is a young release. We have not run it ourselves, and everything about speed and compatibility here is the author's reporting.
Why does this matter beyond the novelty?
It shows what hybrid attention models can be. More open model families are moving to a mix of linear-recurrent and full-attention layers because the recurrent state is cheaper at long context. We have covered other Qwen3.5-based work, such as the Qwen3.5 4B 3D coder preview and the two-neuron steering experiments on Qwen3.5 4B. Nesformer shows the state matrices can be driven as exact memory, which is a different relationship from the fuzzy learned memory most people picture.
It blurs the line between model and tool. Agents normally call out to code interpreters. Qwenputer's author describes a model that hands arithmetic to a CPU in its own weights, with no external tool call. Whether that is ever practical is open, but it changes how you might think about small models doing exact work that normally needs code.
It raises safety questions. Agrawal's Qwenputer page links two follow-ups. One argues hybrid models could learn to hide reasoning in their recurrent state and evade chain-of-thought monitoring. The other argues compiled weights could serve as built-in monitoring tools the model cannot tamper with. We have not reviewed either follow-up in depth, so treat these as the author's arguments rather than established results. They are still a reminder that state inside a recurrent layer is harder to inspect than text.
It is a good "what is a transformer" lesson. Anyone learning how these networks work gets a concrete example that attention heads can do lookups, feed-forward layers can do logic, and the whole thing can be constructed by hand rather than discovered by gradient descent.
Is this the same as an emulator that runs on the GPU?
No. Emulators such as the ones used by AnyPS5 to relink games natively are written to run console code fast. Nesformer deliberately runs it through the slowest possible route to prove a point about model weights. If you want to play Mario, use a regular emulator.
What we verified and what we did not
Verified from the sources: the repository describes an NES CPU compiled into a stock Qwen3.5 model, untrained, using Gated DeltaNet state as memory; the run instructions, the speed figures, the supported cartridge types, the limits and the MIT license all come from the repository page. The Qwenputer mechanism comes from its author's own write-up. Willison's post confirms the Qwen3.5-4B base, the Claude Opus assistance and the nestest verification (the nestest author credit differs between his post and the repository).
Not verified: we did not run the code, check the weights checksum or reproduce the speed numbers. We did not independently check the claim of 18 million error-free RISC-V instructions. We also did not find any independent test of nesformer yet, given how new it is. If you try it, check the SHA-256 file first and run it in a throwaway environment as you would with any downloaded archive.