Nemotron 3.5 Bolt 74B-A9B: How Nvidia Fit a 74B Model in 24GB

Nvidia has now published the technical details behind Nemotron 3.5 Bolt 74B-A9B, the open model it promoted at the RTX Spark laptop launch. The research page dated October 15, 2026 describes a mixture-of-experts hybrid Mamba-Transformer with 74B total and 9B active parameters, built to "run everyday agentic tasks locally on NVIDIA RTX/DGX Spark," with three stated goals: fit in 24GB, decode at 80+ tokens per second, and maximize intelligence per GB.

We already covered the announcement in our RTX Spark software roundup. This post is the deeper look: how the model was made, what Nvidia's own benchmark table actually says, and how to try it.

TL;DR: Nemotron 3.5 Bolt in one table

Every number here is Nvidia's own measurement on its own hardware. Independent reviews had not been published at the time of writing.

How was a 120B model squeezed into 24GB?

Bolt is a pruned and quantized descendant of Nemotron 3.5 Super VL. Three steps do the work.

A green ball routed through one of three chutes, illustrating mixture-of-experts routing in Nemotron 3.5 Bolt

1. Star Elastic compression. Nvidia describes this as a router-guided architecture search and compression framework. A budget-conditioned router chooses how many routed experts to keep and how wide each MoE layer's intermediate dimension should be, layer by layer. For Bolt the target top-K is 9 experts, down from 22 in the parent, and the search was aimed at a footprint below 24GB after quantization. The method has its own arXiv paper.

2. Mixed-precision quantization with distillation. Nvidia uses quantization-aware distillation to recover accuracy and quantizes weights only, because on RTX and DGX Spark generation is mostly limited by weight bandwidth rather than compute. The layout:

The authors argue that a larger model at one to two bits beats a smaller model at higher precision for the same memory budget. That is the central bet of the release, and it is a claim about this family and this recipe, not a universal rule. The QAD method is described in a separate paper.

3. Post-training. Nvidia restores vision capability, then runs end-to-end reinforcement learning in simulated environments for agent skills, and a final RL step to improve reasoning paths, tool use and token efficiency.

What is the LiLiCorr drafter?

Speculative decoding lets a small drafter propose several tokens that the big model verifies in one pass. Nvidia's drafter, LiLiCorr, has 1.13B parameters excluding the shared target embeddings and LM head, and it scores token candidates jointly across the whole draft in a single lightweight pass. It drafts 7 tokens. In vLLM at concurrency 64 with an NVFP4 drafter, mean acceptance length is 4.68 under greedy decoding and 4.25 with probabilistic sampling. The method has an arXiv paper.

For readers who want a refresher, our post on llama.cpp decision-model support and the Llama Windows app cover the runtime side.

How fast is it, honestly?

Nvidia benchmarked llama.cpp on Windows 11 ARM64 on a Surface Laptop Ultra with 128GB and RTX Spark:

The headline is less flattering than the "84 tokens per second" framing suggests. Bolt is slower than both Qwen3.6 35B-A3B and Gemma 4 26B-A4B on throughput and on the agent replay time, and faster only than the dense Qwen3.8 27B. Nvidia's case is accuracy: the page says Bolt scores higher than Qwen3.6 35B-A3B and Gemma 4 26B-A4B on general agentic, web browsing and terminal benchmarks, and ranks #1 on PinchBench, #2 on WebArena and #2 on BrowseComp among models of similar memory footprint. The per-benchmark scores appear only in figures on the page, not in text, so we cannot reproduce a table of them here.

The trade is therefore speed for capability: a model with more total knowledge that still runs at an interactive pace, rather than the fastest model that fits.

Which model for which job?

This table uses only Nvidia's figures plus the design intent each model states. It is a guide to what to test first, not a ranking.

Treat the accuracy claims with care: they come from figures on Nvidia's page rather than a published table, and the comparison models were run with different speculative-decoding methods. The only way to know which model suits your workload is to run your own prompts on all of them.

Where does Bolt sit in the Nemotron 3.5 family?

The 3.5 generation now spans several sizes. Nemotron 3.5 Super VL is the 120B parent with 12B active parameters, aimed at multimodal open agents. Bolt is its on-device descendant. Nvidia's launch material also mentions a much smaller Bolt E4B 4GB for in-game characters, covered in the RTX Spark software post, and a multimodal reranker we covered in our Nemotron 3.5 Rerank 8B post. The pattern is a family where one large model is distilled and compressed to fit each hardware tier, rather than separate trainings from scratch.

That matters for builders because the same checkpoints and recipes can be adapted. The BF16 weights plus published methods give a path to compress your own fine-tune to a memory budget, assuming you have the compute for the search and distillation steps, which Nvidia does not claim are cheap.

What should you test first?

1. Tool-call reliability. Run twenty of your real tool-use prompts and count malformed calls. Heavy quantization often shows up here before it shows up in benchmarks.
2. Long-context behavior. Load a document near your working context length and ask for specific facts from the middle.
3. Speed with and without the drafter. Nvidia's 84 tokens per second includes speculative decoding. Measure your own with the drafter off to see the base rate.
4. Memory headroom. Watch total memory under a realistic context, not just at idle after load.

What checkpoints are available?

Nvidia lists five Hugging Face repos under the nvidia organization:

• BF16 weights for customization.
• 24GB checkpoint, the main local model.
• 24GB GGUF for llama.cpp.
• LiLiCorr drafter and its GGUF for speculative decoding.

A technical report PDF is also linked. The research page labels several checkpoints "Super VL 74B-A9B," which is inconsistent with the Bolt name, so confirm you have the right repo before downloading roughly 24GB.

How do I run it?

1. On an RTX Spark laptop: use llama.cpp with the 24GB GGUF. Add the LiLiCorr GGUF drafter if your build supports it. The Llama Windows tray app wraps this for non-terminal users.
2. On DGX Spark: use vLLM with the quantized checkpoint. Nvidia's acceptance-length numbers come from vLLM with an NVFP4 drafter.
3. Check memory first. 24GB is the weight footprint; the KV cache and your OS still need room, so a 32GB machine will be tight and 64GB or more is comfortable.

For hardware context, see our posts on the Surface RTX Spark dev box and RTX Spark power envelopes.

What are the caveats?

• Vendor-run benchmarks. The speed table is Nvidia's, on Nvidia's partner hardware, with each rival tested using different speculative methods (MTP, DFlash).
• 2-bit quality risk. Aggressive quantization can hurt long-context or rare-skill behavior even when headline benchmarks hold. Test on your own tasks.
• License unknown from the page. Do not assume permissive terms.
• Naming inconsistencies on the page suggest a quick release; expect corrections.

Who should care?

Anyone building local agents on a Windows on Arm or DGX Spark machine, and anyone tracking the open-model race. The 120B-to-74B-to-24GB chain shows a repeatable recipe: search the expert structure, quantize hard, distill to recover. If it holds up in independent tests, expect other labs to copy it. For the wider open-weights landscape, compare with Meta Muse Spark Open.

Related reading

• RTX Spark launch software: Nemotron 3.5 Bolt, PAIR 1.0, Llama Windows
• Nemotron 3.5 Super VL 120B-12B
• Llama Windows app
• llama.cpp decision-model support
• Surface RTX Spark dev box
• Meta Muse Spark Open

Sources: Nvidia Nemotron research page, Star Elastic, QAD and LiLiCorr papers linked above.

Specifications are as published by Nvidia on October 15, 2026 and may change.