A new open-source engine called Janas claims you no longer need to fit a mixture-of-experts model in memory to run it. Janas, written in plain C by Maurizio "camauri" Cammalleri, keeps the weights on an NVMe SSD and pulls in only the experts each token actually uses. The README says an 80-billion-parameter model "runs, and answers, on a laptop with 32 GB of RAM." It landed on Hacker News on October 13, 2026, and the numbers are interesting enough, and the caveats large enough, that it deserves a careful read before anyone installs it.
If you follow local AI, this is the latest entry in a growing family of "stream the experts from disk" projects. We have covered several: Colibrì running GLM-5.2 in 25 GB of RAM, TurboFieldfare running Gemma 4 26B in about 2 GB, and Edge0's 35B model in 2.5 GB. Janas differs in being a general engine with chat, an MCP server and an OpenAI-style HTTP API, and in publishing head-to-head numbers against llama.cpp.
TL;DR: the questions people ask first
| Question | Short answer |
|---|---|
| What is it? | A C inference engine for MoE models that streams experts from NVMe into a RAM cache |
| Headline claim | Qwen3-Next-80B-A3B (48.4 GB file) answers on a 32 GB laptop, 37 tok/s writing with drafts |
| Which models? | Qwen3 (dense and MoE), Qwen3.5, Qwen3.6, Qwen3-Next-80B-A3B, Qwen3-Coder-Next, Gemma 4 variants |
| Which platforms? | Linux x86-64 and Windows 10/11 x86-64, optional Vulkan GPU |
| Is it free? | Yes, GPL-3.0 or later. Model files keep their original licenses |
| Is it stable? | No. The author calls it "an experiment, not a product" |
| Does it need Python? | No. C with no dependencies beyond the C library |
| Where do I get models? | .jns files on Hugging Face under prabanta-dev, or convert GGUF with the bundled tool |
How Janas runs a model bigger than your RAM
A mixture-of-experts model splits each feed-forward layer into many "experts" and routes every token through only a few of them. Qwen3-Next-80B-A3B has 80 billion parameters in total but activates roughly 3 billion per token, which is what the "A3B" in the name means. Most of the file therefore sits idle on any given step.
Janas exploits that. According to its README, the model weights stay on disk and the experts a token needs are read from an NVMe SSD. Experts that are needed repeatedly stay in an in-memory cache that fills whatever free memory is available when a model opens. Reads use O_DIRECT, which bypasses the operating system page cache so the engine controls what stays in RAM, and it picks thread counts and GPU use from measurements it takes on the machine.
The result is a trade. You swap a hard requirement (the whole model must fit in RAM) for a soft one (the SSD must be fast and the hot experts must fit in cache). If routing keeps hitting the same experts, the cache absorbs most reads and speed holds up. If it wanders, the disk becomes the bottleneck.
A ball routed through one of several chutes, illustrating how an MoE model activates only a few experts per token
Speculative decoding is built in
Janas also supports "drafts": a small model proposes several upcoming tokens and the large model verifies them in one pass. On the 80B model the author reports 37 tokens per second writing with drafts and 24 without. The README is candid that draft gains depend on the text, and that on Italian prose they were small. Treat the 37 figure as English-friendly best case.
What the benchmarks show (and what they do not)
All numbers below are the author's own, measured on one machine: Debian 13, Intel Core Ultra 9 185H with integrated Arc graphics, 32 GB DDR5 and a 1 TB NVMe SSD. The main comparisons were run October 9 to 11, 2026 against a llama.cpp build dated September 18, 2026.
| Model | Janas reading / writing (tok/s) | llama.cpp reading / writing (tok/s) |
|---|---|---|
| Qwen3-Next-80B-A3B (48.4 GB) | 62 / 24 (37 with drafts) | 16 / 2.8 |
| Qwen3-30B-A3B | 135 / 32 | 122 / 15 |
| Qwen3-4B | 193 / 27 | 164 / 18 |
| Qwen3-0.6B | 939 / 130 | 670 / 45 |
The README reads this as Janas ahead on every row. Our reading is more cautious. The 80B row is the one that matters, because it is the only case where the model genuinely exceeds RAM, and there the gap is large: 62 versus 16 tokens per second on prompt reading and roughly 9 to 13 times faster on writing. That is expected, since a general-purpose engine not tuned for expert streaming will thrash.
On models that fit in memory the gaps are small for reading and bigger for writing, and the README adds an exception: on the 3-bit Qwen3-30B (UD-Q3_K_XL) Janas reads slower than llama.cpp (100 versus 112 tokens per second) but writes about three times as fast. The author also says some llama.cpp runs needed special build flags and some Janas figures are single rounds. A single machine, single rounds and a self-run comparison are exactly the conditions under which numbers flatter the author's project. They are a reason to run janas-bench on your own hardware, not a verdict.
Other figures from the README: draft speeds in chat of 67 tokens per second on Qwen3.5-2B, 24 on Qwen3.5-9B, 34 on Qwen3.6-35B-A3B, 52 on Gemma-4-E2B and 26 on Gemma-4-26B-A4B. On Windows 11 (measured October 12, 2026), Qwen3-4B writes at 25.6 tokens per second and Qwen3-30B-A3B at 34.2. With other applications open, one model kept about three quarters of its speed in the author's test.
A short fast arrow ahead of a longer checking arrow, illustrating speculative decoding with draft tokens
How to try Janas without hurting your machine
Prebuilt binaries exist for Linux and Windows. On Linux, per the README:
curl -LO https://github.com/prabanta-dev/janas/releases/latest/download/janas-linux-x86_64.tar.gz
tar xf janas-linux-x86_64.tar.gz && cd janas
./janas-chat
On Windows, download janas-windows-x86_64.zip, unpack it and run .\janas-chat from the janas folder. To build from source you need build-essential, plus libvulkan-dev glslang-tools vulkan-tools if you want the GPU path, then ./build.sh.
Models are published as .jns files on Hugging Face under prabanta-dev, and a GGUF converter is bundled if you already have models for llama.cpp. Given the stability warnings, three precautions are sensible:
- Cap the cache with
--cache <GiB>so the engine cannot swallow all free memory and push your desktop into swap. - Set
JANAS_GPU=0first. The author reports a GPU run that triggered a driver reset and took down the desktop. - Run
janas-benchbefore trusting any speed claim, and keepCtrl-C(stop a reply) andCtrl-D(exit chat) in mind.
Janas writes only to ~/.cache/janas and to any conversion output location you choose.
More than a chat binary
The repository lists several components beyond the engine: janas-chat (chat with drafts and tools), janas-mcp (an MCP server), janas-server (an OpenAI-style HTTP API with code completion for VSCodium and VS Code), janas-bench, and service tools for flights, earthquakes, weather, maps, prices, GitHub, git and Wikipedia. If you are new to the protocol, our MCP explainer covers what an MCP server gives a local model. The OpenAI-compatible server is the more practical hook: existing editors and agent harnesses can point at it, which is how most people use open models locally with coding agents.
Limitations and open questions
- One test machine. The README lists AMD CPUs, CPUs without AVX-VNNI, discrete GPUs, slower disks and other memory sizes as untested. A SATA SSD would likely change the picture entirely.
- x86-64 only. There is no macOS or ARM build listed, so Apple Silicon users should look at TurboFieldfare or llama.cpp instead.
- Early software. The API and CLI change without notice; only the
.jnsformat (version 2) is promised to stay readable by later versions. The repo had 48 stars and 54 commits when we read it. - Licensing. The engine is GPL-3.0 or later, which matters if you want to embed it in a closed product. Converted
.jnsfiles keep their original model licenses. - SSD wear and heat. The README does not discuss it. Sustained random reads are gentler on flash than writes, but we have no data and would not run it unattended for days.
What developers are saying
The Hacker News thread was small (about 19 points and 7 comments when we read it), but it surfaced two themes. One commenter, 3eb7988a1663, argued that 62 tokens per second is plenty for someone who wants a local model and is not running a swarm of agents. Another, KerrAvon, noted there are now many stream-experts-from-SSD projects and said what the field needs is a guide to all of them, or a merge of the best ideas. A third, InsidiousWombat, speculated about small accelerators integrated with NVMe drives. Those are individual opinions, not verified results; nobody in the thread reported independent benchmarks.
Why it matters for local AI
The pattern is the story. Dense models need their whole weight file in fast memory for every token. MoE models do not, and a growing number of open-weight releases are MoE, including Tencent's Hy3 and the Qwen3 line. Every project that turns "model size" from a RAM problem into an SSD-bandwidth problem pushes bigger models onto ordinary laptops. For a builder, the practical question is not whether Janas wins, but whether an 80B-class model at 24 to 37 tokens per second on a 32 GB machine is good enough for your coding or chat workload. If it is, the hardware bar for useful local AI has just dropped. If you are weighing the opposite approach, a big GPU, see running state-of-the-art LLMs on an RTX 6000 Pro.
Related reading
-
Qualcomm: 100B-parameter phones by 2028, claimed vs verified
-
Colibrì: GLM-5.2 on 25 GB RAM by streaming experts from disk
-
Source: Janas on GitHub and the Hacker News discussion
Specs, benchmark figures and repository details reflect the Janas README as read on October 13, 2026 and may change as the project evolves.