kimi-k3-in-c: A 2.78-Trillion-Parameter Model on One CPU in 8 GB of RAM, at 26 Seconds a Token

A GitHub repository called kimi-k3-in-c is trending this week with a bold claim: the full 2.78-trillion-parameter Kimi K3 model, running on one CPU in 8.24 GB of RAM, with no GPU, no BLAS and no ML framework. The claim holds up as described by the author, with a large catch. The model streams from disk, so an 8 GB laptop needs about 26.5 seconds per token. That is a proof that the model fits, not a way to chat with it.

This post explains how the trick works, what the README measures, how to try the free part, and when to use a hosted API instead. All speeds below come from the author's README and its measurement files. We did not rerun the 1.56 TB download.

Quick answers

What kimi-k3-in-c actually does

Kimi K3 is Moonshot AI's open-weight flagship, covered in our Kimi K3 open-weights explainer. It is a mixture-of-experts model: for each token, a router picks a small set of experts out of a very large pool, so most of the parameters sit idle on any one step. The README says the checkpoint is 1.56 TB, of which 1.45 TB are routed experts stored in packed 4-bit form.

The engine exploits that sparsity. The README describes four decisions about where bytes live:

• The dense trunk stays in memory to whatever depth you choose and the rest of it is streamed from disk.
• The routed experts are never resident in full. They are multiplied straight out of their packed 4-bit form as they are read.
• An expert cache keeps the most requested experts in RAM when you give it a budget.
• A prefetcher pulls experts off disk moments before they are needed.

The consequence, in the author's words, is that "the same model runs in 8 GB and in 224 GB and produces byte-identical output at every budget between." More memory only buys speed.

The measured speed ladder

The README's headline table was measured on one machine with 124 cores and a fast NVMe drive:

The author notes that the first three rows still read from disk every step, so a slower drive makes them slower, and that network volumes run several times slower than local NVMe. The README also lists presets with peak memory: laptop at 8.2 GB, desktop at 31.9 GB, workstation at 95.5 GB, server at about 128 GB, and max at about 224 GB. The max preset is not faster than server in these measurements.

A concrete run in the README: eight generated tokens for the prompt "The capital of France is" took 261.5 seconds, 32.69 s per token on average, with a peak resident set of 8.24 GB. A second run on the server preset generated 28 tokens of a Python function at 10.69 s per token with 127.92 GB peak. The first token takes far longer than the steady rate, because every pinned layer loads from disk once; at the server preset that is about 108 GB.

What the speeds mean in practice

Take a 500-token answer. At 5.6 s per token that is about 47 minutes. At 26.5 s per token it is about 3.7 hours. Reasoning models that think for thousands of tokens would run for days. The project is honest about this: it calls the 8 GB result "slow, and answering correctly."

For comparison, our post on the Kimi K3 842 tokens per second claim covers the opposite end of the range, with GPU clusters. The gap, thousands of times, is the price of replacing accelerators with a disk.

Why no BLAS and no framework

The README's answer is reproducibility. Every matrix multiply has to give the same bits on every machine, so the test suite can compare against a PyTorch reference exactly rather than to a tolerance. A BLAS library picks its own blocking and reduction order, which can differ between OpenBLAS, MKL, Accelerate and thread counts. The engine's own kernels pin the summation order, and the AVX2 and NEON paths reproduce the scalar reduction exactly. The author adds that the matrix math is not the bottleneck: a run is limited by how fast the trunk and experts come off disk.

The test suite is the most approachable part. make test needs no checkpoint, no network and no Python, and the README says it takes under a minute. It runs three gates over a 13-layer model built with the same tensor graph as the released model: teacher forcing, greedy decoding, and incremental decoding with the KV cache and carried linear-attention state. The expected verdict is "ENGINE MATCHES THE REFERENCE EXACTLY".

How to try it

The free part, no 1.7 TB needed:

git clone https://github.com/FareedKhan-dev/kimi-k3-in-c.git
cd kimi-k3-in-c
make -j
make test
./scripts/k3-doctor.sh

The doctor script checks your toolchain, sizes your RAM to a preset, measures your disk the way the engine reads it, and prints the exact command to run next. The full path has six steps: clone, check the machine, build, verify before downloading, fetch the checkpoint, pack the trunk, then run. Only the download is slow. The README's laptop-class command looks like this:

./bin/k3 ~/k3model --trunk ~/k3trunk --preset laptop \
         --tok ~/k3model --prompt "The capital of France is" --gen 8 --incremental

Requirements per the README: GCC 9 or Clang 10 or newer, GNU make or CMake, Python 3.9 or newer for the helper tools, Linux x86-64 as the reference platform, with macOS arm64 and Windows x86-64 builds also reported to pass every gate. Put the packed trunk on the fastest local disk you have.

If you only want to experiment with the pipeline, the --layers N option runs against a partial shard set. The README says one layer is about 7 GB and eight layers about 125 GB, but a layer prefix is not the model and does not produce its output.

Limits and gotchas

• Disk is the real gate. About 1.7 TB free, and the speed depends on NVMe throughput. The README measured 5,908 MB/s reads in one workstation run.
• Context costs memory. The KV cache costs about 2.37 MB per position regardless of budget, and the engine refuses a plan above 95 percent of available memory.
• First token is slow. All pinned layers load from disk once.
• Non-ASCII prompts should go through --prompt-file, because the shell may re-encode arguments.
• One-person project. The README credits contributors for macOS, Windows and NEON ports, chat mode and faster MXFP4 kernels, and the author states they are open to AI research roles. The numbers are self-reported. Check the docs/data folder, which the README says holds every measurement.
• Model license is separate. Review Moonshot's terms for Kimi K3 before using outputs commercially.

What developers are saying

The Hacker News thread was small when we checked: two points and one comment. The one comment, from user poxlr, called it an "absolute beast of a project" and said they were not used to seeing a complete research paper in the README of a GitHub project. That matches the repo: the README is organized as Part I for getting started and Part II for how it works, with diagrams. It is one reader's reaction and not a review of the claims.

Where this fits among local-inference projects

Local-first engines for giant models are a growing category. Our coverage includes antirez's DwarfStar DS4 for a different local approach, and Apple's RDMA over Thunderbolt 5 route for clustering Macs to run large open models. Those paths buy speed with memory or more machines. kimi-k3-in-c does the reverse: it trades speed for the ability to run at all. If you want a primer on why parameter count and memory matter so much, see our guide to LLM parameters and model sizes. For the hardware angle, the AI chip architecture guide shows why 4-bit weights and sparse routing are what make this feasible.

Which route should you pick?

Bottom line

kimi-k3-in-c shows that a 2.78-trillion-parameter model can be run on commodity hardware by treating disk as the main memory and keeping the math deterministic. The reported 8.24 GB peak and 26.5 s per token are self-measured and plausible given the design, and the exact-match test suite is the strongest evidence in the repo. For readers, the value is educational and architectural, not practical throughput.

Related reading

• Kimi K3 open weights: the 2.8-trillion-parameter release
• Kimi K3 842 tokens per second: claimed vs verified
• Kimi K3.1 Pro open weights, benchmarks and price
• How Cerebras runs a trillion-parameter model
• DwarfStar DS4: local inference from antirez
• Together Compiler: agents tune inference
• Sources: kimi-k3-in-c on GitHub, Moonshot's Kimi K3 post, Hacker News thread

Speeds and requirements are from the repository README as of November 4, 2026 and may change with later versions.