Colibrì: Run GLM-5.2 on 25 GB RAM by Streaming MoE Experts From Disk

On July 10, 2026, Colibrì hit 453 points on Hacker News — Show HN: Getting GLM 5.2 running on my slow computer. Author vforno (JustVugg) set a deliberately modest goal: make GLM-5.2 — Z.ai's 744B MoE frontier open model — answer correctly on a 12-core laptop with 25 GB RAM, even if speed is measured in minutes per paragraph, not tokens per second.

The result is not a faster llama.cpp fork. It is a disk-streaming MoE runtime: ~9.9 GB dense weights stay in RAM; ~370 GB of routed experts live on NVMe and load on demand. Pure C, zero runtime deps, token-exact against a transformers oracle.

TL;DR — what people are asking

The idea — why only ~11 GB changes per token

GLM-5.2 activates ~40B parameters per token, but MoE routing means only ~11 GB of expert weights are new each decode step — the rest is shared dense stack (attention, embeddings, shared experts).

Colibrì splits the model physically:

The engine (c/glm.c, ~1,300 lines) implements faithful glm_moe_dsa forward — validated 32/32 teacher-forcing and 20/20 greedy against a tiny-random oracle with the real architecture.

MLA attention with compressed KV: 576 floats/token vs 32,768 naive — critical for long sessions on RAM-starved boxes (GLM-5.2 specs).

What's actually implemented (not vapor)

From the README as of the HN launch:

• DeepSeek-V3-style router — sigmoid, noaux_tc, routed_scaling_factor, shared expert, first-3-dense layers
• MTP speculative decoding — GLM-5.2's layer-78 head drafts; main model verifies in one batched forward
• MTP head must be int8 — at int4, draft acceptance collapses to 0–4%; int8 hits 39–59% acceptance, 2.2–2.8 tokens/forward (community #8)
• Honest MTP caveat — cold cache adds expert loads per verified draft; speculation can lose until cache warms; DRAFT=0 disables it
• Integer-dot kernels — Q8_0-style activations, AVX2; int8 matmuls 1.4–2.5× faster than f32 paths measured
• MLA weight absorption on decode — DeepSeek trick; exact match with absorption forced
• Async expert readahead — WILLNEED while multiplying current block
• Learning cache — .coli_usage records routed experts; startup auto-pins hottest in spare RAM — "gets faster the more you use it"
• Batch-union MoE on prefill — each unique expert read once per batch
• BPE tokenizer in C — 320k merges, no Python at runtime
• DSA sparse attention — in progress (indexer weights ~108 GB extraction downloading)

Honest speed numbers — read before you clone

Colibrì is explicit: this is not fast. It is a 744B-class model talking on hummingbird rations.

Author dev box (WSL2, 12 cores, 25 GB RAM, VHDX NVMe ~1 GB/s random)

Community benchmarks (stock setup.sh, greedy, --ngen 32)

HN consensus: 0.05–0.1 tok/s is a proof-of-life demo, not a daily driver. ~1 tok/s on Apple Silicon with fast SSD starts to resemble "walk away overnight" agent workflows walrus01 noted on HN. 5–15 tok/s remains a back-of-envelope target for 128–256 GB RAM + many cores + warm pin — not yet measured at scale.

SSD, swap, and the corrected wear story

Early README drafts worried about SSD wear from page cache writes. The July 10 README clarifies:

• Expert streaming = read-only — does not wear NAND meaningfully
• Swap under memory pressure does write — colibrì sizes expert cache from MemAvailable to stay out of swap
• Thermals — hours at full read duty cycle heat cheaper drives; monitor health

For soldered-SSD laptops, HN commenters treat this as an experiment, not default daily tooling — external NVMe or a desktop with replaceable storage is saner for multi-hour runs.

Quick start — commands that work

cd c
./setup.sh                      # gcc + OpenMP, self-test (expects 32/32)

# One-time convert (needs python: torch safetensors huggingface_hub numpy)
./coli convert --model /nvme/glm52_i4     # ~400 GB free, resumable

# Or skip convert — download prebuilt:
# https://huggingface.co/jlnsrk/GLM-5.2-colibri-int4

COLI_MODEL=/nvme/glm52_i4 ./coli chat

Useful knobs: --temp 0.7 (int4-tuned), --topp 0.7 (adaptive expert top-p, ~1.6× speedup in #2 benchmark), THINK=1 reasoning block, DRAFT=n MTP depth, PIN=stats.txt PIN_GB=20 hot-expert pinning.

Quality sweep (needs fast disk):

./coli bench                    # hellaswag, arc_challenge, mmlu — 40 each
./coli bench mmlu --limit 200

The project has not published int4 accuracy loss yet — the harness exists; a full run on the dev box disk is ~a day. That benchmark is the highest-value community contribution right now.

Colibrì vs the rest of the GLM-5.2 local stack

Colibrì does not replace Perplexity's GLM orchestrator or Ollama's $88M hybrid cloud bet. It extends the sovereign extreme: frontier open weights when you refuse to rent GPUs and refuse to buy 256 GB RAM.

Hacker News themes worth keeping

Ticket / async interface — several commenters argued chat UI is wrong for 0.1 tok/s models; queue a task, check back in hours (loop engineering mindset without cloud).

RAM is the hidden cap on 24 GB machines — even with 2.7× faster disk, expert cache auto-caps to 2 slots/layer, so decode stays cold. More RAM → more pin → fewer reads.

Parallel NVMe / RAID0 — untested multiplier; author invites benchmarks.

OpenCode integration — in-flight PR per OP; until merge, wire agents manually or wait.

Related experiments — flash-moe reported 5+ tok/s on M3 Max (different approach); hypura for general SSD streaming — same design space, different targets.

Who should try Colibrì — and who should not

Try it if:

• You have ≥25 GB RAM, ≥400 GB NVMe, Linux or WSL2, AVX2
• You want to validate int4 GLM-5.2 quality on hardware faster than the author's laptop
• You are researching MoE disk streaming for local LLM economics

Skip it if:

• You need interactive coding agents today — use API, Cline, or 256 GB Unsloth
• You have only soldered 256 GB Mac SSD and fear thermals — treat as read-only experiment
• 0.07 tok/s would frustrate you — cloud GLM-5.2 harness is cheaper time-wise

Star the repo, run iobench, open an issue with numbers — the README's scaling table is explicitly waiting for datapoints from better machines.

Related on explainx.ai

• 28.9M LLM on $8 ESP32 — flash PLE table
• GLM-5.2 MIT open source and Code Arena adoption
• Run GLM-5.2 locally with Unsloth — 256 GB path
• What is llama.cpp? GGUF and local APIs
• How to run GLM 5.2 in agent harnesses
• Fix local LLM looping — samplers and sandboxing
• Ollama $88M — open models runtime
• Perplexity GLM 5.2 orchestrator — hosted contrast
• Qwen 3.6 27B local — when dense beats MoE

Official: Colibrì on GitHub · GLM-5.2-colibri-int4 weights · Show HN thread

Benchmarks and README details reflect the July 10, 2026 colibrì launch and community issues #2, #4, #5, #8. Tok/s varies enormously by NVMe, RAM pin budget, and MTP settings — run coli bench and iobench on your hardware before assuming interactive speed.