explainx / blog
Z.ai's GLM-5.2 is an open-weights 744B MoE model (40B active parameters, 1M context) that matches Claude Opus 4.8 and GPT-5.5 on reasoning benchmarks. Unsloth's dynamic GGUFs make it runnable on a 256GB unified-memory Mac or a machine with 245GB total RAM. This is the complete setup guide, hardware requirements, and quantization trade-offs.
Jun 23, 2026
The standard assumption for running a 70B model locally: you need 140GB of VRAM. AirLLM breaks that assumption by loading layers one at a time from disk, holding only one layer in GPU memory at any moment. 21K+ GitHub stars, three lines of code to start. Here is what it actually buys you and what it costs you.
Jul 10, 2026
JustVugg's colibrì streams GLM-5.2 routed experts from a 370 GB int4 container while keeping 9.9 GB dense weights in RAM — 453-point Show HN. explainx.ai maps the physics, honest tok/s, and when disk-streaming beats 256 GB Unsloth setups.
Jul 3, 2026
Pasting a YouTube link into ChatGPT reads the transcript, not the picture. Claude often rejects video files outright. Here is what actually works in 2026 — native multimodal APIs, local frame+transcript pipelines like claude-real-video, and transcript-first agents like video-use — with honest limits and cost math.
GLM-5.2 has 744 billion parameters. That sounds impossible to run locally.
Update — July 10, 2026: Colibrì streams GLM-5.2 MoE experts from a 370 GB int4 disk container with only ~9.9 GB dense weights in ~25 GB RAM — pure C, 0.05–1+ tok/s depending on NVMe. See explainx.ai's Colibrì guide for the low-RAM path vs this 256 GB Unsloth guide.
But it's a Mixture-of-Experts model — only 40 billion parameters are active at any given token. The other 704B are idle experts, waiting for the routing layer to call them. That distinction is what makes local inference possible.
Unsloth's dynamic GGUFs compress the model further. The 2-bit version fits in 239GB of combined RAM and VRAM. A 256GB unified-memory Mac can run it. A PC with 245GB of total memory can run it.
The benchmark position: On AIME 2026 (99.2), GPQA-Diamond (91.2), and SWE-bench Pro (62.1), GLM-5.2 sits in the same tier as Claude Opus 4.8, GPT-5.5, and Gemini 3.1 Pro. It's not close to them on every task — but on the tasks it's measured on, it's in the conversation. And it's open weights, locally runnable, free to use.
Z.ai (Zhipu AI, a Beijing-based research lab) built GLM-5.2 as their frontier open-weights model. Key specs:
| Property | Value |
|---|---|
| Total parameters | 744B |
| Active parameters | ~40B per token (MoE routing) |
| Context window | 1,048,576 tokens (1M) |
| Architecture | Mixture-of-Experts Transformer |
| Thinking modes | Non-thinking / High / Max |
| License | Open weights (check Z.ai license for commercial terms) |
The 1M context window is the other notable specification. Most frontier models cap at 128K–200K tokens. GLM-5.2 can process book-length inputs, entire codebases, or long document sets in a single context.
Unsloth's dynamic GGUFs are the accessible path to running GLM-5.2. "Dynamic" means different parts of the model are quantized to different bit depths based on how much information loss that layer can tolerate — preserving quality in sensitive layers while compressing aggressively elsewhere.
| Quantization | Disk/RAM required | Best for |
|---|---|---|
| 1-bit (UD-IQ1_S) | 223 GB | Tight memory budget; biggest quality trade-off |
| 2-bit (UD-IQ2_M) | 239 GB | Recommended — best accessibility/accuracy balance |
| 3-bit | 290–360 GB | Better quality if you have the memory |
| 4-bit | 372–475 GB | Near-lossless for most use cases |
| 5-bit | 570 GB | Practically lossless |
| 8-bit | 810 GB | Near full-precision |
For a 256GB Mac: the 2-bit quant (239GB) fits with a small buffer. The 1-bit quant (223GB) fits more comfortably. Both run — the 2-bit is recommended for practical accuracy.
For a PC setup: total memory = VRAM + system RAM. A machine with a 24GB GPU and 224GB of RAM can run the 2-bit quant by offloading layers to RAM. Unsloth Studio handles this automatically.
Unsloth ran KL Divergence analysis on the quantization tiers. The 2-bit GGUF achieves ~82% top-1 accuracy while being 84% smaller than the full 1.5TB model.
This number is widely misunderstood. 76–82% top-1 accuracy does not mean 18–24% of outputs are wrong.
The metric measures token-level distribution similarity across the full corpus, including high-frequency filler tokens where the model has multiple acceptable continuations. For a prompt like "Write a novel," the baseline might use "I" 100% of the time, but the quantized model might use "I" 76% of the time and "The" 24% of the time — both grammatically correct openings.
For practical use:
Unsloth Studio is a web UI that handles model download, VRAM/RAM offloading, and inference settings automatically.
Install:
Mac/Linux/WSL:
curl -fsSL https://unsloth.ai/install.sh | sh
Windows PowerShell:
irm https://unsloth.ai/install.ps1 | iex
Launch:
unsloth studio -H 0.0.0.0 -p 8888
Open http://127.0.0.1:8888 in a browser.
For HTTPS via Cloudflare tunnel (no SSL certificate setup required):
unsloth studio --secure
Find and download GLM-5.2:
Unsloth Studio automatically configures temperature (1.0) and top-p (0.95), handles VRAM/RAM offloading, and lets you toggle thinking modes via the UI.
Build llama.cpp first:
apt-get update
apt-get install pciutils build-essential cmake curl libcurl4-openssl-dev -y
git clone https://github.com/ggml-org/llama.cpp
cmake llama.cpp -B llama.cpp/build \
-DBUILD_SHARED_LIBS=OFF -DGGML_CUDA=ON
cmake --build llama.cpp/build --config Release -j --clean-first \
--target llama-cli llama-mtmd-cli llama-server llama-gguf-split
cp llama.cpp/build/bin/llama-* llama.cpp
Download the model manually (faster than letting llama.cpp download it):
pip install huggingface_hub
hf download unsloth/GLM-5.2-GGUF \
--local-dir unsloth/GLM-5.2-GGUF \
--include "*UD-IQ2_M*"
Run:
./llama.cpp/llama-cli \
--model unsloth/GLM-5.2-GGUF/UD-IQ2_M/GLM-5.2-UD-IQ2_M-00001-of-00006.gguf \
--temp 1.0 \
--top-p 0.95 \
--min-p 0.01
Disable thinking mode:
./llama.cpp/llama-cli \
--model path/to/model.gguf \
--temp 1.0 \
--reasoning off
Extended context via KV cache quantization:
To push toward the 1M context window, quantize the KV cache to reduce its memory footprint:
./llama.cpp/llama-cli \
--model path/to/model.gguf \
--temp 1.0 \
--cache-type-k q4_1 \
--cache-type-v q4_1
q4_1 is 5 bits per weight — extends context ~3.2x beyond default. For default f16 KV cache at 128K context, q4_1 extends to ~400K. Getting to the full 1M requires the most aggressive cache quantization.
GLM-5.2 has three thinking modes that trade speed for reasoning depth:
| Mode | Use When |
|---|---|
| Non-thinking | Fast responses, simple Q&A, summarization |
| High thinking | Moderate reasoning tasks, code review, analysis |
| Max thinking | AIME-level math, complex coding, extended reasoning |
For most tasks, start with High thinking. Max thinking is significantly slower but measurably better on tasks that require multi-step reasoning — the 97.1 AIME 2026 score (up from 94.3 base) comes from claim-level test-time scaling with Max thinking.
In Unsloth Studio: toggle via the UI dropdown.
In llama.cpp: --reasoning on (High) or specify via chat template kwargs.
From the Unsloth benchmarks against frontier closed models:
| Benchmark | GLM-5.2 | Claude Opus 4.8 | GPT-5.5 | Gemini 3.1 Pro |
|---|---|---|---|---|
| AIME 2026 | 99.2 | 95.7 | 98.3 | 98.2 |
| GPQA-Diamond | 91.2 | 93.6 | 93.6 | 94.3 |
| SWE-bench Pro | 62.1 | 69.2 | 58.6 | 54.2 |
| HLE | 40.5 | 49.8 | 41.4 | 45.0 |
| Terminal Bench 2.1 | 81.0 | 85.0 | 84.0 | 74.0 |
GLM-5.2 leads on AIME and SWE-bench Pro. It trails on HLE (hard science/analysis questions) and GPQA-Diamond (expert domain reasoning). The coding bench advantage (SWE-bench Pro) is the most practically relevant signal for developers.
Running GLM-5.2 locally means:
The limitation is hardware. If you don't have 245GB+ of total memory, the 2-bit quant doesn't fit. In that case, the smaller quantizations (4-bit for models in the 7B–70B range) via ollama or Unsloth Studio for smaller GLM variants are the practical path.