Unsloth Shrinks MiniMax M3.1 Flash to 2-Bit GGUF: Runs on 150GB RAM

Unsloth has released 2-bit dynamic GGUF quantizations of MiniMax M3.1 Flash, the 428-billion-parameter open-weight agent model, shrinking it by 45 percent to 143.9GB so it can run on a machine with about 150GB of RAM. The accuracy figure that goes with it, 91.5 percent top-1 token agreement with the original, is Unsloth's own claim. The files are real and downloadable; the quality number is not independently verified yet.

This post sorts out what is confirmed from the Unsloth model card and the MiniMax weights page, what is claimed on X, and what independent evaluators found when they tested the model itself. For the launch, pricing and license, see our earlier write-up of MiniMax M3.1 Flash.

Quick facts: claimed vs. verified

What Unsloth actually released

The repo lists several quantization families under the Unsloth Dynamic naming: UD-IQ1_M, UD-IQ2_XXS, UD-IQ3_XXS, UD-IQ3_S, UD-Q2_K_XL, UD-Q3_K_XL, UD-Q4_K_XL and UD-Q8_K_XL. Each is split into four to seven shard files. There are also two vision projector files (mmproj-BF16.gguf and mmproj-F32.gguf) because M3.1 Flash is natively multimodal, an imatrix_unsloth.gguf calibration file, and a separate draft-head file for speculative decoding.

The headline claim comes from Unsloth's X account: "We made 2-bit GGUF retain 91.5% top 1 accuracy while being 45% smaller. Run on 150GB RAM." Co-founder Daniel Han added that the model ships with NVFP4 for the mixture-of-experts layers and BF16 or MXFP8 for the rest, and that the smallest UD-IQ1_M build keeps roughly 89.9 percent top-1.

Two caveats matter. First, "top-1 token accuracy" measures how often the quantized model picks the same next token as the original. It is a useful proxy for damage, but it is not the same as keeping a benchmark score such as Terminal-Bench or DeepSWE. Second, MiniMax's own card says its results are for the official weights and that "third-party quantizations may differ."

Why 150GB is the number to know

The original model is already stored in low precision. MiniMax used quantization-aware training for the routed experts and attention paths, which is why Unsloth's headroom for shrinking it further is smaller than with a BF16 release. A 45 percent cut that lands at 143.9GB means:

• A 128GB unified-memory machine cannot hold the 2-bit build without offloading to disk.
• A 192GB Mac Studio-class or a 256GB workstation can, with room for KV cache.
• A 150GB host fits the weights but leaves very little for long contexts, so do not expect the full 1M-token window.

If you are weighing hardware for this class of model, our guides to Mac Studio M5 Ultra for local AI and running large models on Apple silicon with MLX cover the memory side. For a comparable "huge model on a consumer machine" story, see Strata and Qwen3-8 Flash Next.

How to run it: llama.cpp with DSpark

Unsloth documents a speculative-decoding setup that pairs a main GGUF with MiniMax's draft head. From the model card, the example is:

./llama-server \
  -m MiniMax-M3.1-Flash-UD-Q4_K_XL-00001-of-00007.gguf \
  -md dspark-MiniMax-M3.1-Flash-Q8_0.gguf \
  --spec-type draft-dspark --spec-draft-n-max 1 --spec-draft-p-min 0 \
  -ngl 99 -ngld 99 -np 1 --jinja

Swap the first file for a 2-bit shard if memory is tight. Unsloth's notes, in its own words paraphrased:

• The draft head is about 2.1GB at Q8_0 and reuses the main model's embeddings and output head, so it only works with an M3.1 Flash GGUF from the same repo.
• You need a llama.cpp build that includes MiniMax-M3.1 support and the draft-dspark speculative type. An older build will not load it.
• Use a single server slot (-np 1); that is the configuration Unsloth tested.
• --spec-draft-n-max 1 gave the best speed in its tests, about 1.5x on code and math and smaller gains on chat. Larger values raise drafted tokens per step but lower acceptance.

MiniMax recommends temperature=1.0 and top_p=0.95, and the chat template accepts a reasoning_effort of low, medium, high, xhigh or max. For background on how llama.cpp tracks new architectures, see llama.cpp model support decisions and the Metal speculative decoding work on M3 Ultra.

The benchmark dispute you should know about

Running the model locally is only worth it if the model is as good as advertised. MiniMax's table on the model card lists M3.1 Flash at 38.1 on Terminal Bench 4.0, 73.7 on DeepSWE v1.1, 61.7 on OSWorld 2.0, 76.2 on BankerToolBench and 82.2 on CyberGym, next to DeepSeek V4.1 Flash, Kimi K3, GLM-5.3, GPT-5.6 Sol and Opus 5. The card's own table shows it behind DeepSeek V4.1 Flash-1017 on Terminal Bench (42.3) and DeepSWE (77.0), and well behind Opus 5 on Terminal Bench (51.8).

The outside number is lower. Vals AI measured 25.25 percent on Terminal-Bench 4.0 through the official MiniMax API, against MiniMax's 38.1, and ranked the model 27th of 58 on that benchmark. Vals also reported large gains over MiniMax M3 on Vibe Code Bench, Tax Agent Bench and Finance Agent. Read that as: stronger than its predecessor on several tasks, but below the vendor's table on long-horizon terminal work. Differences in harness and settings can explain part of such gaps. Our DeepSeek V4.1 Flash-1017 breakdown and the small-model comparison show how the same caveat applies across the Flash tier.

Another X commenter claimed the 3.1 release is M3 with sparse attention applied in three more layers, quantized, with DSpark added. The model card is consistent with that description: it lists sparse attention replacing the first three full-attention layers, quantization-aware training, and DSpark replacing the earlier MTP approach. Whether that deserves a new version number is a matter of opinion, not a fact to verify.

Which quant should you pick?

The repo's file names give a rough ladder. UD-IQ1_M is the smallest and the one Unsloth says keeps about 89.9 percent top-1 agreement; it is the choice when memory is the hard limit. UD-IQ2_XXS and UD-Q2_K_XL are the 2-bit tier the headline refers to, split across four shards. The three-bit builds (UD-IQ3_XXS, UD-IQ3_S, UD-Q3_K_XL) use five shards and need noticeably more RAM. UD-Q4_K_XL spans seven shards and is the build used in Unsloth's own server example. UD-Q8_K_XL is closest to the original and needs a workstation-class memory pool.

A practical rule: pick the largest build that leaves at least 10 to 15 percent of your memory free for the KV cache and the operating system, then lower the context length before you lower the quantization level. Agent workloads that re-read long tool outputs punish a too-small cache more than they punish one extra bit of weight precision. Unsloth's published top-1 numbers are per-build averages, so your own task may land above or below them.

One more reason to be careful with claims on this release: the base model already shows a gap between vendor and independent scores, and a quantized copy can only be as good as that base. If your use case is agentic coding, run the same small task suite against the hosted API and the local build before you rely on either.

What this means for what you build or pay

Self-hosting a 2-bit 428B model trades dollars for friction. The API costs $0.15 per million input tokens and $0.60 per million output tokens, per MiniMax, so you would need very heavy, private or offline use to justify buying 150GB-plus of fast memory. Local makes sense when:

1. Your code or data cannot leave the machine.
2. You run long agent loops where per-token billing adds up, and you accept slower tokens.
3. You want to experiment with the architecture, speculative decoding or fine-tuning.

It makes less sense if you need the full benchmark accuracy, since quantization loss compounds with a model that is already below its vendor table in outside tests. Test your own task on the 3-bit or 4-bit build first. Our local-agent coding stack on RTX Spark is a sensible hardware reference.

How to check the claims yourself

1. Download one build, for example UD-Q2_K_XL, and run your own coding or tool-use prompts at the same temperature as the API.
2. Compare against the hosted MiniMax API on the same ten tasks, tracking pass rate and tokens used.
3. Run a perplexity or KL-divergence check from llama.cpp's tooling against a higher-bit build, since that is closer to what Unsloth's top-1 agreement measures.
4. Watch the Unsloth repo's discussion tab (four open threads at the time of writing) for reported failures on specific llama.cpp builds. The repo showed about 11,700 downloads in its last month at the time of writing.
5. Check the license before shipping a commercial product. The MiniMax Community License is not MIT or Apache.

What to watch next

Expect independent perplexity and agent-benchmark runs of the individual quants, a mainline llama.cpp release that folds in MiniMax-M3.1 and DSpark support, and possibly smaller community builds. Until then, treat 91.5 percent as a promising signal about fidelity, not a guarantee of task performance.

Related reading

• MiniMax M3.1 Flash: pricing, benchmarks and weights
• DeepSeek V4.1 Flash-1017 open weights and benchmarks
• Small models compared: Haiku 5.5, GPT-6 Luna, Flash-tier rivals
• Strata: a 125B Qwen model at 100 tokens per second on a consumer GPU
• llama.cpp model support decisions
• Unsloth 3-bit GLM-5.3-Flash GGUFs on 128GB RAM

Details are accurate as of October 29, 2026. Model files, llama.cpp support and independent scores change quickly.