PrismML Bonsai 2 27B: 9x Smaller, 98.2% of Full-Precision Quality

PrismML released Ternary Bonsai 2 27B on September 17, 2026 — a compressed version of Qwen3.8 27B that fits in 5.9GB while retaining 98.2% of the full-precision model's aggregate benchmark performance, up from roughly 95% retention in the first Bonsai release two months prior. The pitch is that a 27B-class model can now run at meaningful throughput on a single consumer GPU or an Apple Silicon Mac, with PrismML reporting 143 tokens/second on an RTX 5090. A 244-point Hacker News thread pushed back on parts of the story — real-world throughput inconsistency, at least one clear looping failure, and friction from PrismML's proprietary llama.cpp fork — worth reading alongside the benchmark table before treating "near-lossless" as settled.

TL;DR

What actually changed from the first Bonsai

PrismML's original Bonsai 27B, which explainx.ai covered at launch in July 2026, proved a 27B-class model could run on a phone but retained only around 95% of its full-precision base model's aggregate score — a real gap for agentic and coding tasks, where PrismML itself notes small errors compound across many steps. Bonsai 2 rebuilds on Qwen3.8 27B rather than Qwen3.6, and PrismML reports the retention gap closing from roughly 95% to 98.2%, which it's now calling "practically lossless."

The compression method is ternary weights — each weight takes one of exactly three values, — combined with FP16 group-wise scaling, for an effective 1.76 bits per weight across the entire language model. PrismML states this is meaningfully more sophisticated than naive rounding: weights are rotated into a quantization-friendly basis first, then ternarized with per-group scales and error compensation, which several commenters flagged as the actual differentiator versus a standard low-bit GGUF quant, where naive quantization below roughly 4 bits per weight typically degrades sharply.

The benchmarks

PrismML's published benchmark suite spans reasoning, math, coding, instruction following, vision, and agentic tool use. The headline: Bonsai 2 27B scores 83.9 in aggregate, against 85.4 for the full-precision Qwen3.8 27B baseline — a 98.2% retention ratio. Individual category scores are close to the full-precision baseline across most domains, with coding (81.58 vs. 82.17) and agentic/tool-calling (77.57 vs. 79.74) both landing within a couple of points. PrismML frames the retention pattern as deliberately weighted toward the domains that matter most for real deployment — coding agents, tool-use systems, and multimodal workflows — since those are the tasks most sensitive to compounding small errors across long-horizon runs.

On throughput and efficiency, PrismML reports up to 143 tokens/second on an NVIDIA RTX 5090 and 46.8 tokens/second on an Apple M5 Max, with 0.714 mWh consumed per token on an RTX 4090 — a claimed 40% energy-efficiency improvement over an 8B model running at full precision. All of these are PrismML's own reported figures; the section below covers what independent testing found.

What independent testing on Hacker News actually found

A 244-point HN thread ran the model directly rather than taking the benchmark table at face value, and the results were mixed in informative ways. One commenter running the browser-based WebGPU demo reported the model "immediately started looping" on a test prompt, directly contradicting the "near-lossless" framing for at least that case — a useful reminder that aggregate benchmark scores don't guarantee stability on arbitrary real-world prompts. Others noted the pattern from the first Bonsai generation persists: it's "amazing they work as well as they do" on short tasks, but "use it for any longer task and they fall apart spectacularly and in interesting ways."

Throughput reports varied widely by setup: one tester on an M5 Pro saw roughly 20 tokens/second, then 44 tokens/second after a server restart, with no clear explanation for the difference, alongside a Metal API warning suggesting the tensor API wasn't properly enabled in their environment. Others reported 7-8 tokens/second on a 16GB Mac Mini M2, and 120 tokens/second on a workstation-class RTX 6000 Blackwell GPU with 7.9GB of RAM used — a wide enough spread that PrismML's headline 143 tok/s figure should be read as a best-case number on specific hardware, not a typical result.

The tooling friction drew its own criticism. Running the GGUF weights requires PrismML's own llama.cpp fork, not upstream llama.cpp, because mainline doesn't yet support the ternary format — a real practical cost commenters flagged directly ("they always sound exciting to try out but are such a pain to get working"). Current backend support covers CUDA, CPU, and Vulkan, but not SYCL, leaving some GPU users without a working path yet.

Honest limitations

• The 98.2% retention figure is PrismML's own aggregate benchmark score, not an independent third-party audit, and independent testers found at least one clear failure case (looping) the benchmark table doesn't capture.
• Real-world throughput varies substantially by setup, from roughly 7 tok/s to 120+ tok/s across different hardware and configurations reported by independent testers, well outside PrismML's own headline numbers.
• Requires a proprietary llama.cpp fork, not mainline support — a real adoption friction point until (if) the ternary format lands upstream.
• The compression method itself is proprietary — PrismML hasn't published the exact rotation/scaling/error-compensation algorithm, so the "why does this beat naive quantization" claim can't be independently verified beyond PrismML's own comparison table.

Why the star-to-issue ratio matters less than it sounds

A tangent worth addressing directly, since it came up repeatedly in discussion of PrismML's first Bonsai release and shapes how much weight to put on community sentiment here: a project's GitHub star count is a weak signal of sustained real-world usage on its own, particularly for a model that went viral off the back of one striking claim ("runs on an iPhone"). Stars accumulate from anyone who saw an interesting headline and clicked a button; forks, issues, and sustained discussion threads are better (though still imperfect) proxies for people actually building on something over time. None of that invalidates Bonsai 2's benchmark numbers or the genuine engineering behind ternary compression with error correction — it just means the appropriate level of trust to place in "this is the future of local AI" enthusiasm should track the harder evidence (independent throughput tests, reasoning-stability reports) more than the star count alone.

The bigger pattern: compression as a deployment unlock, not just a storage trick

PrismML's own framing is worth taking seriously on its merits, independent of any single release's rough edges: the argument is that the meaningful question going forward isn't just "how capable is a model," but "how much useful intelligence can be delivered within a given memory, compute, and power budget." If that framing holds, low-bit compression techniques like Bonsai's stop being a niche workaround for phone deployment specifically and start mattering for server-side economics too — fitting more effective capability into the same GPU memory footprint, serving more concurrent users per machine, and reducing energy cost per inference at data-center scale, not just on a single consumer device. That's the more ambitious claim sitting underneath the "runs on your Mac" headline, and it's the one worth watching PrismML (and competitors racing to match this compression quality) execute on over the next several releases, rather than judging on this one launch alone.

What this means for builders

If you're evaluating local models for coding-agent workflows on consumer hardware — the same use case explainx.ai covered for Gemma running via MLX on Apple Silicon — Bonsai 2 27B is worth testing directly against your own workload rather than trusting either the vendor benchmark table or a single Hacker News anecdote. The realistic takeaway from the community reaction: this is a genuine step forward in low-bit compression quality specifically for the domains PrismML optimized for, but "near-lossless" should be read as "closer than before," not "indistinguishable," until it's been stress-tested on your own longer-horizon tasks, using your own prompts, on your own actual hardware setup.

Related on explainx.ai

• PrismML Bonsai 27B: the original 1-bit, 3.9GB Qwen3.6 model
• What is AI model quantization? A complete guide
• Gemma 4 26B-A4B on MLX: Mac speedup
• dFlash 2: MLX M5 Max local inference
• RAM prices and the cost of local AI inference
• How to run open-source models locally with OpenCode
• Official source: PrismML — Introducing Bonsai 2 27B
• Hacker News discussion (244 points)

This post is sourced to PrismML's own September 17, 2026 blog post and whitepaper, plus contemporaneous Hacker News community testing. Benchmark figures are PrismML's self-reported results; independent throughput and stability reports came from community testers on varied, unaudited hardware setups.