PrismML Ternary Bonsai 2: 2B, 4B and 9B Models From 0.43 GB, Apache 2.0

PrismML has released three new small models in its Ternary Bonsai 2 line: 2B, 4B and 9B parameters, built on Qwen3.5 and shipped under Apache 2.0. According to the company's announcement dated October 15, 2026, the files weigh 0.43 GB, 0.95 GB and 2.00 GB, "more than 9x smaller" than the full-precision versions. The family now covers four sizes up to the 27B model we covered last month.

The pitch is agents on devices with little memory: phones, smart glasses and wearables. The claims are PrismML's own, so this post separates what the company says from what is still unverified.

TL;DR

What is a ternary model?

A normal model stores each weight as a 16-bit number. A ternary model stores three possible values, so each weight needs roughly 1.58 bits. Multiplications turn into additions and sign flips, which is why such models can be tiny and fast on the right kernels. The tradeoff is accuracy, because squeezing weights this hard usually costs quality unless the model is trained or tuned for it.

PrismML's earlier models were 1-bit and ternary versions at 1.7B, 4B and 8B. We covered the flagship in PrismML Bonsai 2 27B: 9x smaller at 98.2% of full-precision quality and the earlier 1-bit 27B that runs on an iPhone. The new three are the small end of that line.

What happened with the 27B, and why it matters here

The new release should be read against how the 27B landed. At launch on September 17 PrismML reported 98.2% of full-precision benchmark score at 5.9 GB and up to 143 tokens per second on an RTX 5090. Our write-up of that release noted a large Hacker News thread in which testers reported lower and less consistent speeds, roughly 7 to 44 tokens per second depending on setup, at least one looping failure in the browser demo, and friction from needing PrismML's own llama.cpp fork because the ternary format is not in mainline.

None of that makes the company's results false. It does show the pattern to expect for the small models too: headline numbers come from the best configuration, and real use varies with the runtime, the kernels and the task length. So for the 2B, 4B and 9B, ask three questions before adopting them. Does your runtime support the ternary format without a custom fork? Does the model stay coherent on the long tool-calling sequences your agent actually produces? And does it beat a plain 4-bit quantization of the same Qwen3.5 base at a similar memory size? PrismML's own comparison on the 27B claimed parity with a Q4_K_XL quantization at lower memory, but that comparison was for a different model.

On-device AI illustration of a small model running locally on a phone, for Ternary Bonsai 2 on-device use

What PrismML claims about quality

Across 19 benchmarks, the company says the new ternary models "match or exceed" their full-precision counterparts on coding, math and instruction following. On vision, agentic and tool-calling tasks, and knowledge and reasoning, it says they keep "well over 90%" of the full-precision Qwen3.5 scores. It also says all three show "considerable performance gains" over first-generation Ternary Bonsai.

The release does not list the 19 benchmarks or the scores in the text we read. That is the main gap: a ratio against full precision is only as meaningful as the tasks behind it. Treat "matches full precision on math" as a claim to test on your own prompts.

What PrismML says it is for

PrismML founder and CEO Babak Hassibi is quoted in the announcement saying that "on-device agents have to be smart enough to make the right decisions while working within limited memory and power budgets." The company describes the target as agents that interpret a request, reason over text or images and call application tools for short action sequences, running on personal computers, phones, smartglasses and other wearables, with inference tuned for consumer CPUs and edge GPUs. One detail worth noting: the smart glasses demo shown at the Snapdragon Summit used the first-generation 1-bit Bonsai model, not these Bonsai 2 models.

Speed numbers

PrismML reports, on hardware such as an RTX 4090 and a MacBook M5 Pro, up to 124 tokens per second for the 9B and 276 for the 2B, plus up to 5x throughput and 3x energy efficiency versus full precision. On an iPhone 18 Pro Max, it reports up to 34 tokens per second for the 9B and 120 for the 2B.

We have not reproduced these numbers, and we did not find independent speed reports we could cite. The press release gives no details on batch size, context length, quantization format or kernels, and the iPhone figures are labelled "up to." The 27B launch showed how far real-world speeds can sit below a vendor's best case, so budget for a lower number until you measure on your own device.

Why a local first pass is the realistic use

Small models like these pay off most as a local first pass: filter, classify or draft on the device, then send only what needs a bigger model to the cloud. How much you save depends entirely on how much of your data a small model can handle on its own, which PrismML's release does not quantify.

Where this fits: phones, glasses and wearables

In September PrismML demonstrated a 1-bit Bonsai vision-language model on smart glasses with the Snapdragon AR1 Gen 1 platform. The new release builds on that with better reasoning, vision and tool use, which are the three things a basic assistant on glasses needs: understand the scene, decide what to do, call an app function. PrismML says developers can build assistants that interpret a request, reason over text or visual context and use application tools for short sequences of actions.

For other small local models, compare with Tether's 250M speech model, the 1M-parameter Indra OCR model and our roundup of the top 10 open-weight models for laptops.

How to try it

1. Download Ternary Bonsai 2 9B, 4B or 2B from the links in PrismML's announcement (the company points to Hugging Face, GitHub and docs).
2. Use PrismML's tooling. Per third-party descriptions of the earlier Bonsai 2 release, ternary models needed PrismML's llama.cpp and MLX forks until kernels are upstreamed; check the model card for the current requirement before assuming stock llama.cpp works.
3. Start with the 4B on a laptop and test your own prompts for tool calling, since agent reliability is the claim that matters most for these sizes.
4. Compare against the unmodified Qwen3.5 of the same size on your task, not on the vendor's benchmark list. If you run models locally with a coding agent, see running open models locally with OpenCode.

What this means for what you build or pay

• Memory is the constraint these models remove. A 2 GB file for a 9B model fits in places a 16 GB file never will, including mid-range phones.
• Latency for agents. Short tool-calling loops are dominated by prefill and decode speed; that is where the 5x throughput claim, if it holds on your hardware, would show up.
• Licensing is simple. Apache 2.0 allows commercial use, but the base is Qwen3.5 (published by the Qwen team on Hugging Face), so check Qwen's terms as well. PrismML's earlier launches are listed on its news page.
• Test before you trust. "Well over 90%" of full-precision scores can still mean the failures land on your hardest cases.

Which size for which job

The fit column is our reasoning from the sizes and PrismML's stated aims, not a measured result. The practical advice is to pick the smallest model that passes your own test set, because every step down in size buys memory and battery.

Claimed versus verified

Figures are from PrismML's October 15, 2026 announcement and linked posts, and may change.

Related reading

• PrismML Bonsai 2 27B near-lossless compression
• PrismML Bonsai 27B on an iPhone
• Tether on-device speech recognition
• Indra OCR on CPU
• Top 10 open-weight models for laptops