PrismML's Bonsai 2 27B Shrinks Qwen3.8 27B by More Than 9x

Introduction

PrismML released Ternary Bonsai 2 27B on September 17, 2026, the second generation of its compressed language model built on Alibaba's Qwen3.8 27B. The company's pitch is compression with almost no quality loss: Bonsai 2 27B ships at 5.9GB, more than 9x smaller than its full-precision base model, while retaining 98.2% of that model's aggregate benchmark performance. According to TechCrunch, PrismML has raised a $22.25 million seed round and is led by CEO Babak Hassibi, a Caltech professor specializing in compression technologies; PrismML's own announcement describes the company as founded with support from Khosla Ventures, Cerberus, and Google, with continuing support from Samsung.

Bonsai 2 27B is the successor to the original Bonsai 27B, which PrismML announced two months earlier, on July 14, 2026. That first release retained roughly 95% of its full-precision base model's benchmark performance; PrismML says the new generation closes most of that remaining gap, pushing retention to over 98%.

Feature Overview

Ternary weights. The core technique is a ternary weight format: each parameter is stored as one of three values, -1, 0, or +1, with a single FP16 scale factor shared across every group of 128 weights. PrismML's launch post puts the resulting effective cost at 1.76 bits per weight, applied end to end across the language model rather than to a subset of layers. The model's Hugging Face model card additionally lists two packaging formats for llama.cpp deployment: a dense "PTQ1_0" pack at 1.75 bits per weight (5.95GB) and a "PQ2_0" pack that stores each trit in its own 2-bit slot at 2.13 bits per weight (7.21GB), trading footprint for faster unpacking on certain GPUs.

Benchmark retention. Across a six-category suite spanning agentic and tool-calling tasks, coding, instruction following, knowledge and reasoning, math, and vision, PrismML reports Bonsai 2 27B scoring an aggregate 83.9 in thinking mode, against 85.4 for the full-precision Qwen3.8 27B and 83.6 for the older Qwen3.6 27B. Reasoning-heavy categories hold up closely to the full-precision baseline: math scores 96.57 versus 97.06, and coding scores 81.58 versus 82.17. Instruction following is the one category where the compressed model scores above its full-precision base, 82.66 versus 81.25. Vision and knowledge-and-reasoning show wider gaps, trailing by roughly three points each. Separately, the model's Hugging Face card reports its own 14-benchmark thinking-mode average of 84.78 against an 86.32 full-precision reference, the same 98.2% retention figure measured a different way.

Context, modality, and license. The model supports a 262K-token context window and multimodal text-and-image input, and PrismML releases the weights under the Apache 2.0 license on Hugging Face, covering both a GGUF build for llama.cpp and an MLX build for Apple Silicon.

Throughput and efficiency. PrismML reports Bonsai 2 27B reaching up to 143 tokens per second on an NVIDIA RTX 5090 and 46.8 tokens per second on an Apple M5 Max. On an RTX 4090, the company measured 0.714 mWh per generated token, which it says makes the model 40% more energy-efficient than an 8B model running at full precision.

Usability Analysis

The practical pitch is running 27B-class reasoning, coding, and vision capability on hardware that couldn't hold the 54GB full-precision Qwen3.8 27B checkpoint at all: a single consumer GPU, a gaming laptop, or an Apple Silicon Mac. PrismML demonstrates the model driving the Cline coding agent and a computer-use workflow on an RTX 5090, positioning Bonsai 2 27B for local coding-agent loops, computer-use tasks, and private document analysis where sending data to a cloud model isn't desirable or where 262K tokens of local context is more useful than round-tripping to an API. TechCrunch reports, citing the company, that the original Bonsai model has been downloaded more than 11 million times and PrismML's even smaller models another 2.6 million times, suggesting real developer uptake of the earlier releases.

Two adoption factors matter here. First, actually running the model still requires llama.cpp or MLX builds with the custom low-bit kernels PrismML ships, rather than being a drop-in replacement in every existing inference stack. Second, PrismML's own benchmark suite, while broad, is self-reported; independent, third-party reproduction of the 98.2% retention figure was not available at the time of writing.

Pros and Cons

The ternary compression approach delivers a genuinely small footprint without the steep accuracy collapse that PrismML says conventional low-bit quantization methods suffer at similar sizes, based on its own comparison data. A permissive Apache 2.0 license and public weights on Hugging Face let developers deploy or fine-tune the model without negotiating separate commercial terms. The 262K-token context window paired with multimodal input covers use cases, like long-document or coding-repository analysis, that would otherwise require a larger cloud-hosted model. Support for both CUDA and MLX kernels means the same underlying weights target NVIDIA GPUs and Apple Silicon rather than one hardware ecosystem exclusively.

The compression still gives up real ground in vision and knowledge-heavy tasks, trailing the full-precision baseline by roughly three points in each category even as other categories stay near parity. The reported partnership talks with a major device maker remain an unconfirmed third-party report, since PrismML's CEO declined to comment when TechCrunch asked directly. A $22.25 million seed round is modest next to some competitors in the model-compression space, which is worth watching as PrismML hopes to release models in the several-hundred-billion-parameter range in the next couple of months.

Outlook

The move from about 95% retention with Ternary Bonsai 27B to 98.2% with Bonsai 2 27B, roughly two months apart, suggests the compression method is still improving from one generation to the next. CEO Babak Hassibi told TechCrunch that larger models should be easier to compress without losing capability, and that the company's next target is the several-hundred-billion-parameter range. If that scaling claim holds, ternary compression could extend beyond on-device use cases into changing how much memory and power server-side deployments need per model, a claim PrismML makes directly in its own announcement.

Conclusion

Bonsai 2 27B is a meaningful step for PrismML's compression approach, closing most of the gap between a ternary-weight model and its full-precision base while keeping the deployment footprint small enough for a single GPU or a laptop. It's a strong fit for developers who want local, private inference with a large context window and are comfortable working with llama.cpp or MLX rather than a hosted API, and worth watching as PrismML applies the technique to substantially larger models next.

Editor's Verdict

PrismML's Bonsai 2 27B Shrinks Qwen3.8 27B by More Than 9x earns a solid recommendation within the Other LLM space.

The strongest case for paying attention: the ternary compression approach keeps 98.2% of the full-precision model's aggregate benchmark score while cutting the footprint by more than 9x, keeping the model usable on a single consumer GPU or laptop. That alone raises the bar for what readers should expect in this space. Reinforcing that, a permissive Apache 2.0 license and open weights on Hugging Face let developers deploy or fine-tune the model without negotiating separate commercial terms — practical value rather than just headline appeal. The broader signal worth registering is straightforward: the ternary weight format stores each parameter as one of three values, -1, 0, or +1, with a shared FP16 scale per group, cutting effective storage to roughly 1.76 bits per weight. On the other side of the ledger, one constraint is real rather than a marketing footnote: the compression still gives up real ground in vision and knowledge-heavy tasks, trailing the full-precision baseline by roughly three points in each category. It should factor into any serious decision. Layered on top of that, the reported Apple partnership talks remain an unconfirmed rumor from a third-party report, since PrismML's chief executive declined to comment when asked directly — which narrows the set of teams for whom this is an obvious yes.

For multi-model deployment teams, cost-conscious operators, and developers willing to evaluate beyond the major labs, this is a serious evaluation candidate, not just a curiosity to bookmark. For everyone else, the safer posture is to monitor coverage and revisit once the use cases that matter to your team are demonstrated in the wild.