American startup Prism ML released Ternary Bonsai 2 27B — a ternary rework of Qwen3.8-27B that takes up 5.9 GB versus approximately 54 GB for the original in FP16 and retains 98.2% of its average quality on 14 benchmarks. A full-featured 27B-class model with reasoning and vision at such density fits in approximately 6 GB and runs locally — from gaming graphics cards to iPhone, without sending data to the cloud.

!image.png)

image
image

What happened

The release took place on September 17, 2026. On 14 benchmarks, Ternary Bonsai 2 27B scores an average of 84.78 points versus 86.32 for the original Qwen3.8-27B in FP16 and 72.59 for the standard 2-bit quant IQ2_XXS; the effective density is 1.72 bits per weight. The package included two GGUF builds — PTQ1_0 (1.75 bits, 5.95 GB, faster on Ada/L4) and PQ2_0 (2.13 bits, 7.21 GB, faster on H100/Blackwell) — for a special llama.cpp fork from Prism ML, an 8.60 GB MLX build for Apple Silicon with vision, and a WebGPU demo that runs directly in the browser.

Context

Prism ML emerged from stealth mode in 2026: the company was founded by Caltech professor Babak Hassibi, with investors including Khosla Ventures, Cerberus, Google, and Samsung. Ternary quantization means that each weight is rounded to one of three values: −1, 0, or +1. Bonsai 2 differs from a standard post-training quant in that the scheme is built into the model format: weights are split into groups of 128 with one common FP16 multiplier, and a fixed Hadamard transform is embedded in the weights themselves. During inference, this transform must be repeated on the activations, otherwise the model outputs nonsense, so the release cannot be reproduced with standard tools. Against this backdrop, the contrast with standard 2-bit quants is notable: at a comparable file size, they lose noticeably more quality, and if Prism ML's measurements are confirmed by independent runs, this will be the strongest technical argument of the release.

Why this matters for the industry

Ternary density translates a 27B-class model into the memory of an 8-billion model: with the same hardware, this means more users per GPU and lower power consumption — according to Prism ML's measurements, 0.714 mWh per token on an RTX 4090, which is 40% more efficient than a full 8B model. The bet on local agents — code agents like Cline, computer-use scenarios, parsing corporate documents without external servers — turns private on-device inference from a demonstration into a working scenario and opens up a class of products without expensive GPU hosting. The entire low-bit landscape is now competing not only on accuracy but also on "intelligence density per gigabyte" — a metric that looks like a sustainable bet with investors at the level of Google and Samsung.

Why this matters for users

The weights are published on Hugging Face under Apache 2.0, and the model can be tried without installation through a ready-made WebGPU demo in the browser. According to Prism ML's measurements, it outputs about 143 tokens per second on an RTX 5090 and about 46.8 tokens per second on a laptop with M5 Max, and can be launched via MLX even on iPhone and iPad. The vision module is separated into a 0.63 GB file and is loaded only when images are provided, so the base build remains compact. A context of 262k tokens and tool calling allow parsing large documents and building agents on your own device: correspondence and files do not leave it, and the cost of the experiment is almost zero — no cloud budget is needed.

What is still unknown / limitations

All key figures — quality, speed, and power consumption — are currently vendor-provided: no independent community runs have been published, so the reliability of deployment in production will have to be proven with your own evals. The claimed advantage over the full Qwen3.6 27B (83.9 versus 83.6) should be treated with caution: the 0.3-point difference lies within the noise of typical benchmark rigs, and different models are being compared — a ternary rework of Qwen3.8-27B and Qwen3.6 27B, which requires the same set of tasks and methodology. Speed depends on the optimization of repeating the Hadamard transform on activations in the Prism ML fork, so measurements cannot be directly transferred to other backends. Finally, it is unknown whether the ternary kernels and Hadamard support will make it into upstream llama.cpp and MLX — without this, the format will remain vendor-specific.

Sources

Author

Look at AI, editorial team