Bonsai 27B: A 27B-Class model that runs on a phone

Announcing Bonsai 27B: The First 27B-Class Model to Run on a Phone

Today, we're announcing Bonsai 27B, based on Qwen3.6 27B, the new multimodal flagship of the Bonsai family and the first model of its capability class to run on a phone.

Our earlier releases proved that models with 1-bit and ternary weights could produce commercially useful language models. Bonsai 27B extends that frontier to a new capability tier: multi-step reasoning, structured tool calls, vision tasks, and computer-use agentic loops that stay coherent across many steps. Until today, deploying that tier locally has been impractical for a concrete reason: a 27B model occupies roughly 54GB in 16-bit precision, and even a good 4-bit build, at 18GB, is too large for a phone and for most laptops.

Bonsai 27B changes that. It comes in two variants:

Ternary Bonsai 27B uses ternary {−1, 0, +1} weights with FP16 group-wise scaling, giving a true 1.71 effective bits per weight. At 5.9 GB, it is the quality-oriented variant: it runs on an everyday laptop with the full reasoning, tool-calling, and agentic capability.

1-bit Bonsai 27B uses binary {−1, +1} weights with the same group-wise scaling, giving 1.125 effective bits per weight. At 3.9 GB, it is the footprint-oriented variant, which fits within the memory budget of an iPhone 17 Pro, bringing a 27B-class model onto a phone for the first time.

As with every Bonsai release, the low-bit representation runs end to end across the language network, embeddings, attention, MLPs, and the LM head, with no higher-precision escape hatches. Both variants are multimodal, with the vision tower shipping in a compact 4-bit form so on-device workflows can see screenshots, documents, and camera input, not just text. Bonsai 27B carries a full 262K-token context, and supports speculative-decoding, compounding the speed with lossless draft-and-verify acceleration. Everything is available today under the Apache 2.0 License.

Retaining the intelligence

Across a 15-benchmark suite spanning knowledge, reasoning, math, coding, instruction following, tool calling, and vision (evaluated in thinking mode, where the model's full reasoning is exercised) Ternary Bonsai 27B retains 95% of the full-precision baseline, and 1-bit Bonsai 27B retains 90%.

Fig I: Benchmark scores of Bonsai 27B (thinking mode) against the full-precision baseline. Full per-benchmark results are in the whitepaper.

Read the table by capability and the story is sharper than the averages: math and coding are nearly untouched, tool calling stays within a few points of full precision - exactly the capabilities that agentic workloads depend on. For comparison, the most aggressive conventional low-bit build of the same base model scores significantly lower than 1-bit Bonsai 27B while occupying 2.5x more memory.

This is the same Pareto shift we demonstrated with our earlier language and image models, now at 27B scale: 27B-class capability at a footprint smaller than a full-precision 2B model. By intelligence density — the measure we introduced with 1-bit Bonsai 8B — 1-bit Bonsai 27B delivers 0.53 per GB: more than 10x the full-precision baseline, and roughly 2.7x the best low-bit alternative available.

Why this is an important paradigm shift

The most valuable AI workloads are shifting from single responses to sustained work: assistants that operate real tools, workflows that run unattended before returning a result, and research that synthesizes dozens of documents. This shift changes the shape of the workload — an agent doesn't make one model call, it makes hundreds, each one carrying context, producing structured output, and feeding the next.

Cloud APIs will remain the right choice for many products. But for agentic workloads, cloud-only execution imposes structural constraints: every step is a remote request, per-token cost accumulates with every iteration, and every plan, tool call, and intermediate result crosses the network including the user's private files, screen, and data.

Local execution changes the equation. When a model capable of sustained agentic work fits on the device, the agent can live inside the product: the marginal cost of a hundred-step loop is zero, and the user's data never leaves the machine. Entire categories open up — persistent on-device agents, assistants that work offline, assistants that reason over private local data by construction. What has been missing is a model small enough to deploy this way and capable enough to trust with the work. Bonsai 27B is that model.

It also unlocks a new system architecture: hybrid deployments that route non-frontier and privacy-sensitive tasks to a capable local model and reserve frontier cloud models for the hardest steps — collapsing the cost-per-task of agentic systems.

Bonsai 27B reaches up to 163 tok/s in 1-bit and 134 tok/s in Ternary on an NVIDIA GeForce RTX 5090. On an M5 Max, it reaches up to 87 tok/s in 1-bit and 58 tok/s in Ternary.

Fitting a phone is a stricter gate than storage numbers suggest. A phone never exposes its full memory to an app - a 12 GB iPhone offers about 6 GB for the model to use on-device, and the model shares that budget with its KV cache and activations. No conventional build of a 27B model comes close to clearing it. At about 4 GB, 1-bit Bonsai 27B is the first to pass through with room to work.

That constraint is why the family ships two deliberate operating points, specifically keeping that in mind: ternary for laptop-class quality, 1-bit for phone-class footprint.

The frontier keeps moving

Every Bonsai release has moved the intelligence-per-gigabyte frontier left, and Bonsai 27B moves it past a practical threshold: the full capability set of a modern model with thinking, multimodal understanding, vision, reliable tool use, now fits on the devices people already own.

We believe intelligence density will be one of the defining axes of the next stage of AI progress. Raw capability determines what a model can do; density determines where it can do it. Every leftward shift of the frontier expands the set of devices, products, and environments where advanced AI can operate and changes the economics of every deployment surface it touches, from phones to single-GPU serving. The methodology behind Bonsai is architecture-agnostic, and the frontier will keep moving: larger models and new architectures are already in progress.

Early computers filled rooms; today they live in our pockets. Intelligence is making the same journey, and Bonsai 27B is its largest step yet.

Platform Coverage

Bonsai 27B runs natively on Apple devices (Mac, iPhone, iPad) via MLX and on NVIDIA GPUs via CUDA, through custom low-bit kernels built for its hybrid-attention architecture. Model weights are available today under the Apache 2.0 License. With this release, we’re offering a free, limited-time developer preview API so developers can easily try our model.

Full technical details of our compression, evaluation, and benchmarking processes are available in our whitepaper.

Join Us

PrismML emerged from a team of Caltech researchers and was founded with support from Khosla Ventures, Cerberus, and Google, with continuing support from Samsung. We've spent years tackling one of the field's hardest problems: compressing neural networks without sacrificing their reasoning ability.

If you want to help build the next generation of state-of-the-art AI, we'd love to hear from you. Check out our careers page.