The most useful question in local AI is not which model has the highest score. It is where the model can actually run. Two releases this week make that question unusually clear. PrismML’s Bonsai 27B aims to put a multimodal model in the memory budget of a phone (the same lab already shipped a 1-bit 8B Bonsai earlier this year), while Thinking Machines’ Inkling offers open weights at a scale that still belongs mostly to specialized infrastructure. Both are open-weight releases, but they describe very different futures for people who want more control over their AI workloads - a contrast our open-weight hardware showdowns have been tracking all year.
Bonsai shrinks a 27B model toward phone scale
PrismML released Bonsai 27B as a model derived from Qwen3.6-27B. Its 1-bit version uses 1.125 effective bits per weight and weighs about 3.9 GB, compared with about 54 GB for the full-precision reference in the model card. A separate ternary version is listed at 5.9 GB. The release puts both under the Apache 2.0 license and describes them as multimodal models. (PrismML, model card)
That compression is not just a smaller download. The model card reports a 76.11 average across its 15-benchmark thinking suite for the 1-bit Bonsai, against 85.07 for the Qwen3.6-27B FP16 reference. PrismML describes that as 89.5% of the reference score. Those are vendor-reported evaluations, so they are best read as a picture of the trade-off rather than an independent verdict. The same card reports a 262K-token context window for the model. (Bonsai model card)
The more striking demonstration is the target device. The Decoder reports that the 1-bit model ran on an iPhone 17 Pro Max at about 11 tokens per second. PrismML’s demo repository also documents local runs across macOS, Linux, Windows, CPU, and GPU paths, with the 27B models supporting images, tool calling, and adjustable reasoning effort. (The Decoder, Bonsai demo)
The caveat is important. A phone demonstration is not the same as a polished mobile app, and a peak speed does not tell us how a device behaves during a long session. Still, the central change is real: quantization has moved a model in the 27B class from an obvious desktop or server requirement toward hardware that many readers already carry.
Inkling chooses scale over convenience
Inkling takes the opposite path. Thinking Machines describes it as a sparse Mixture-of-Experts transformer with 975 billion total parameters and 41 billion active parameters. Each layer has 256 routed experts and two shared experts, with six routed experts active for each token. Its context window reaches up to 1 million tokens. The model card lists text, image, and audio inputs with text output, and the release says the training mix included text, images, audio, and video. (Thinking Machines, Inkling model card, Hugging Face)
The company is unusually direct about what Inkling is not. “Inkling is not the strongest overall model available today, open or closed,” the release says. Instead, Thinking Machines presents it as an open base for customization, with fine-tuning available through its Tinker platform. That is a different product proposition from a model whose main selling point is winning a general benchmark. The goal is to give developers a large, multimodal starting point that they can adapt to a particular task. (Thinking Machines)
Open weights do not make Inkling a practical local download for most people. Its model card says the BF16 checkpoint needs at least 2 TB of aggregate VRAM, while its NVFP4 checkpoint needs at least 600 GB. Those figures place the uncompressed and lower-precision versions in a very different class from Bonsai’s phone-sized 1-bit file. Inkling is open in the sense that its weights are available under Apache 2.0, but running it still assumes access to serious hardware or a hosted inference provider. (Inkling model card)
Open weights now mean two different things
Bonsai and Inkling are often grouped under the same open-model label, but they answer different user needs. Bonsai is an attempt to reduce the cost of inference until a large model can fit on consumer hardware. Inkling is an attempt to make a very large model adaptable, while leaving the heavy computation to infrastructure that can support it.
That distinction matters for privacy and for budgets. A model that runs on a device can make local inference possible, but the surrounding app still determines what leaves the device. An open model that requires hundreds of gigabytes of VRAM can be inspected, downloaded, and fine-tuned without being a realistic self-hosting option for an individual reader. The license answers who may use the weights; it does not answer who can afford to run them.
What This Means
For local-AI users, the useful split is no longer simply open versus closed. Bonsai shows how aggressive compression can make a 27B-class model fit a phone-sized memory budget, with a measurable drop from its full-precision reference. Inkling shows how open weights can instead serve as a customization substrate, even when the hardware requirement remains far beyond a normal laptop. (Bonsai model card, Thinking Machines)
The practical choice follows from that difference. If the priority is experimenting locally, Bonsai is the more relevant release to watch. If the priority is adapting a broad multimodal model to a specialized workflow, Inkling’s Tinker integration is the more relevant promise. Neither release makes the other obsolete. They mark two ends of the same argument about who should control AI inference.
The Bottom Line
Bonsai 27B pushes open weights toward a phone, while Inkling pushes them toward customization at data-center scale. The important development is not that one is the better model. It is that “open” is becoming a choice between local execution and adaptable access, with very different hardware and privacy consequences.