AI startup PrismML has released Bonsai 2 27B, a compressed version of Alibaba’s Qwen3.8 27B model that takes up just 5.9GB of memory while retaining most of the original model’s reported performance.
The new model is more than nine times smaller than its full-precision counterpart, according to PrismML, and is designed to make capable AI models easier to run directly on personal devices rather than relying entirely on cloud servers.
PrismML was founded by a team of researchers with roots at Caltech and focuses on reducing the memory and computing requirements of AI models.
A 27B Model in a 5.9GB Footprint
Bonsai 2 is based on Qwen3.8 27B, meaning PrismML did not train a new 27-billion-parameter model from scratch. Instead, the company compressed the existing model using a low-bit approach that represents its weights using three possible values: -1, 0 and +1.
This reduces the amount of memory required to store and run the model.
PrismML says Bonsai 2 occupies 5.9GB while retaining 98.2% of Qwen3.8 27B’s aggregate performance across a collection of 20 benchmarks covering areas such as reasoning, coding, mathematics, vision and tool use.
Those figures come from PrismML’s own testing and should not be treated as independent measurements. However, the company has published its model weights and technical evaluation materials for further testing.
The model is available under the Apache 2.0 license.
Designed to Run AI Locally
Reducing the size of a model can make it practical to run on hardware with much less available memory.
PrismML says Bonsai 2 supports NVIDIA GPUs through CUDA and Apple hardware, including Macs, iPhones and iPads, through Apple’s MLX framework.
Running models locally can reduce the need to send every request to a remote server. That can be useful for applications involving private documents, coding assistants or other workloads where latency, connectivity or data privacy are important.
PrismML demonstrated Bonsai 2 running coding-agent and computer-use workloads on an NVIDIA RTX 5090. The model also supports text and image input and a context window of up to 262,000 tokens.
Bonsai 2 follows PrismML’s earlier Bonsai 27B release and improves the amount of performance the company says it can preserve after compression.
PrismML reported roughly 95% performance retention with the previous generation, compared with 98.2% for Bonsai 2.
The startup’s broader goal is to make larger AI models practical on consumer hardware by reducing their memory and power requirements.
TechCrunch reports that PrismML has raised a $22.25 million seed round and was founded by Caltech researchers led by professor Babak Hassibi. Databricks co-founder Ion Stoica serves as an adviser to the company.
PrismML says it plans to apply its compression technology to models with several hundred billion parameters in future releases.
For now, Bonsai 2 provides another example of a growing effort across the AI industry to make capable models smaller enough to run locally instead of requiring increasingly large cloud infrastructure.
Leave a Reply