Friday Jul 17
COOL TOOLSOther

PrismML Fits 27B On An iPhone

17JUL
11 TOK/SEC27B PARAMS3.9GB

AI models are shrinking small enough for your phone. PrismML shrank a 27B model to 3.9GB, running on an iPhone. It hits 11 tokens a second, and Apple is testing it, per CNBC.

Most on-device AI models are toy-sized compared to cloud models. Bonsai 27B breaks that pattern with real reasoning power.

It handles multi-step reasoning, tool calls, and images, not just text. The 1-bit version keeps 90% of full-precision accuracy. Weights are free under Apache 2.0, so any developer can ship it today.

PrismML's CEO says Apple and others are already testing it for speed and battery drain. If a phone-sized model holds most of full power, cloud inference bills start looking optional.

full brief & sources

⚡ Why this matters

  • Cloud inference is the biggest cost line for AI products. A model this small kills that cost for many use cases.
  • It proves compression, not bigger GPUs, can close the capability gap for on-device AI.

🔍 What happened

  • PrismML released Bonsai 27B on July 14, 2026, compressed from a Qwen3.6 27B base.
  • The 1-bit variant is 3.9GB and runs on an iPhone 17 Pro at 11 tokens a second.
  • A larger 5.9GB ternary variant targets laptops and keeps 95% of full-precision performance.
  • It supports text, images, tool calls, and multi-step agentic tasks.
  • Weights ship under Apache 2.0, free for commercial use, via MLX on Apple devices and CUDA on NVIDIA GPUs.
  • PrismML CEO Babak Hassibi told CNBC that Apple and other companies are testing the compression for speed and power draw.

💬 Smart takes

  • Babak Hassibi (PrismML CEO): confirmed Apple and other companies are testing the model for speed, power draw, and performance.
  • Skeptic: a 1-bit model still loses real accuracy, and hard agentic tasks may expose that gap fast.

🧭 Where this goes

  1. Likelymore labs race to ship sub-4GB models as the phone becomes a real inference target.
  2. LikelyApple evaluates the technique for a future on-device Apple Intelligence upgrade.
  3. Possibleenterprises start offloading simple agent tasks to phones to cut cloud inference bills.
  4. Wild CardApple licenses or acquires PrismML's compression tech within 12 months.

🥄 The Spoon Take

Cloud AI margins depend on inference being expensive. A free 3.9GB model that runs an agent on a phone chips at that math directly. It isn't the smartest model out there, but it's smart enough for a lot of real work, and it costs nothing to run.

🤔 Pushback

The accuracy loss is real, and the tasks that need it are exactly what enterprises pay cloud prices for.

·9to5Mac - Bonsai 27B on iPhone·PrismML - Bonsai 27B announcement·The Decoder - Bonsai 27B reasoning model