Thursday Aug 27
HARDWAREOpenAI

OpenAI's Own Chip Beat Blackwell On Power

27AUG
PERF PER WATTONE PLUGBLACKWELL

OpenAI showed the first Jalapeno benchmarks at Hot Chips. A third-party test measured 104x the throughput per kilowatt of an Nvidia GB300 on DeepSeek R1. Deployment starts late this year, in tiny volumes.

Jalapeno is OpenAI's inference chip, built with Broadcom and announced last October. This is the first time real numbers appeared.

SemiAnalysis ran the InferenceX benchmark. Result: 104.3x throughput per kilowatt versus a GB300 on DeepSeek R1, and 3.6x lower end-to-end latency.

Richard Ho, OpenAI's head of hardware, called it a very significant advance over state of the art. He also warned the comparison is against chips shipping today.

full brief & sources

⚡ Why this matters

  • Power is the constraint on inference, not chip count.
  • A lab designing its own inference silicon changes who it has to buy from.
  • The number is big enough that skepticism is the correct first reaction.

🔍 What happened

  • Presented Tuesday Aug 25 at Hot Chips.
  • Design minimizes prefill and communication delays and keeps the KV cache local.
  • Beat Blackwell on performance per watt in nearly every tested scenario.
  • OpenAI's own models helped design it. Gen 2 is in development, Gen 3 taking shape.
  • Ships end of 2026 in very small volumes, larger in 2027.

💬 Smart takes

  • Third-party benchmark helps. SemiAnalysis is not OpenAI's marketing team.
  • Ho's own caveat is the honest one. Nvidia ships something new before Jalapeno reaches volume.
  • A purpose-built inference chip beating a general-purpose GPU on watts is expected. The margin is what surprises.

🧭 Where this goes

  1. LikelyOpenAI keeps buying Nvidia at scale through 2027 anyway.
  2. Possibleother labs accelerate their own silicon programs on this proof point.
  3. Wild CardOpenAI sells or rents Jalapeno capacity to outside customers.

🥄 The Spoon Take

Inference economics decide which products survive. A lab that controls its own cost per token can price things a reseller cannot. The volume caveat matters more than the benchmark. Late 2026 in very small volumes means this is a 2028 story dressed up as a 2026 headline.

🤔 Pushback

One vendor-selected benchmark on one model. Blackwell is the current generation, not the one Jalapeno will actually compete against.

·Jalapeno: first results·OpenAI shows first benchmarks for its custom Jalapeno chip·OpenAI says its custom chip beats Nvidia on performance per watt