Tuesday Sep 8

OpenAI's Chief Scientist Asks For Brakes

8SEP
RECURSIVE AICHIEF SCI

Jakub Pachocki says no lab has solved alignment well enough to keep scaling at full speed. He wants voluntary slowdowns now and mandated safety bars enforced by outside auditors.

The essay is called An Alien Mind. Its core claim: internal results make recursive self-improvement look reachable at the current pace.

He wants the Preparedness Framework and Anthropic's Responsible Scaling Policy turned into mandated bars, enforced by third-party auditors, agencies or international bodies.

It landed the same day OpenAI published the data showing 3.1 agent workdays per human. Both posts are the same argument.

full brief & sources

⚡ Why this matters

  • The person running the fastest research org is asking to be constrained, on the record, with a named mechanism.
  • He is not asking for principles. He is asking for auditors and enforcement, which is a very different ask.
  • It sets a marker other labs now have to answer, publicly or by staying quiet.

🔍 What happened

  • OpenAI chief scientist Jakub Pachocki published 'An Alien Mind' on September 6.
  • He writes that no lab has solved alignment and monitoring well enough to keep scaling at maximum speed.
  • He expects and hopes voluntary slowdowns become commonplace until shared safety bars exist.
  • He argues scaling has to be constrained by confidence in safety, not by capability alone.
  • He wants the Preparedness Framework and the Responsible Scaling Policy to become widely mandated bars, enforced by third-party auditors, government agencies or international bodies.
  • He calls international coordination on future AI development a top priority for governments.
  • On recursive self-improvement, he says OpenAI pursues it because it is the only way to stay at the research frontier.

💬 Smart takes

  • Pachocki: based on internal results, he has a strong expectation that the current speed of progress could be sustained into recursive self-improvement.
  • Pachocki, on the mechanism: voluntary commitments need to evolve into mandated safety bars with outside enforcement.
  • Skeptic: a company that publishes 'we are three times faster with agents' and 'please slow us down' on the same day is hedging, not braking.

🧭 Where this goes

  1. Likelyother lab leads get asked to endorse or reject the mandated-bar idea within weeks.
  2. Likelythe phrase 'safety bars' shows up in a legislative draft before the end of the year.
  3. Possiblea third-party audit body forms specifically to certify frontier training runs.
  4. Possiblea lab announces an actual voluntary pause and cites this essay.
  5. Wild Cardan international coordination process starts and the labs end up writing their own rules inside it.

🥄 The Spoon Take

Two OpenAI posts, one day. One says the agents are compounding. The other says nobody knows how to hold the wheel. The interesting part is the specific ask: not 'be careful' but 'audit us, by law'. That is a company trying to buy a speed limit it cannot impose alone.

🤔 Pushback

Asking for regulation you helped design is also a moat. And nothing in the essay commits OpenAI to slowing down first.