Thursday Sep 24

Lab CEOs Briefed The Security Council

24SEP
FIRST BRIEFINGRULESANTHROPICOPENAI

Bengio, Altman, Amodei and Delangue addressed the UN's top body for the first time. They asked for licensing, evaluators and incident reporting. Trump had rejected global AI control a day earlier.

France chaired through Foreign Minister Jean-Noël Barrot. Yoshua Bengio opened: "The dangers are real and imminent." He wants aviation-style licenses and mandatory liability insurance for frontier developers.

Dario Amodei joined remotely and said poorly managed AI could be a risk to humanity as a whole. He proposed embedded testers, antitrust waivers so labs can coordinate, and a speed limit on self-improvement.

Sam Altman said the biggest decisions cannot be made by labs in San Francisco alone. Clément Delangue of Hugging Face rejected slowing down and asked for shared agent traces. DeepSeek and Moonshot sent statements.

full brief & sources

⚡ Why this matters

  • The Security Council handles wars and sanctions. AI safety just got a seat at that table, with the builders as the witnesses.
  • The people asking for rules are the people who would be regulated. That is either statesmanship or a moat request, and the answer shapes who gets to compete.
  • Washington was in the room and against the premise. The US prefers bilateral deals, including the new incident-notification channel with Beijing.

🔍 What happened

  • The UN Security Council held its first high-level briefing on AI safety on September 23 under the French presidency, chaired by Foreign Minister Jean-Noël Barrot.
  • Briefers were Yoshua Bengio, Sam Altman in person, Dario Amodei by video, and Clément Delangue. Chinese labs DeepSeek and Moonshot were invited to submit statements.
  • Bengio asked for licensing modeled on aviation and nuclear power plus compulsory liability insurance. Amodei asked for embedded evaluators, antitrust waivers for coordination, and limits on the pace of recursive self-improvement.
  • Delangue argued the answer is acceleration with transparency: mandatory sharing of agent traces and disclosure of incidents.
  • On September 22 President Trump told the General Assembly the US rejects any attempt to construct a globalist scheme to control artificial intelligence. Treasury's Bessent and China's He Lifeng agreed an AI incident-notification mechanism on September 21.

💬 Smart takes

  • Yoshua Bengio: "The dangers are real and imminent." Licensing and insurance are how every other dangerous industry earned public trust.
  • Dario Amodei, Anthropic: "If managed poorly, I even believe that AI could be a risk to humanity as a whole." The ask is testers inside the labs, not press releases outside them.
  • Clément Delangue, Hugging Face: "It's not time to slow down but to accelerate." Open traces beat closed promises.
  • Skeptic: Aidan Gomez of Cohere has called the labs' proposed self-regulatory body "a cartel by any other name." The same companies face an antitrust suit over coordination. Rules written by incumbents tend to fit incumbents.

🧭 Where this goes

  1. LikelyFrance pushes a Council statement on AI incident reporting before its presidency ends.
  2. Possiblethe US and China route everything through the bilateral channel and the UN track stalls.
  3. Wild Carda member state proposes a binding resolution on frontier model licensing, and the veto question becomes real.

🥄 The Spoon Take

Ignore the speeches and watch the seating chart. Four private citizens briefed the body that handles wars, while the largest AI power said no thanks the day before. The realistic outcome is not a treaty. It is two systems: bilateral US-China guardrails and a UN process everyone else joins. Plan for both.

🤔 Pushback

The Council has no AI mandate and the US just rejected one. A briefing is theater until a resolution follows.