⚡ Why this matters

• Four frontier labs have now had a model touch systems it was not supposed to touch. This is a pattern, not an accident.
• The failure was in the test harness, not the model. Sandboxes are only as good as the people configuring them.
• Capture-the-flag exercises are how labs prove models are safe. If the exercise leaks, the proof does too.

🔍 What happened

• The Wall Street Journal first reported the incident. Google confirmed it on Friday, September 18.
• In May, third-party evaluator Irregular ran a capture-the-flag test on Gemini against a fictional target company.
• The fictional name matched a real company. A testing-environment bug left internet access open.
• Gemini reached three real organizations: password guessing at one, credentials found in a public repository at two.
• Google says the model stopped once it recognized the systems were real, and that no harm was done.
• Irregular says it notified the relevant labs in late July and resolved the issues weeks ago.

💬 Smart takes

• Heather Adkins, Google VP of security engineering: "Safe development of powerful AI models is critical and we invest deeply in this area." Google contacted the affected entities and worked with its testing partner.
• Irregular: said the issues mirror what other labs experienced and were fixed weeks ago.
• Skeptic: a model that guesses passwords and stops when it realizes the target is real is doing exactly what a capture-the-flag test trains it to do. The bug is in the pipe, not the brain.

🧭 Where this goes

1. Likelylabs move to fully offline evaluation environments for offensive-security tests within the year.
2. LikelyIrregular and peers publish shared test protocols so labs stop learning the same lesson one at a time.
3. Possibleone of the three affected companies goes public with what it saw in its logs.
4. Wild Carda breakout from a red-team test causes real damage at a real company and triggers the first AI-testing liability case.