Tuesday Aug 11
RESEARCHOther

A Researcher Fingerprinted The Frontier Labs

11AUG
AI MODELSCUTOFF DATE

You can date a model just by asking it questions. Researcher Shrivu Shankar built a quiz that infers a model's training cutoff. Some Anthropic models self-identify as GPT-4, hinting at their training data source.

No lab publishes its exact release date anymore. Shankar's probe fills that gap using trivia and self-reports.

Opus 4.7-and-up systems cluster around late December. The GPT-5.6 family checks in around late February instead. Opus 5 seems to know less than its official date implies.

The GPT-4 echo hints some training leaned on older outputs. That's a quiet admission the industry rarely makes on its own.

full brief & sources

⚡ Why this matters

  • Labs stopped disclosing exact training cutoffs, so outsiders built their own tests.
  • Knowing a model's real cutoff matters for anyone building on 'knows current events' claims.
  • The GPT-4 self-identification pattern raises questions about where training data actually comes from.

🔍 What happened

  • Shankar published the methodology and results on Aug 10 on his blog.
  • The quiz asks models to self-report dates and answer historical-fact questions scored against known answers.
  • Anthropic's Opus 4.7+ models share a late-December-2025 cutoff, suggesting one shared training run.
  • OpenAI's GPT-5.6 family clusters around a late-February-2026 checkpoint.
  • Opus 5's knowledge state looks closer to January 2026 despite an official May cutoff.
  • Some Anthropic model responses self-identify as GPT-4, an artifact of training on prior-generation outputs.

💬 Smart takes

  • Shankar: frames the probe as filling a transparency gap labs have quietly stopped closing themselves.
  • Skeptic: self-reported dates and quiz answers are indirect signals, not a lab's actual training logs.

🧭 Where this goes

  1. Likelylabs face more pressure to publish exact cutoff dates in model cards going forward.
  2. Possibleother independent researchers replicate the method on newer model releases.
  3. Wild Carda lab issues a public rebuttal specifically about the GPT-4 self-ID finding.

🥄 The Spoon Take

Every model card says 'knowledge cutoff' like it's a fact, not a guess. Turns out you can check the guess yourself with nothing but a chat window and some clever questions. The real story isn't the exact dates, it's that outsiders can now audit a claim labs used to fully control.

🤔 Pushback

A quiz-based probe infers patterns, not certainty. A model self-identifying as GPT-4 could be a prompting quirk, not proof of what data it trained on.

·Shrivu Shankar - Knowledge cutoff probe·AI Weekly - Aug 10 roundup mention