Monday Aug 24
COOL TOOLSOther

Coding Agents Get Their Own Index

24AUG
WASTED CALLSAGENT INDEX

Coding agents waste most tool calls hunting for docs. Firecrawl launched an index of 70 million READMEs, issues, and pull requests for agents. It beats general web search by 18 points on recall.

It holds artifacts, not web pages. Repo landing files, bug threads, merge requests, API specs. Refreshed every day. No source code is stored.

An open benchmark, DevDex, scores 1,179 real developer questions. The new layer hit 0.63. Parallel got 0.57. Context7 managed 0.17.

Claude Code, Cursor, and Codex can query it now. Two credits per ten results. No key needed to start.

full brief & sources

⚡ Why this matters

  • Retrieval quality is now the ceiling on coding-agent quality, not model quality.
  • The 0.45 to 0.63 recall gap is the size of the problem general web search leaves on the table.
  • A public benchmark means agent retrieval can finally be shopped and compared like a model.

🔍 What happened

  • August 20, 2026: Firecrawl shipped Developer Index, a retrieval layer built for coding agents.
  • 70 million-plus artifacts: READMEs, external docs, issues, pull requests, OpenAPI specs, skill files. Most refreshed daily.
  • Reached through /search/developer or standard /search with a developer category. Stable IDs like issue:owner/repo#123.
  • DevDex ships alongside: 1,179 developer queries across repo discovery, docs lookup, and issue resolution. Half the dataset and the harness are open source.
  • Scores: Firecrawl Developer Index 0.63, Firecrawl Search 0.58, Parallel 0.57, Mintlify and Exa 0.54, native web search 0.45, Context7 0.17.
  • No API key needed to start. Two credits per ten results.

💬 Smart takes

  • Neha Patil, research engineer at Firecrawl: a coding agent does not want a web page, it wants an artifact.
  • Firecrawl on the control group: the gap between no tools and every other row is the size of the retrieval problem.
  • Skeptic: the benchmark was built by the vendor that wins it, and Parallel still beats it on repository discovery at 0.82 to 0.76.

🧭 Where this goes

  1. Likelyrival providers publish DevDex numbers within a quarter, because the harness is open.
  2. Likelyagent harnesses start shipping a default retrieval provider the way they ship a default model.
  3. PossibleGitHub responds with its own agent-facing artifact API rather than letting a third party index it.
  4. Wild Cardretrieval recall becomes a line item in enterprise agent contracts alongside token price and latency.

🥄 The Spoon Take

The interesting move is not the index. It is the benchmark. Firecrawl published the scoreboard it currently leads, which invites everyone to beat it and makes agent retrieval a measured category instead of a vibe. That is how a commodity layer turns into a market.

🤔 Pushback

Vendor-built benchmarks flatter their authors, and a 0.18 recall gap may not survive contact with agents that just retry three times.

·Firecrawl - Developer Index launch·Firecrawl - DevDex benchmark·Firecrawl docs - Developer Index