Real-SWE: Best AI Agent Solved 38.8% of Tasks on Real Enterprise Code

Specific Labs released Real-SWE — a benchmark that tests AI agents not on practice problems but on the private code of real companies, which the lab licensed. The top result belongs to Fable 5.1 in Claude Code: 38.8% of tasks solved, followed by GPT-6 Astra at 33.8% and Gemini 3.8 Flash at 31.2%. An attempt with Fable 5.1 costs $6.96; with Gemini 3.8 Flash it's $2.50 — and the authors state plainly: more expensive does not mean more successful.

Even the best agent on real-world code closes fewer than two out of five tasks, so verifying the output is part of the unit economics — not an optional step. And you should be calculating cost per solved task, not cost per attempt: a cheap run that misses more often can easily end up costing more.