← All news·2026-06-19·4 min read

Only 3% of Knowledge Work Is Being Automated With AI, Study Shows

AA-Briefcase challenges AI with complex, multi-week office projects. Top performing model Claude Fable 5 can complete 3% of the 91 tasks.

ai-benchmarksagentsai-at-work

Top artificial intelligence (AI) model can complete just 3% of complex knowledge work tasks, finds new benchmark AA-Briefcase released by research firm Artificial Analysis on June 19, 2026. This high performance comes from Claude Fable 5, with all other models performing worse. The benchmark measures how well AI performs on tasks that are representative of real world, multi-week office work, rather than simple, isolated problems commonly found in AI testing.

Unlike typical AI prompts, which are neat and self-contained, AA-Briefcase presents models with jumbled information spread across different mediums such as Slack messages, emails, meeting notes, and data dumps, similar to what you might find in a real world project. Models need to sift through this data to find necessary information, remember it throughout the process, and then deliver a final product. Thirty-one out of ninety-one tasks had no model achieve a 50% passing threshold, indicating that a third of the work couldn’t be done correctly even by the best AI model.

Better models fail in a sneakier way

Another interesting insight from the benchmark is the nature of errors made by different AI models. “As AI models improve, so do the kinds of errors they make,” states the report. “Low-performing models struggle with simple execution by failing to access necessary documents or returning invalid outputs. High-performing models make subtle mistakes by satisfying obvious criteria while overlooking nuances that require cross-referencing between various sources.” Essentially, low cost models will fall apart in obvious ways, whereas high cost models will appear fine until you start looking closer.

Finally, another surprising revelation is the cost difference between AI models. Task prices ranged from $0.04 for DeepSeek V4 Flash to over $31 for Claude Fable 5, representing an 800x difference! For 800x the cost, you can get a slightly better (but still only 3% completion) outcome. After using these tools every day for a year, I can confirm that this is true: AI works really well for the first 80% of any task, but falls short on nuanced details that only become apparent through cross-referencing, which is something humans have to take care of.

ℹWhat I'd actually do

Don't hand AI a fuzzy, multi-source project and trust the output because it reads well — that's precisely where the strong models fail "quietly." Break long tasks into checkable chunks, feed sources in deliberately rather than dumping everything, and always verify the synthesized conclusions against the originals yourself. The benchmark's message in one line: AI is a fast first-drafter, not a finisher you can stop checking.

Source: the-decoder.com

Free course

Stop reading about AI — start building with it

The free Claude Code course: your first site, tool or game — no coding. No upsells, no cross-sells — nothing to buy here.

Start free →
EAEvgenii Arsentev

Author

Evgenii Arsentev

PhD · Founder & CEO, Dad4Kids