Nearly Half of AI Model Findings Are Already Outdated

Stanford University researchers Owen Quinn and James Zou tested 50 popular claims about AI language models. They ran the tests across 20 models from 8 different companies. 44% of these claims turned out to be false already. On average, a claim became outdated after 15.7 months. Two years out, 39% of the claims no longer held.

For example, the researchers looked at how often AI language models invent their own sources. Anthropic's Claude Sonnet 5.5 fabricated only 1% of its sources. Qwen3.7-Flash, a Chinese AI model from 2026 with strong benchmark scores, fabricated 69% of its sources.