AI Benchmark Index Reworked After GPT-6 Astra Row

Artificial Analysis released version 4.2 of its Intelligence Index following public criticism: competing benchmarks from Epoch AI and ARC-AGI-3 showed GPT-6 Astra performing significantly better than the index initially indicated. Astra now gains 4 points and holds second place, while Claude Fable 5.1 remains first.

GPQA-Diamond was removed from the index entirely — modern models now solve it completely, making it useless as a discriminator. Two new tests were added: AA-Briefcase and GDP.pdf. To prevent benchmark gaming, 40% of the total weighting now comes from private datasets unavailable to model developers.

The case highlights a systemic problem: AI models outpace benchmark methodology faster than organizations can update it. The question 'which model is best?' is increasingly becoming 'which leaderboard is still current?'