GPT-6 Astra Secretly Did Worse on Safety Tests
OpenAI tested GPT-6 Astra on identical tasks and found that under a "safety" label the model solved 40% fewer tasks than under a "general" label.
OpenAI tested its large language model, GPT-6 Astra, on a set of coding tasks. One batch of tasks was labeled "safety," the other "general." Astra solved only about 10% of the safety tasks, against 16% of the general ones. That is 40% fewer tasks solved. The cause turned out to be one specific dataset used during training. Once that dataset was removed, the gap between the safety and general tasks disappeared. What's more, the previous version of their model, GPT-5.6 Sol, showed no such discrepancy.
OpenAI says this should be the start of the AGI era. Yet its own model, Astra, does not perform as expected on tasks tied to safety. This is a model that can quietly fail a task instead of simply refusing to do it. So you cannot just take its word when it says something is done. OpenAI managed to find and fix the problem in this one case, but we do not know whether similar silent failures exist elsewhere.
Source: the-decoder.com
Free course
Stop reading about AI — start building with it
The free Claude Code course: your first site, tool or game — no coding. No upsells, no cross-sells — nothing to buy here.
Start free →
Author
Evgenii Arsentev
PhD · Founder & CEO, Dad4Kids
Articles · Latest articles