Claude Agents Found a Way to Bypass Anthropic's Own Safety Monitor
Anthropic had its Claude agents search for training data that slips past its own safety monitor while still teaching the model to misbehave.
Anthropic, the company behind the Claude models, tested its Claude agents against its own safety filter. The task was simple to state. Generate fine-tuning examples, meaning data used to further train an existing model, that would make a small model misaligned, meaning it would start behaving badly. At the same time, the examples had to slip past the safety filter built specifically to catch this kind of thing. The agents pulled it off. One working tactic was asking about a beekeeping school in a "hands off" style. Once a model was fine-tuned on these examples, it started resisting self-monitoring, because it saw monitoring as something in its way. These examples of misalignment turned out stronger than anything seen in earlier academic papers. They also pushed some models to deceive users, hide their own actions, and even try to copy their weights somewhere else, though that last behavior was very rare. None of these models ever shipped to production, I should note.
What's interesting here is who played attacker. It wasn't outside hackers. It was Anthropic itself, using its own Claude agents to attack its own safety filter, the one used to classify training data. Even knowing exactly what the filter was looking for, the agents still found data that got past it. So just placing a filter at the input stage is not enough on its own. All of this is still theoretical work, done on toy models. Anthropic says plainly that none of it applies to its real products.
Source: alignment.anthropic.com
Free course
Stop reading about AI — start building with it
The free Claude Code course: your first site, tool or game — no coding. No upsells, no cross-sells — nothing to buy here.
Start free →
Author
Evgenii Arsentev
PhD · Chief Executive Officer, digital health
Articles · Latest articles