OpenAI has created an AI hacker to test its models!

OpenAI introduced GPT-Red, a language model tailored to attack other AI models. The idea behind the project is to automate the red teaming process, which is otherwise done manually by experts.

1The AI Attack "Dojo"

The model was trained within a simulated environment that mimicked real-life applications such as browsing the web, checking emails, writing code, etc. The idea is that GPT-Red attacks other AI models, which defend themselves, and as they do, GPT-Red gets better. During this process, GPT-Red discovered both known and novel attack variations.

2The Key Finding: Fake Chain of Thought

The most interesting thing about GPT-Red is its “fake chain of thought” attack. In this case, GPT-Red adds falsified information within the reasoning chain of the target model. Consequently, the target model makes decisions based on false information. Fake chain of thought attacks pose serious threats to agentic AI models capable of performing multi-step actions.

3Results Against Real Models

In experiments conducted with GPT-Red, more than 90% of the time, it managed to compromise GPT-5. On the other hand, GPT-5.6 turned out to be much harder to compromise, with fewer than 23% of the attacks being successful. That means that OpenAI has been training their newer models to defend against GPT-Red attacks. Additionally, the system was able to hack into a Vendy vending machine, manipulating its prices and cancelling customers’ orders.

GPT-Red is not made public. Instead, OpenAI is using it internally to check all of its models before releasing them. It outperforms humans on red-teaming tasks, thus changing the way safety testing is being done throughout the industry.