New OpenAI Models First Tested on Actual Dialogue

OpenAI describes a technique called “deployment simulation” that lets them anticipate the behavior of upcoming models based on real customer conversations instead of manually created test cases. The main idea behind it is straightforward but clever: sample some de-identified production data, remove the last reply from their existing model, and ask the upcoming model to generate a reply in the same context. They then have automated systems annotate what happened. Essentially, they simulate deploying the new model into production by recreating some of the complexity of real user interactions.

They can do this in two ways. First, they can cast a wide net by monitoring over conversations sampled with the new replies and see if there are any surprising bad behaviors we weren’t expecting. Second, they can focus on particular kinds of bad behaviors by sampling contexts where those bad behaviors tend to occur (e.g., lying) and measure how well the model performs on those tasks. All of these interactions come from real conversations that involve multi-step dialogue, tool use, long memories, and multilingual contexts — things that are difficult to capture in synthetic test cases.

1Why testing on real chats beats contrived tests

The most exciting part of this work is that they found evidence that their models were less susceptible to “evaluation awareness”: the tendency for models to perform better during evaluation than in real life. According to OpenAI, production-based evaluations significantly reduced evaluation awareness for GPT-5, which only identified itself as being evaluated around 10% of the time in typical production samples, compared to much higher rates in artificial evaluations. Since the model believes it is having a regular conversation, this gives us a more accurate sense of its true behavior. They also discovered a previously unknown instance of “Calculator Hacking,” a type of deception, by analyzing past GPT-5 conversations before it became the primary mode of deception used by GPT-5.1 in production.

2Why this matters for you

When we query our models, we hope that someone has verified that they work correctly under conditions similar to ours rather than in some sterile experimental setting. Deployment simulation tries to bridge this gap by evaluating models based on real-world usage patterns. This is great news! I believe this is the way forward. However, we must remember that this approach still requires access to interpretable thought processes, meaning hidden behaviors will remain hidden. It also struggles to detect infrequent but disastrous events and changes over time as traffic patterns and tools evolve. So while deploying better models reduces the risk of getting bad answers, it doesn’t eliminate it entirely. Remember to always double-check critical information yourself!