OpenAI Hid Their AI’s Thinking Process And Now The Safety Community Is Worried

OpenAI introduced the method “recurrent depth” (or opaque recurrence) in their Astra model. Instead of thinking stepwise like we are used to, it thinks cyclically. Thus, the model’s internal reasoning becomes much harder for us to observe.

1What Is Recurrent Depth

Standard reasoning models like o1 or o3 think stepwise and output their reasoning as plain text. With recurrent depth, some of the thinking happens in latent space (inside the model’s activations), without ever producing chains of thought that can be observed as plain text. That saves compute, but it also takes away our main instrument for oversight.

2Why Safety Researchers Are Sounding the Alarm

Buck Shlegeris, CEO of Redwood Research, stated that “If OpenAI continues to apply this technique, chain-of-thought completely loses its value as a monitoring tool.” His colleagues Ryan Greenblatt, chief scientist at Redwood, and Zvi Mowshowitz, analyst at Redwood, agreed. They argued that transparency of reasoning is an essential component of developing safe AI systems.

Chain-of-thought has become the standard way to interpret what OpenAI’s first reasoning models were doing. Once reasoning happens in latent space, we will lose the possibility to check whether the model was thinking correctly, even if it gave the correct answer.

3OpenAI's Position

Jakub Pachocki, Chief Scientist at OpenAI, wrote that “we have been working to preserve and make use of chain-of-thought monitoring since our very first reasoning models”. In Astra, recurrent depth is currently only used in a restricted way and readily interpretable chains of thought are preserved. Nevertheless, experts in AI safety believe that introducing this technique sets a precedent that might be built upon in future models.