Explainable AI is the prevention. Explainable AI is disappearing.

Google DeepMind researchers Rohin Shah and Anca Dragan wrote a paper on AI reasoning transparency. On their GPT-6 Astra system card they write “there was a large drop in our ability to follow along with the intermediate steps of reasoning.” And they warn us “future models may begin to think in ways that are impossible for us to understand (e.g., in numerical representations).”

I’ve found it helpful to see how Claude Code thinks so I can identify which step he went wrong in. This allows me to correct him. If we can’t see how an AI is thinking then it will be harder to debug its mistakes.