Google Finds a Way to Stop AI Agents From Memorizing Tests

A team at Google Cloud AI Research, working with several universities, built a technique called RRSI. It targets a known problem in self-improving AI agents. When an agent changes its own scaffolding — prompts, tools, the order of its steps — it tends to start fitting the specific test scenarios instead of actually learning to handle new tasks. With RRSI, the team reported a gain of up to 14.1 points on the training set. On five new benchmarks, the gain reached up to 4.7 points. The biggest jump came on JobBench. They also cut token usage by about 30% compared to the unregularized version of the system, measured on these same benchmarks.

One interesting detail: the underlying model never changed during the experiment. It was Claude Opus 4.8 from Anthropic the whole time. Only the scaffolding around the model got better. The roughly 30% drop in token use is calculated against the unregularized version, specifically on these benchmarks.