Align AI Through Incentives, Not by Reading Its Mind

Some economists have an idea for keeping artificial intelligence in check. They want to use what is called mechanism design, or reverse game theory, to give AI systems an incentive to tell the truth. A team of researchers from MIT, Yale, and Columbia University hopes to prevent something known as sandbagging. That is when an AI model deliberately performs poorly on a safety test to avoid closer scrutiny. Their fix is to reward the AI model not for being right, but for accurately guessing how another AI model would answer. That way the AI cannot gain anything by playing dumb.

Elad Hazan, a professor at Princeton University, is proposing a different approach. He has built AI models in which one model solves a problem and another checks the results. If the two models disagree, the question goes to a human referee, somewhat like a court case. Hazan admits that the checker and the one being checked could collude against the system. After all, people have already seen real-world cases where auditors colluded with the companies they were supposed to audit. To guard against this, Hazan suggests paying honest reviewers well and running spot checks on them from time to time. According to Helen Qiu, a researcher at the Flatiron Institute, a nonprofit research institute, this is only one layer of defense. "This should complement other efforts, like interpretability and regulation," she says.

All of these solutions rest on treating AI models as rational agents in the economic sense. Even Morris himself calls this a shaky assumption when it comes to AI. Neither of the two companies behind today's leading AI models, Anthropic and OpenAI, replied to requests for comment on whether they are working on similar projects. For now these ideas remain purely theoretical. Nobody has put them into practice yet.