World Models Don’t Have a Crystal Ball — Here’s Why They Often Fail

World models are AI agents that forecast future events (e.g., Sora/Genie). However, they lack human beliefs, goals, and emotions, resulting in incorrect predictions.

MENTIS is a framework that introduces 6 “mental variables” (beliefs, attention, goals, intentions, emotions, norms, and social relationships), which increases the accuracy of world models from 63.3% F1 to 87.9%.

Surprisingly, using a less powerful model with MENTIS (84.9%) outperforms a more powerful model without MENTIS (83.6%). Especially in interpersonal situations (+26.4 points). Humans achieve almost perfect performance (98.5%).

Most of the remaining errors come from state transition simulation, i.e., joint dynamics of physical and mental states. (80%)

This study shows that if your agent/robot does not know what humans want, it will continue to make errors, even when it has all the knowledge about the surrounding physical world. It’s time to incorporate “psychology” into world models for practical purposes.