Dreamer Agents Learn the World Model Rather Than Playing Millions of Games.

Danijar Hafner is a 31-year-old researcher from a small town in Germany. Self-taught, he started programming at his neighbour’s house. In 2015, he worked as a student at Google Brain, alongside Geoffrey Hinton and Ashish Vaswani. Today, he is building a startup in San Francisco, in secret, and MIT Technology Review has dubbed him one of the most important people who will shape the future of AI agents.

1The Dreamer Series: From Atari to Minecraft

He developed the Dreamer agents, which use so-called “world models” — an internal model of the outside world that can be used by an agent to make decisions. PlaNet was the first agent to learn to plan. Dreamer 2 achieved human-level performance in Atari 2600 games without directly interacting with the environment by using world models. Dreamer 3 was the first agent to autonomously find a diamond in Minecraft, something that the AI community thought would be impossible to automate. Dreamer 4 even managed to learn to mine diamonds in Minecraft by watching videos, without ever interacting with the game itself.

2DayDreamer and Real-World Robots

With DayDreamer, he brought the method into the real world: he taught robots to act in new environments and unexpected scenarios. And this is what he’s now doing in his startup: developing humanoid robots that can move through apartments with unknown layouts and furniture. His ex-boss at Google, Timothy Lillicrap, calls him “in the top half-percent of researchers and could single-handedly build what entire engineering teams need weeks to accomplish.”

3Why This Changes the Approach to Agents

Rather than going through millions of interactions in the same environment, world models allow agents to learn an internal model of the world they live in and act accordingly. So they can plan in new situations without needing to retrain themselves. That means that if we can make it work for robots, we won’t need to have a chasm between training in simulation and being able to apply it to real-life scenarios.