Nvidia Trains Robots to Learn Using Coded Agents

A team from Nvidia, Carnegie Mellon University and UC Berkeley developed a system named ENPIRE in which robots learn to manipulate objects autonomously with minimal human involvement. A group of 8 such robots achieved up to 99% success rate on complex tasks.

The key idea behind ENPIRE is that instead of having humans manually adjust hyperparameters, the job of teaching robots is done by AI coding agents. These agents automatically modify the training code of the robots. In particular, they first configure the environment by defining safety constraints, resetting automatically after each try and verifying automatically whether the current try was successful or not. In traditional settings, humans would manually inspect every try to determine its success. Instead, here the agent learns to define a reward function that distinguishes between successful and unsuccessful tries by observing just a few minutes of demonstration videos. Then, the agent reads scientific papers, generates hypotheses and modifies the training code accordingly. Depending on the current situation, the agent decides whether to use behavior cloning or reinforcement learning.

1Eight robots that share notes over Git

The team deployed 8 dual-arm YAM robot stations which communicate with each other using Git, a version control system commonly used by software developers. All stations run hypotheses in parallel and distribute successful training scripts throughout the group. This means that if one robot finds a good way to solve a problem, other robots can benefit from it as well. As the results show, this approach significantly improves performance. For example, in the ‘Push-T’ task (pushing a T-shaped block into place) 8 agents reduced the required time from around 5 hours to 2 hours. When inserting pins, the system achieved 100% speedup compared to human-in-the-loop approaches, reducing the time needed from over 90 minutes to around 40 minutes. They also tried more complicated tasks like cutting cable ties and inserting a GPU into a motherboard socket.

2Why this matters

Over the last year, we’ve seen coding agents slowly replacing humans in software development tasks. With ENPIRE, we’re starting to see evidence of these same agents entering the physical world — writing code to train machines to perform manual tasks. Of course, there are some caveats here, which are clearly stated in the paper, and I respect that. For instance, the current performance in the real world lags behind simulations due to uncertainties related to robot dynamics and friction. Also, the robots spend much time idling while the agent writes code and reports findings. Finally, the cost of tokens increases faster than the benefits obtained by scaling up the size of the robot team. Therefore, this is still a research project, but the process described here — where an AI reads papers, generates hypotheses, writes code, runs experiments on physical devices and spreads successful outcomes across the group — is precisely the type of self-improving system that becomes cheaper and faster each year. One of the main challenges in robotics is gathering enough real-world training data. By offloading this tedious task to relentless agents, we might finally be able to overcome it.