Embodied AI

Embodied AI is AI with a body: a model that controls a robot or a character in a simulation, and learns by seeing and acting in its surroundings. Where a chatbot writes text, an embodied system has to grasp an object it has never seen. The promise for businesses: a robot that takes on a new task from an instruction, without reprogramming. For now that is mostly research.

Where does the idea come from?

In 1991 Rodney Brooks of MIT argued that AI should be built step by step in robots that deal with the real world through perception and action. His lab's mobile robots moved around ordinary offices without supervision (Brooks). A 2021 survey by Duan et al. describes the shift from "internet AI", which learns from collected images, videos and text, to embodied AI, in which an agent learns by interacting with its environment from its own point of view, as people do. Much of that work starts in simulators, which save cost and time and are safer than a real robot (Liu et al., 2024).

How does it work?

Many current systems use a VLA model (vision-language-action): it takes camera images and an instruction and outputs movements. Google DeepMind introduced the term with RT-2 in July 2023. It trained a vision-language model further on robot data and wrote each movement as numbers in text tokens, the same way the model writes words. Knowledge from the web then carries over to the robot. RT-2 carried out instructions that never appeared in its robot data, such as moving a can "to the person with glasses". On such tests the best RT-2 model reached more than three times the average success rate of its predecessor RT-1. On unseen objects, backgrounds and environments it did about twice as well.

NVIDIA's GR00T N1 (March 2025), an open model for humanoid robots, splits the work in two: a vision-language module interprets the scene and the instruction, and a second module generates fluid movements in real time. NVIDIA calls this field physical AI. In March 2026 an effort of 35 organisations, started by researchers from Johns Hopkins, the Technical University of Munich and NVIDIA, published Open-H-Embodiment: 778 hours of openly licensed data from surgical, ultrasound and colonoscopy robots, and GR00T-H, a VLA model for surgical tasks trained on about 600 hours of it.

Where does it go wrong?

• Robot data is scarce. Collecting it is slow and expensive, according to Liu et al. For GR00T N1, NVIDIA had 88 hours of its own recordings of people steering the robot, from which it generated 827 hours of video. Its simulation pipeline also produced 780,000 practice runs in 11 hours, the equivalent of 6,500 hours of human demonstrations.
• Simulation differs from reality. Models often overfit to the simulator, struggle with noise from real sensors and fail on unexpected events (Liu et al.).
• Long tasks remain hard. A command such as "clean the kitchen" breaks down into many steps, and current planners often fall short in varied settings (Liu et al.).
• The web does not teach new movements. RT-2's physical skills stayed limited to those in its robot data; the web knowledge taught it where to apply them.

Example

In the RT-2 paper a robot arm stands at a table with fruit and bowls and gets the instruction "put strawberry into the correct bowl", a task that never appeared in its robot data. To do it, the model has to know what a strawberry is and that it belongs with the other fruit, which it learned from web data. The grasping and placing come from the robot demonstrations.