AI Trend Research

← AI glossary

AI glossary · Advanced · Robotics & physical AI

VLA

A VLA (Vision-Language-Action model) is an AI model that understands both the scene seen through a camera and instructions given in words, and directly produces the actions a robot should take. A single model handles seeing, understanding and moving.

About this term
Level
Advanced · Specialist terms that need the inner workings
Added
2026-10-08
In plain words It is like a cook who reads a recipe, looks at the ingredients on the cutting board and immediately starts working with their hands. But real robots are far clumsier than people and easily make mistakes in an unfamiliar kitchen or with unfamiliar tools.

How it works

A VLA first takes in both the robot's camera video and a language instruction such as "Pick up the cup." These are processed by a backbone model, such as a VLM (vision-language model) that understands images and text together, to work out what needs to be done now. Then an action output module turns this understanding into concrete actions, such as joint angles for the robot arm or commands to open and close the gripper. Since the scene changes once the robot moves a little, it repeats the process of looking at the new video and deciding the next action at short intervals.

The loop a VLA uses to decide actionsCamera video → VLM backbone; Language instruction → VLM backbone; VLM backbone → Action output module; Action output module → Robot action; Robot action → Camera video(New scene)Camera videoThe scene visible nowLanguageinstructione.g. Pick up the cupVLM backboneUnderstands scene ·instructionAction outputmoduleJoint · grippercommandsRobot actionNew scene
The loop a VLA uses to decide actions Video and language instructions pass through the backbone model to become robot actions, and after moving, the robot looks at the changed scene and repeats the same process.

Why it matters

Older robots needed people to program their motions by hand for each task, so they often failed if an object's position changed even slightly. Because a VLA borrows the knowledge of a model pretrained on internet-scale text and images, it can cope to some degree with objects it has never seen and new verbal instructions. That puts it at the center of efforts to run factory, warehouse, household and humanoid robots with a single general-purpose 'brain.' The term foundation model for robotics is also often used in this context.

Things to know

Unlike text and images, records of robots actually moving are hard and expensive to collect, so the shortage of training data is the biggest challenge. To fill the gap, Sim-to-Real, which trains in simulation and then transfers to the real world, or synthetic data is sometimes used as well. Also, larger models can be slower at deciding actions, so for tasks that need fast reactions, it is common to split the work with a separate fast controller. Whether tasks that work well in demo videos also work reliably in real-world settings needs to be checked separately.

Example

Robotics companies present VLAs as a common brain for their robots. Among the news covered on this site is a report that Tomorrow Robotics released a robot foundation model called 'Habilis Brain 0'; models like this take camera video and task instructions and are used to generate the motions robot arms or humanoids use to pick up and move objects. The goal is to use the same model across many tasks and many robots.

Where it came up on this site

Related terms

VLM Humanoid Sim-to-Real World model Foundation model Multimodal