# How robots learn from video of people working (egocentric data, teleoperation, explained)

> How robots learn from video of people working: egocentric data, imitation learning, teleoperation and world models explained, and what it means for a business.

Canonical: https://origindatapartners.com/physical-ai/how-robots-learn-from-video
By: Origin Data Partners
Status: Published
Published: 2026-11-05
Updated: 2026-11-05
Sources checked: 2026-11-05

## Quick answer

Robots learn physical skills by studying many examples of a task being done. The cheapest examples are egocentric video: footage from a camera on a person's head that shows their hands doing the work. Labs combine that video with smaller amounts of teleoperation data, where a person drives a robot directly, and with simulation. Variety matters most, so footage from thousands of different real workplaces is valuable, which is why companies now pay people and businesses to record.

## Key points

- Egocentric video is first-person footage with the hands in frame. It is cheap to collect and adds variety.
- Teleoperation data includes the robot's own motor commands. It is the highest quality but the most expensive.
- Models learn general skills from broad data first, then are fine-tuned on a specific task.

## Why do robots need video of people working?

Chatbots learned language from text on the web. Robots have no equivalent. [Physical Intelligence](https://www.pi.website/blog/pi0), a robotics lab, puts it plainly: there is no such treasure trove of robot data. To fold a shirt or fit a pipe, a robot has to learn from examples of the movement itself, across many objects and rooms. That is why chore apps, data collection jobs and business recording programs now exist.

Egocentric data is [video recorded from the perspective of the person doing the task](https://www.abc.net.au/news/2026-09-20/egocentric-training-gig-workers-helping-robots-learn-tasks/107135216), usually with a head- or chest-mounted camera that keeps the hands in frame. It shows what a third-person camera misses: how the hand approaches an object, the grip, the order of steps and the small corrections.

The best-known research dataset is [Ego4D](https://arxiv.org/abs/2110.07058), with 3,670 hours of daily-life video recorded by 931 people in 74 locations. Industry datasets now dwarf it. [Build AI](https://huggingface.co/datasets/builddotai/Egocentric-10K) released 10,000 hours recorded exclusively in real factories, and Figure says its Index app ingests 30 minutes of video every second.


## What are the main ways robots are trained?

Labs mix several kinds of data. Each has a different cost and use.

| Method | What it is | Strength | Weakness |
| --- | --- | --- | --- |
| Egocentric human video | People record real tasks with a head-mounted camera | Cheap, fast to scale, huge variety of places | Shows the hands, not the robot's motor commands |
| Teleoperation | A person drives a robot, and its sensors and commands are recorded | Highest quality: includes the exact robot actions | Expensive and slow; needs the robot on site |
| Motion capture | Suits or multi-camera rigs record body and hand position | Precise joint and finger positions | Specialist gear; mostly in labs |
| Simulation and world models | Software generates or predicts scenes and physics | Unlimited and safe to repeat | Never fully matches real physics and clutter |

Source: [Washington Post: how robots are trained](https://www.washingtonpost.com/technology/interactive/2026/robot-chores-video-data/)

## How does a robot learn from watching a person?

The core idea is imitation learning: show a model many demonstrations of a task, and it learns to predict the next action from what it sees. Modern systems, called vision-language-action models, take in camera images and a spoken or typed instruction and output movements. [Google DeepMind](https://deepmind.google/discover/blog/gemini-robotics-brings-ai-into-the-physical-world/) and Physical Intelligence both build models of this kind.

Training happens in two stages. [Physical Intelligence](https://www.pi.website/blog/pi0) describes pre-training as teaching the model about the physical world in general, and fine-tuning as making it do one task well. Human video helps most in the first stage, by showing the endless ways real people handle real objects.

It can go further. In 2025 [Figure](https://www.figure.ai/news/project-go-big) reported a robot that learned to navigate homes from human video alone, with no robot demonstrations.


## What is teleoperation, and why is it still used?

In teleoperation a person controls a robot directly, often with a VR headset or a matching set of arms, while the robot records what it sees and the commands it receives. A Georgia Tech researcher told the [Washington Post](https://www.washingtonpost.com/technology/interactive/2026/robot-chores-video-data/) it is probably considered the highest quality data, but it is the most expensive to collect. One estimate in [Forbes](https://www.forbes.com/sites/johnkoetsier/2026/08/26/figure-launches-gig-platform-to-get-humans-to-do-work-to-train-robots-will-spend-1-billion/) puts it at $50 to $200 an hour against $2 to $5 for human video.

Companies change their mix over time. Business Insider reported that [Tesla shifted Optimus training](https://www.businessinsider.com/tesla-musk-optimus-humanoid-robot-training-motion-capture-cameras-2025-8) in 2025 from motion capture suits and teleoperation toward recording videos of workers doing tasks, to scale data collection faster. The people doing that work are [data collection operators](https://origindatapartners.com/physical-ai/tesla-data-collection-operator).


## What are world models?

A world model is an AI system that predicts how a scene will change, for example what happens when a cup is pushed. [NVIDIA Cosmos](https://www.nvidia.com/en-us/ai/cosmos/) is one example: it can run as a physics-grounded simulator and generate training video. World models let labs multiply real data and test robots safely, but they are built and checked against real footage, so they add to the need for real video rather than replace it.


## Why do diverse real workplaces matter?

A robot trained in one kitchen fails in the next one. micro1's chief executive told [MIT Technology Review](https://www.technologyreview.com/2026/04/01/1134863/humanoid-data-training-gig-economy-2026-breakthrough-technology/) that you need lots and lots of variations for a robot to generalize. A robotics professor told ABC News that human egocentric data is cheaper than robot data and adds much-needed variety.

That is why footage from working businesses is in demand. Job sites, shop floors, commercial kitchens and warehouses each have their own tools, layouts and habits, and skilled workers move differently from someone doing a chore for the first time. Buyers want exactly that spread.


## What does this mean for a business with a hands-on crew?

Your crew's normal work is the kind of example robotics labs are short of. Amelia Robotics pays businesses for approved recordings of agreed tasks, typically paying $500 to $2,000 a month per recording employee. At its typical rate of about $10 per approved hour, a [manufacturing](https://origindatapartners.com/physical-ai/manufacturing) or [warehouse](https://origindatapartners.com/physical-ai/warehouses) team of 8 recording 20 approved hours a week each would bring in about $6,400 a month (8 x 20 x 4 x $10). That is illustrative; your agreement and approved hours set the real figure.

Source: [Amelia Robotics: for robotics labs](https://ameliarobotics.com/robotics-labs)

## Next step

See which tasks count and how the business program works on the [physical AI hub](https://origindatapartners.com/physical-ai).

Origin Data Partners is independent. We may be paid by our partner when a business we introduce is paid; it never comes out of your earnings. [How Origin is paid](https://origindatapartners.com/referral-disclosure).


## Questions and answers

### What is egocentric data?

First-person video or sensor data recorded from the point of view of the person doing a task, usually with a head-mounted camera that keeps their hands in frame.

### How do robots learn from videos of humans?

Models study many demonstrations and learn to predict the next action from what the camera sees. Broad human video teaches general skills; smaller amounts of robot data fine-tune specific tasks.

### What is robot teleoperation?

A person controls a robot remotely, often with a VR headset, while its sensors and commands are recorded. It is high-quality training data but expensive and slow to collect.

### Can a robot learn from video alone?

Partly. Figure reported a robot that learned home navigation from human video alone in 2025, but most manipulation skills still combine video with robot data.

### Why do AI companies want footage from real workplaces?

Robots must handle the variety of real tools, layouts and objects. Footage from many different working businesses supplies that variety in a way a single lab cannot.

### What is physical AI?

AI that perceives and acts in the physical world, such as robots and autonomous vehicles, as opposed to AI that only works with text or images.

## Sources

- [Amelia Robotics: business program (checked Nov 2026)](https://ameliarobotics.com/)
- [Amelia Robotics: for robotics labs](https://ameliarobotics.com/robotics-labs)
- [Ego4D: Around the World in 3,000 Hours of Egocentric Video (arXiv)](https://arxiv.org/abs/2110.07058)
- [Physical Intelligence: π0, our first generalist policy](https://www.pi.website/blog/pi0)
- [Google DeepMind: Gemini Robotics](https://deepmind.google/discover/blog/gemini-robotics-brings-ai-into-the-physical-world/)
- [Figure: Project Go-Big](https://www.figure.ai/news/project-go-big)
- [NVIDIA Cosmos](https://www.nvidia.com/en-us/ai/cosmos/)
- [Build AI: Egocentric-10K on Hugging Face](https://huggingface.co/datasets/builddotai/Egocentric-10K)
- [Washington Post: why tech companies pay people to do chores](https://www.washingtonpost.com/technology/interactive/2026/robot-chores-video-data/)
- [Forbes: Figure Index and data costs](https://www.forbes.com/sites/johnkoetsier/2026/08/26/figure-launches-gig-platform-to-get-humans-to-do-work-to-train-robots-will-spend-1-billion/)
- [Business Insider: Tesla shifts Optimus training](https://www.businessinsider.com/tesla-musk-optimus-humanoid-robot-training-motion-capture-cameras-2025-8)
- [MIT Technology Review: humanoid data gig work](https://www.technologyreview.com/2026/04/01/1134863/humanoid-data-training-gig-economy-2026-breakthrough-technology/)
- [ABC News (Australia): egocentric training explained](https://www.abc.net.au/news/2026-09-20/egocentric-training-gig-workers-helping-robots-learn-tasks/107135216)

## Related reading

- [/physical-ai](https://origindatapartners.com/physical-ai)
- [Companies that pay businesses and people to record work for AI and robot training (2026)](https://origindatapartners.com/physical-ai/companies-that-pay-to-record-work)
- [Tesla data collection operator (Optimus): pay, the job, and alternatives](https://origindatapartners.com/physical-ai/tesla-data-collection-operator)
- [Manufacturers and fabrication shops: get paid to record assembly and welding work](https://origindatapartners.com/physical-ai/manufacturing)
- [Do robotics companies pay warehouses for video of picking and packing?](https://origindatapartners.com/physical-ai/warehouses)
- [Can a company sell its video footage to AI and robotics companies?](https://origindatapartners.com/guides/sell-video-footage-physical-ai)