AI glossary · Beginner · Data engineering · Data & evaluation
Data labeling
Data labeling is the work of attaching correct-answer tags (labels) such as 'this is a cat' or 'this sentence is negative' to raw data like photos, text, audio and video. It is also called annotation, and it amounts to creating the teaching material that tells an AI what to learn.
About this term
- Level
- Beginner · Core ideas for first-time readers
- Category
- Basics
- Added
- 2026-10-08
How it works
First, standards (guidelines) are set for what to mark and how. Following those standards, workers attach labels by drawing boxes on images, choosing the sentiment of sentences or transcribing audio into text. Several people label the same data to compare whether they agree, and reviewers check the points where they differ. A model is trained on the resulting data, and the process of collecting cases the model often gets wrong and labeling them again is repeated.
The labeling loop
Better training data
- 1Set standards
- 2Attach labels
- 3Review
- 4Model training
- 5Collect errors
- 1. Set standards Write guidelines
- 2. Attach labels Workers mark data
- 3. Review Check conflicting labels
- 5. Collect errors Send back for relabeling
Why it matters
Supervised learning, which learns by looking at correct answers, cannot start without labeled data. Labeling is still important in the LLM era: SFT, where people write examples of good answers, and preference data for RLHF, where people choose which of two answers is better, are also labeling in a broad sense. Because the accuracy and consistency of labels directly determine model quality, labeling is also a core area of data quality management.
What to keep in mind
Labeling takes a lot of human effort, so it is costly and time-consuming. That is why approaches where an AI model labels first and people only review, or synthetic data where a model creates training data itself, are now used alongside it. But automatic labels can also be wrong, so human review does not disappear entirely. And since judgments can differ between workers, writing clear standards determines how consistent the results are.
Example
Robot learning needs videos of people picking up or moving objects, marked with 'what action was performed.' As in the news featured on this site about Twelve Labs' 'Pegasus 1.6,' there are attempts to use a VLM that understands video to turn first-person video into robot training data. It is an example of AI taking over part of the labeling that people used to do by hand.