AI Trend Research

← AI glossary

AI glossary · Advanced · Data & evaluation · Data engineering

Weak Supervision

Weak supervision is a method in which, instead of people labeling each data point one by one, many 'weak' signals of uneven accuracy, such as rules, keywords and existing models, are used to produce correct answers (labels) in bulk, and a model is then trained on them.

About this term
Level
Advanced · Specialist terms that need the inner workings
Added
2026-10-09
In plain words It is like having several grading assistants, each of whom sometimes makes mistakes, mark a stack of exams and then combining their marks into a score, instead of having one expert grade every exam. However, if the assistants all make the same mistake, the wrong answer remains even after combining.

How it works

First, people write several simple rules called 'labeling functions.' A short rule such as 'if a sentence contains the word refund, it is a complaint,' an existing dictionary or database, or another model's predictions can all serve as labeling functions. Each rule gives answers for only part of the data, and the rules sometimes disagree. Next, a 'label model' looks at how often the rules are right and how much they overlap, then decides the most likely label for each data point along with how confident it is. Finally, training the model you actually want to use on these labels can give you a model that generalizes even to cases the rules did not cover.

How weak supervision flowsUnlabeled data → Keyword rules; Unlabeled data → Dictionaries, databases; Unlabeled data → Existing models, LLMs; Keyword rules → Label model; Dictionaries, databases → Label model; Existing models, LLMs → Label model; Label model → Final model training(Labels); Validation data → Final model training(Quality check)Unlabeled dataKeyword rulesDictionaries,databasesExisting models,LLMsLabel modelCombines signals,confidenceFinal modeltrainingValidation dataChecked by peopleLabelsQuality check
How weak supervision flows Several weak signals each produce labels, a label model combines them into training labels, and those labels are used to train the final model.

Why it matters

Supervised learning needs a lot of labeled data, but in fields that require expert knowledge, such as medicine, law or internal company documents, labeling is very expensive. Weak supervision lets expert knowledge be captured as 'rules' rather than applied one data point at a time, so it can be applied to large amounts of data at once. Because fixing a rule regenerates all the labels, there is no need to relabel from scratch when the criteria change. These days it is also common to have an LLM assign labels and use the result as one of the weak signals.

Good to know

The resulting labels contain noise, so a small validation set checked directly by people is needed to measure quality. If all the rules lean in the same direction, the label model inherits that bias as is. Rare cases that the rules seldom cover have few labels, so the model tends to be weak on them. For this reason, weak supervision is often used together with methods such as active learning, which pick out the cases people really need to look at.

Example

Suppose a company wants to automatically sort customer service inquiries into 'shipping,' 'refund' and 'technical issue.' Instead of having people sort tens of thousands of inquiries, staff write a few dozen keyword rules and add existing FAQ matching results and an LLM's classifications as signals. The label model combines these signals to create training labels, the team checks accuracy against a few hundred items verified by people, and then trains a classification model and puts it into service.

Related terms

Supervised learning Data labeling Active Learning Synthetic data LLM-as-a-Judge Data quality