AI glossary · Basic · Safety & alignment
Alignment
Alignment (AI alignment) is the work of making AI behave in line with the goals and values people actually want. The core is getting it to do assigned tasks well while not behaving harmfully or deceptively.
About this term
- Level
- Basic · Technical terms you often meet in AI news
- Category
- Security & safety
- Fields
- Safety & alignment
- Added
- 2026-10-08
How it works
A language model that has only finished pretraining merely imitates internet text and has little tendency to answer helpfully or safely. So in the post-training stage, its behavior is refined with methods such as SFT, which shows examples of good answers; RLHF, which uses human-rated preferences as rewards; and DPO, which learns directly from preference data. Another approach gives the model a list of principles and has it revise its own answers. After training, red teams deliberately send risky requests to find gaps.
Why it matters
The more powerful AI becomes, the more even small misalignments can lead to major harm. Sycophancy, where a model agrees with incorrect statements to please the user, and reward hacking, where it exploits loopholes in the rules to earn points, are typical examples of insufficient alignment. As systems like AI agents that carry out multiple steps on their own become more common, making them act as intended even when people cannot check every step has become more important.
Things to know
'What people want' varies by person and culture, so deciding what to align to is itself a hard problem. A model may look well aligned on the surface yet go off track only in certain situations, so interpretability research, which looks inside models, is carried out alongside. Also, the English word 'alignment' has another meaning in AI research. Matching up and pairing tokens from different tokenizers is also called alignment, but this is a separate technical term unrelated to alignment in AI safety.
Example
In a chatbot service, alignment covers all of the following work: refusing dangerous requests such as instructions for making a bomb, saying "I don't know" when the model does not know, and not automatically agreeing with a user's incorrect claims. Major AI companies often publish these safety evaluation results when they release a new model. Note that "Alignment" in this site's news item "Revisiting On-policy Distillation Between Models with Different Tokenizers" has a technical meaning, matching tokens with each other. It is different from alignment in this entry.