AI glossary · Advanced · Safety & alignment
Interpretability
Interpretability is the research field that works out, in a way people can understand, why an AI model gave a particular answer and what computation actually happens inside the model.
About this term
- Level
- Advanced · Specialist terms that need the inner workings
- Category
- Security & safety
- Fields
- Safety & alignment
- Added
- 2026-10-08
How it works
A deep learning model takes input and produces an answer by passing it through a huge number of parameters (the numbers inside the model). Interpretability research looks at this intermediate process to find which neurons or internal signals (Representations) are linked to specific concepts, such as 'city names' or 'lying'. Some methods highlight which inputs most influenced the answer, and others deliberately change internal signals to see how the results change. Recently, mechanistic interpretability research, which tries to find the 'circuits' through which information flows inside a model, has been active.
Why it matters
The more AI is used for important decisions in areas like medicine, finance and law, the more it needs to be able to explain 'why it made that judgment'. A model may give the right answer on the surface while relying on irrelevant cues underneath, and catching this requires looking inside. It also helps find and fix the causes of problems such as hallucination, sycophancy and reward hacking. That is why interpretability is considered a key tool in alignment research, which aims to make AI safe.
Things to know
Today's large models are so big and complex that we have not yet reached the point of fully understanding their internals. There is also no guarantee that the explanations a model gives itself (e.g., a thought process written as CoT) match its actual internal computation. In addition, internals rarely divide neatly into 'this neuron means this concept'; often one part handles several concepts at once, which makes interpretation tricky. So it is safer to treat interpretability findings as 'partial clues'.
Example
AI research companies have used interpretability research to find internal signals inside large language models that respond to specific concepts, and have published how the models' behavior changes when those signals are deliberately amplified or suppressed. In industry, interpretability techniques are used in services that require accountability, such as loan screening or fraud detection, as features that show which input fields a model based its judgment on.