AI Trend Research

← AI glossary

AI glossary · Basic · Language models

Backpropagation

Backpropagation is a method that passes the difference (error) between a neural network's answer and the correct answer backward from the output layer toward the input layer, calculating how much each weight is responsible for the error. The network is trained by adjusting the weights based on these values.

About this term
Level
Basic · Technical terms you often meet in AI news
Added
2026-10-08
In plain words It is like a team project that turned out badly: you trace back from the last person in charge, work out 'whose decisions affected the result and by how much', and give each person feedback to do things a little differently next time.

How it works

First, the input is passed through the network from front to back to get a prediction (forward pass). The difference between the prediction and the correct answer is calculated with a loss function, turning 'how wrong it was' into a single number. Then this error is passed backward from the output layer to earlier layers, calculating how much the error changes when each weight is changed slightly (the gradient). Finally, gradient descent adjusts the weights little by little in the direction that reduces the error, and this process is repeated countless times.

The training loop of forward pass and backpropagationInput data → Neural network forward pass; Neural network forward pass → Prediction; Prediction → Loss calculation; Correct answer → Loss calculation(Compare); Loss calculation → Backpropagation(Error); Backpropagation → Weight update(Gradient); Weight update → Neural network forward pass(Repeat)Input dataNeural networkforward passFront→backcomputationPredictionCorrect answerLoss calculationHow wrong was itBackpropagationBack→front gradientcomputationWeight updateGradient descentCompareErrorGradientRepeat
The training loop of forward pass and backpropagation The top row is the forward pass that produces an answer, and the bottom row is backpropagation, which sends the error backward to adjust the weights. The loop repeats by running the forward pass again with the adjusted weights.

Why it matters

A neural network with many layers has an enormous number of weights, so testing and adjusting each one separately is nearly impossible. Backpropagation efficiently computes the gradients of all weights in a single backward calculation. This made it practical to train deep neural networks, and today almost all deep learning models, including LLMs, are trained this way.

Things to know

Backpropagation is 'a method for computing gradients'; actually updating the weights with those gradients is the job of an optimization method such as gradient descent. When a network is very deep, the signal passed backward can shrink (vanishing gradients) or grow (exploding gradients), and this is mitigated through the choice of activation function or architecture design. It is used only during training; in the inference stage, where the trained model is used, only the forward pass is run.

Example

If an image classification model wrongly labels a cat photo as 'dog', backpropagation traces the error backward and calculates which weights tilted the answer toward 'dog'. In deep learning tools such as PyTorch, this calculation happens automatically, so developers run backpropagation with a single line of code after computing the loss.

Related terms

Gradient descent Loss function Artificial neural network Activation function Deep learning Perceptron