AI Trend Research

← AI glossary

AI glossary · Advanced · Language models · Efficiency & infra

LoRA

LoRA (Low-Rank Adaptation) is a fine-tuning method that leaves a huge AI model's original parameters unchanged and trains only a very small add-on component alongside them to adapt the model to a new use.

About this term
Level
Advanced · Specialist terms that need the inner workings
Added
2026-10-08
In plain words It is like slipping thin sticky notes with just the needed content into a thick textbook instead of rewriting the whole book. By swapping the notes, the same textbook can be used for different subjects. The limitation is that notes cannot greatly change the textbook's core content.

How it works

Standard fine-tuning slightly adjusts all of a model's billions of parameters. LoRA freezes (fixes) the original parameters, attaches an adapter made of two small matrices beside each layer, and trains only that. 'Low-rank' here means expressing a large change compactly with very few numbers. At use time, the adapter's output is added to the original model's computation to produce the final output. A trained adapter can also be merged into the original model.

Structure of LoRAInput → Original parameters; Input → Adapter A; Adapter A → Adapter B(Training); Original parameters → Add results; Adapter B → Add resultsTrained part · adapterInputOriginal parametersFrozen, not trainedAdapter AShrink downAdapter BExpand backAdd resultsFinal outputTraining
Structure of LoRA The input passes through both the frozen original parameters and the small adapter, and the two results are added to form the output. Only the adapter below is trained.

Why it matters

Because the number of parameters to train drops sharply, it takes far less GPU memory and cost. As a result, even organizations that are not large companies can adapt open-weight models to their own data. Adapter files are much smaller than the original model, so they are easy to store and share. You can also run use-specific models by swapping multiple adapters on a single base model.

Things to know

LoRA is effective for learning tone, format or ways of handling specific tasks, but it may be limited for putting large amounts of entirely new knowledge into a model. Setting the adapter size (rank) too small reduces how much it can learn, while setting it too large reduces its main advantage of being lightweight. Variants that train LoRA on a quantized model are also widely used. An adapter works properly only with the base model it was trained on.

Example

In image generation communities, people share LoRA files trained on a particular art style or character, and users load them into a base diffusion model to create images in the style they want. Companies also commonly use LoRA to train an open-weight LLM on internal document formats or customer service tone, swapping adapters for different departments on a single base model.

Related terms

Fine-tuning Parameter Open weights Quantization Post-training SFT