AI Trend Research

← AI glossary

AI glossary · Beginner · Efficiency & infra

GPU

A GPU (graphics processing unit) is a chip originally built to draw images on screen, and it excels at running thousands of simple calculations at the same time. This property has made it the core hardware for training and running today's AI models.

About this term
Level
Beginner · Core ideas for first-time readers
Added
2026-10-08
In plain words If a CPU is a few math professors who can solve any problem, a GPU is a classroom of thousands of students who only add and multiply, splitting up a worksheet and solving it all at once. Each problem may be slower, but for work with mountains of similar calculations, the classroom finishes much faster.

How it works

To compute the colors of the many dots (pixels) on a screen at once, a GPU contains a very large number of small compute units. Most of the computation in AI models is also repetition of similar work, multiplying huge tables of numbers (matrices), so it fits the GPU's design well. High-speed memory such as HBM is placed close to the chip to move model parameters and intermediate values quickly. Large models cannot be handled by a single GPU, so thousands to tens of thousands of GPUs are linked by high-speed networks and used together.

Why it matters

Large models such as LLMs require enormous computation to train, and serving them (inference) also requires ongoing computation. So how many GPUs a company can secure is directly tied to its competitiveness in AI. That is why GPU shortages driving up prices, or companies and countries announcing large GPU purchase plans, become news. Data center power issues also ultimately come down to the electricity GPUs use.

Things to know

GPUs are not the only chips used for AI computation. NPUs designed only for specific AI operations and companies' own custom chips are also increasing. Software ecosystems matter as much as hardware, so GPUs from companies with well-developed development tools and libraries tend to be more widely used. Techniques such as quantization can reduce the GPU resources the same model needs.

Example

News about GPU-related infrastructure appears often on this site. The story on AMD CEO Lisa Su's visit to Korea covered plans for heterogeneous infrastructure combining AMD CPUs and GPUs with Korean-made NPUs, showing a trend of using GPUs together with other AI chips. And as with NVIDIA's release of 'AICR v1.0', a tool for validating GPU Kubernetes cluster configurations, software for linking and properly operating many GPUs is also becoming an important topic.

Where it came up on this site

Related terms

NPU HBM Inference Pre-training Scaling laws On-device AI