AI glossary · Basic · Efficiency & infra · AI platforms & solutions
CUDA
CUDA is a GPU programming platform made by NVIDIA. It is the software foundation that lets the GPU's many processing units be used for general computation such as AI training and inference.
About this term
- Level
- Basic · Technical terms you often meet in AI news
- Category
- Infrastructure & using models
- Added
- 2026-10-12
How it works
GPUs were originally built to draw graphics, but they are good at running the same calculation many times at once. CUDA is a bundle of language extensions, a compiler, drivers and libraries that lets ordinary programs use this ability. Developers rarely write CUDA code directly; usually a deep learning framework such as PyTorch calls CUDA libraries like cuDNN and cuBLAS on their behalf. So users only need to move the model to the 'cuda' device in their code, and the computation runs on the GPU.
Where CUDA sits
- My code and modelmodel.to('cuda')
- Deep learning frameworkPyTorch, etc.
- CUDA librariescuDNN·cuBLAS
- CUDA runtime and driver
- NVIDIA GPU
Where you'll see it
The most common encounter is the option on the PyTorch install page where you pick a build such as 'for CUDA 12.x'. Running nvidia-smi in a terminal shows the CUDA version the driver supports. CUDA versions also appear in Docker image names and in install guides for inference engines. If an error message says 'CUDA out of memory', it means the GPU memory (VRAM) has run out.
Why it matters
Most libraries and optimized code in the AI ecosystem are built for CUDA first. That is why NVIDIA's strength in the AI market is often attributed not only to its hardware but to the CUDA ecosystem. When a new technique appears, a CUDA implementation is commonly the first to be released. Conversely, moving to another company's chips means closing this software gap.
Things to know
CUDA works only with NVIDIA GPUs. AMD GPUs use ROCm, and Google TPUs use separate software such as XLA. Many installation problems come from mismatched versions of the driver, the CUDA toolkit and the framework build. If you have a GPU but things are slow, first check whether the framework actually detects CUDA.
Example
A developer who wants to run an open-weight model locally first checks the CUDA version the driver supports with nvidia-smi, then installs a PyTorch build for a CUDA version no higher than that. If an inference engine's documentation says 'requires CUDA 12 or later', the driver must be upgraded first. The models in research news on this site, such as 'Long-WAM', are also usually trained and tested in CUDA-based environments like this.
Where it came up on this site
- Papers Long-WAM: extending the context of World-Action models 2026-10-09