AI glossary · Basic · AI platforms & solutions
GPU cloud
A GPU cloud is a service that lets you rent GPU servers from a cloud provider, for example by the hour, for AI training and inference instead of buying them yourself.
About this term
- Level
- Basic · Technical terms you often meet in AI news
- Category
- Infrastructure & using models
- Fields
- AI platforms & solutions
- Added
- 2026-10-12
How it works
Cloud providers keep GPU servers in data centers, and users choose the GPU type and count they want and rent them as 'instances'. Besides large general-purpose clouds, there are also providers that specialize in GPU rental. Rental options usually fall into on-demand (start and stop anytime), reserved (discounts for committing to a term) and spot (cheap use of spare capacity that can be reclaimed at any time). Inference API services, where you only need to upload a model, broadly belong to the same trend.
Three ways to rent GPUs
On-demandRent and return anytime
- Flexible
- Relatively high unit price
ReservedTerm commitment
- Discounted
- Better for long-term use
- Locked in for the term
SpotSpare capacity
- Usually the cheapest
- Can be reclaimed at any time
- Checkpointing essential
공통 In all cases, cost is set by GPU type, count and time
Where you'll see it
You see it on cloud price lists in the form 'GPU type × count × hourly price'. Instance names or descriptions list the GPU model, GPU memory (VRAM), how GPUs are connected and network bandwidth. To get a feel for costs, estimate training or experiment costs as 'hourly price × hours used × number of machines', and compare that with doing the same work on servers you buy.
Why it matters
GPUs are expensive and hard to obtain, so most companies and researchers use compute through the cloud. A major advantage is that you can start small in early experiments and scale up only when needed. You can also try the latest GPUs without buying them. Conversely, services with steady heavy usage cut costs by comparing reserved contracts with building their own infrastructure.
Things to know
Even with the same GPU name, VRAM capacity, NVLink support and inter-server networking differ by instance, so check the specs carefully. Popular GPUs may be out of stock in the region you want. Beyond GPU fees, storage and data transfer (especially outbound traffic) are charged separately. If you handle sensitive data, also check which country or region the data is stored in and what security certifications apply.
Example
A team that wants to fine-tune an open-weight model first estimates the VRAM it needs, then picks an instance with that much memory from a GPU cloud price list. It runs short experiments cheaply on spot instances, saving checkpoints often in case of reclamation, and runs production inference servers reliably on on-demand or reserved instances.