AI glossary · Advanced · Efficiency & infra · AI platforms & solutions
Interconnect
An interconnect is the high-speed communication network that links multiple GPUs and servers so they can exchange data. Typical examples are NVLink within a server and InfiniBand or high-speed Ethernet between servers.
About this term
- Level
- Advanced · Specialist terms that need the inner workings
- Category
- Infrastructure & using models
- Added
- 2026-10-12
How it works
Large models are split across multiple GPUs because they do not fit on a single GPU or would take too long to train. The GPUs then have to constantly exchange intermediate results and gradients. GPUs within the same server communicate very quickly over dedicated links such as NVLink, while servers are connected through InfiniBand or high-speed Ethernet switches. In general, links within a server are much faster than links between servers.
Why it matters
In distributed training and model parallelism, communication time determines overall speed. With slow links, expensive GPUs spend more time idle waiting for data. So how a model is split is also decided according to link speed. For example, splits that communicate often are placed among GPUs within a server, and splits that communicate less are placed across servers.
Where you'll see it
You'll see it in the NVLink bandwidth entry of GPU spec sheets, in the 'SXM' (NVLink-connected) vs. 'PCIe' (standard slot) distinction in server specs, and in the network bandwidth listed in cloud GPU instance descriptions. Write-ups of large-scale training clusters often describe InfiniBand setups.
Things to know
Doubling the number of GPUs does not double training speed. The added communication holds things back accordingly. Even with the same GPU model name, the PCIe version has slower GPU-to-GPU communication than the NVLink version, so check this if you plan to use several together. For inference work that fits on a single GPU, an expensive interconnect may not matter much.
Example
A team that wants to serve a large model on multiple GPUs first checks the cloud instance description for whether the GPUs are connected by NVLink. If the model is split with tensor parallelism within one server, a configuration with NVLink is better; to scale training across multiple servers, they check whether an inter-server network such as InfiniBand is in place.