What is NVLink?
The fast GPU-to-GPU link behind scale-up designs.
← Back to Compute College
Compute College
What is NVL72? Scale-Up vs Scale-Out
NVL72-style rack systems link many GPUs into one; it illustrates scale-up (bigger tightly-coupled units) versus scale-out (more networked units).
NVL72 scale-up definition
NVL72 refers to a rack-scale system that links many GPUs with high-speed interconnect so they behave like one very large accelerator. It illustrates scale-up — making a single tightly-connected unit bigger — versus scale-out, adding more separate units connected over a network. Both expand AI compute, but in different ways.
Memory trick: Scale-up builds a bigger engine; scale-out adds more engines — both add power, but they solve different problems.
Why it matters
Why NVL72 changes cluster design
Large models need many GPUs working together. Scale-up tightly couples GPUs so they share memory and data at very high speed, which suits big models and low latency; scale-out adds more nodes over slower networks, which suits throughput and capacity. The balance affects performance, cost, power density, and cooling.
- Scale-up makes a single tightly-connected unit larger; scale-out adds more units over a network.
- Tightly-coupled rack systems help big models train and serve efficiently.
- More tightly-packed GPUs raise power density and cooling requirements.
Simple example
Suppose a model is too large for one GPU. Scaling up puts more GPUs in one fast-interconnected rack so they act as a single big accelerator; scaling out spreads the work across many racks over a network. The scale-up approach can be faster for tightly-coupled work but concentrates power and heat in one rack.
- Scale-up favors tightly-coupled work such as large-model training and low-latency serving.
- Scale-out favors capacity and throughput across many independent jobs.
- Tighter coupling concentrates power and heat, raising cooling needs.
Example figures are illustrative calculations, not current quoted market prices.
Common mistake
Counting GPUs without asking how they are connected. The same number of GPUs can behave very differently depending on whether they are tightly coupled (scale-up) or loosely networked (scale-out) — interconnect, not count alone, decides what big models can do.
Practical takeaway
What you can do with this
When you see a cluster described, ask how its GPUs are connected and whether the workload needs scale-up or scale-out.
- Buyers: match interconnect (scale-up vs scale-out) to whether your workload is tightly coupled or parallel.
- Analysts: read rack-scale announcements as signals of model size and power density, not just chip volume.
- Note that denser scale-up designs raise cooling and site-power requirements.
- Separate interconnect design from headline GPU counts when comparing systems.
- Keep vendor performance claims illustrative until tested on your workload.
Decision check: choose scale-up when the work is tightly coupled and latency-bound, scale-out when it parallelizes across independent units.
Compute College track
Power & Data Centers
Step 9 of 17: What is NVL72 scale up vs scale out