AI infrastructure / Network fabric sizing
GPU Network Oversubscription: Ratios, Ports and Trade-Offs
An oversubscription ratio is a bandwidth budget, not a prediction of GPU utilization. Use it alongside actual collective-communication traces.
Sections in this article
An AI networking quote advertises “non-blocking 800G switching” but places 32 server-facing 400G ports behind only eight 800G uplinks. The switch silicon may support full line rate per port; the configured fabric topology still has only half as much aggregate uplink bandwidth as downlink bandwidth. Those are not contradictory claims—they describe different boundaries.
Short answer: the leaf oversubscription ratio is configured aggregate downlink bandwidth ÷ configured aggregate uplink bandwidth. A ratio of 2:1 means the leaf cannot move simultaneous line-rate traffic from every downlink across its uplinks. It does not establish a universal 50% loss in GPU throughput, a packet-loss rate or a guaranteed training-stall duration.
Calculate a proposed leaf-switch oversubscription ratio
Use physical usable link rates, not a switch's advertised total chassis throughput. Count only the ports allocated to the same traffic class and fabric under review; if links have different speeds or are reserved for storage/management, work them out separately. The expected cross-leaf share is a scenario assumption about simultaneously offered traffic, not a networking specification.
At this traffic assumption, offered demand exceeds uplink capacity. Application impact needs a real workload benchmark.
A 1:1 ratio is a necessary budget, not a complete performance proof
Suppose 32 × 400G server links connect to a leaf (12.8 Tbps of nominal downlink capacity). Sixteen × 800G uplinks yield 12.8 Tbps, or a 1:1 aggregate budget. With only eight 800G uplinks, the northbound side is 6.4 Tbps and the ratio becomes 2:1. The calculator exposes both the static capacity ratio and what happens under an assumed fraction of cross-leaf communication.
Even at 1:1 aggregate port capacity, real workloads face congestion from competing flows, poor path distribution, buffer exhaustion, transport behavior, head-of-line blocking and nonuniform placement. And 1:1 at one leaf says nothing by itself about oversubscription deeper in the spine layer or connectivity after link failures. Topology matters.
Conversely, a 2:1 fabric can be adequate when communication is localized within nodes, racks or NVLink domains, or when workloads seldom offer simultaneous traffic at every NIC's full line rate. The right design depends on measured all-reduce, all-to-all, point-to-point and storage patterns over the topology being purchased.
What generates a slowdown: not all GPU jobs are identical
All-reduce and all-to-all patterns may expose uplink pressure and tail-latency problems, depending on placement and the collective algorithm.
Large portions of traffic stay within a server, NVLink domain or other local group. The leaf link budget may have unused capacity.
Ingestion, checkpoint storage, serving, model distribution and back-end networking may share some physical links or remain on distinct fabrics.
NVIDIA NCCL selects communication algorithms and routes based on topology and hardware support. Therefore the rule “every GPU always needs one dedicated 400G uplink” is incorrect. The NIC count and speed follow the actual chosen system design. Effective cross-leaf traffic is also influenced by pipeline, tensor and data parallelism strategies.
Illustrative design review: fit ports before buying spines
An architect proposes four identical leaf switches, each serving the 32 × 400G downlinks in the example. If the budget requires 1:1 leaf connectivity, each leaf needs sixteen 800G uplinks—not eight—assuming the downlink and uplink ports can both be allocated as specified on the particular hardware. Across four leaves, that is 64 800G uplink connections. The available spine ports must match every routed connection, with enough independent paths and capacity after the relevant single failure.
Do not simply divide 64 uplinks by a spine model's port count and announce that two switches guarantee resilience. Link distribution, port breakout capability, oversubscription in the spine, routing, fault domains and spare ports may alter the result. Some fixed-format switches cannot simultaneously supply the desired 400G breakout count and 800G uplinks, despite an apparently sufficient total throughput figure.
| What to request from the network vendor | Why it matters |
|---|---|
| Physical port allocation and supported breakout matrix | Confirms the exact proposed southbound and northbound port counts |
| Oversubscription after a failed uplink or spine | Reveals degraded-mode bandwidth rather than just best-case capacity |
| Congestion, retransmission and pause counters | Allows observed bottlenecks to be distinguished from application stalls |
| NCCL collective benchmark at realistic scale | Measures relevant cross-GPU bandwidth and iteration-time effects |
| Optics, cable routes, power and staff effort | Connects the performance trade-off to total lifetime cost |
Use the ratio to choose a benchmark, not to guarantee performance
- Draw the intended A/B or multi-plane topology and assign every server NIC to a physical port.
- Compute aggregate capacity at every oversubscribed cut, not just one leaf switch.
- Replay representative training collectives and record application step time and cross-leaf bytes.
- Run degraded-mode tests for a failed link, switch and congestion hot spot with approved change control.
- Compare additional fabric capital cost against measured end-to-end job duration and equipment energy.
Performance numbers like “95% efficiency for 1:1” are not generally transportable between applications and GPU generations. Keep conclusions tied to a measured cluster and workload. The interactive ratio is useful precisely because it exposes one explicit physical constraint without pretending to solve the full network.
What happens after losing one uplink?
Redundancy changes the actual bandwidth budget. For example, a leaf with eight 800G uplinks has 6.4 Tbps while all links work. After losing one link it has 5.6 Tbps, so the same 12.8 Tbps downlink plan becomes approximately 2.29:1, not 2:1. Whether this is tolerable depends on routing re-convergence, how much cross-leaf traffic was offered and the particular collective algorithm. Include both normal and degraded ratios in the network acceptance document.
Sources and further reading
Primary frameworks and reference materials. Engineering decisions require site-specific validation; legal classifications require assessment of the actual use case.