Fat Trees Are Falling: AWS's Random-Graph Bet Rewrites the Data Center Network

For about twenty years, the answer to “how do you wire a data center” was settled: build a Clos fabric, also called a fat tree. Top-of-rack switches feed into aggregation switches, which feed into a spine. Bandwidth doubles at each tier so the core never starves. It is regular, it is provable, and it is what every networking textbook and every hyperscaler converged on.

AWS just walked away from it. According to InfoQ’s report, AWS has made a flat, quasi-random topology — internally called Resilient Network Graphs (RNG) — the default for most new non-GPU data center builds as of April 2026. The numbers attached are not incremental: 69% fewer networking devices, up to 33% higher throughput, and a projected 40% drop in network equipment power. That is not a tuning pass. That is a different theory of how a fabric should be shaped.

Why the tree stopped fitting

The fat tree is optimized for a workload assumption that is quietly breaking down. Classic cloud traffic is mostly north-south and loosely correlated: a million tenants doing unrelated things, statistically smooth, easy to aggregate up a hierarchy. The tree’s strength is that it gives any rack a guaranteed path to the core.

But that hierarchy has costs. Every byte crossing the building climbs three tiers and descends three tiers — six switch hops, six chances to queue, and a spine layer that is both the most expensive hardware in the room and the single most catastrophic failure domain. Lose a spine switch in a fat tree and you do not lose a sliver of capacity; you lose a fat slice of cross-sectional bandwidth and watch hotspots bloom. The topology that guarantees bandwidth in the happy path concentrates fragility in the worst one.

The deeper problem is that modern bandwidth-hungry workloads do not look like a million unrelated tenants. They look like one giant coordinated job that wants every node talking to every other node at once.

All-to-all is the real driver

This is where the angle sharpens. A GPU collective — an all-reduce across thousands of accelerators synchronizing gradients, or an all-to-all shuffle in a mixture-of-experts layer — is the pathological case for a hierarchy. It demands near-uniform bandwidth between every pair of endpoints simultaneously. There is no statistical smoothing to lean on; the traffic matrix is dense by design.

A fat tree built to actually sustain full all-to-all bandwidth has to be provisioned for the worst case at every tier, which is ruinously expensive, or it oversubscribes and chokes the moment the collective fires. Random graphs are a known answer to exactly this. Theory from expander-graph research (the Jellyfish line of work) showed years ago that a randomly-wired mesh achieves lower average path length and higher throughput per dollar than a Clos at scale — precisely because uniform random connectivity matches uniform random demand. AWS is now operating that idea in production.

Two pieces of engineering made it shippable. The first is the ShuffleBox: a passive optical interconnect that takes the rat’s nest of “every ToR connects to a pseudo-random set of other ToRs” and turns it into a manufacturable, cable-once unit. Passive means no transceivers, no power, no failure-prone silicon in the shuffle layer — the randomness is baked into fiber routing inside the box. The second is Spraypoint, a custom routing protocol that sprays flows across the many redundant paths a mesh naturally provides, balancing load instead of pinning a flow to one shortest path.

Graceful degradation as a first-class feature

The resilience property is the part that should make every network architect sit up. In RNG, failures degrade proportionally: lose 1% of routers, lose roughly 1% of capacity. Compare the mental model:

Fat tree:    spine switch dies  -> large cross-section bandwidth gone,
                                    hotspots, retransmits, tail latency spikes

Random mesh: 1% of links die     -> ~1% capacity gone, traffic reroutes
                                    across remaining paths, no cliff

There is no special “spine” whose loss is qualitatively worse than any other node’s. Every link is roughly interchangeable, so there is no failure domain to protect with expensive redundancy. AWS reports validating this across production sites in Ireland, Germany, and Spain, on the back of 530 processor-years of simulation. That simulation budget is itself a tell: a random topology has no closed-form “is this wiring correct” check the way a Clos does, so you buy confidence with compute.

The caveat that proves the rule

Here is the twist worth dwelling on. RNG is deployed for general-purpose compute — and AWS explicitly does not use it for AI training clusters, which run on the separate UltraServer architecture. The reason is the centralized, tightly-coordinated traffic of GPU collectives, which AWS handles with its own purpose-built fabric.

So the angle needs a precise reading. All-to-all pressure is what exposed the fat tree’s weakness as a general topology and pushed AWS toward random graphs. But the most extreme all-to-all workload — GPU training — gets an even more specialized fabric still. The lesson is not “random graphs win everywhere.” It is that the one-size-fits-all Clos era is over. The fabric now follows the traffic matrix, and there are at least two matrices that matter: dense-coordinated (training) and dense-but-decorrelated (everything else at scale).

Why the others will follow

The forcing function is economics layered on physics. Across the hall, the rest of the AI buildout is in what one might charitably call a Mad Max phase: Meta is erecting data centers in literal tents outside Columbus to halve construction time, against $145B in planned capex. Upstream, TSMC’s CEO is openly warning it “can only support so much” as the industry races toward a projected trillion-dollar semiconductor market by 2027. When silicon is the bottleneck and capex is measured in hundreds of billions, a topology that delivers a third more throughput on 31% of the switch count and 60% of the network power is not a nice-to-have. It is a direct lever on the scarcest resources in the business.

That is why this is more than an AWS curiosity. The architectural logic — match the fabric to the demand, kill the central failure domain, trade closed-form provability for simulated confidence and statistical uniformity — applies to anyone running a building full of accelerators. The fat tree isn’t wrong. It’s just no longer the default, and defaults at this scale move the whole industry.