Ep. 003 - Deep Dive on NVIDIA Vera Rubin VR NVL72 (AI Supply Chain) | Jordan Nanos, Myron Xie, Copper Wei (Wega), Howie

Summary

• Rubin’s headline 50 petaflops of sparse FP4 is the first sparsity number the team thinks will actually be usable, not just “Jensen math.” Howie’s explanation: instead of structured sparsity — “where you force every other data point… into zeros,” which “wasn’t really used at all” at FP4 because models wouldn’t converge — a new adaptive compression engine in the Transformer Engine dynamically compresses the data stream, delivering up to ~50 PF effective while processing at 35 PF dense (vs 15 PF for Blackwell Ultra). NVIDIA’s 5x marketing claim compares Rubin sparse to Blackwell dense.
• The HBM4 story is a supplier-differentiation trade: NVIDIA wants 22 TB/s per chip vs 8 TB/s per chip on Blackwell, well above the JEDEC spec, and not every vendor can hit it. Copper’s detail: Micron kept a DRAM process for the base die with relatively poor-quality transistors, SK hynix went to TSMC N12, and Samsung to its own 4nm — with Micron “having trouble hitting the speeds.” Consequence: “a lot of Rubin that ships with lower than the advertised memory bandwidth,” with GPUs effectively binned by HBM supplier.
• NVLink 6 doubles scale-up bandwidth without doubling wires via bidirectional SerDes — the marketed “400G SerDes” is really 200G running both directions on the same physical wire. Per-switch bandwidth did not increase, so switch count doubles (18→36 per rack); Howie notes this kept the switch die monolithic, since the prior generation was already at the ~800mm² reticle limit. The backplane hardware is unchanged from Blackwell, and real unidirectional throughput will land below the 400G best case.
• The “cableless” compute tray is a manufacturability play that roughly doubles PCB content — the removed cables are Ethernet and PCIe, not NVLink. Wega ties it to GB200’s ramp pain: cables scratched at connectors in tight assemblies caused low yield and debug problems, whereas the new modular Strada/midplane/Orchid design cuts compute-tray assembly “from two hours to five minutes” with automation that only three vendors can do or are working with NVIDIA to do. Trainium did cableless first — with one massive baseboard — and the trade-off is cost: quartz-fiberglass midplane material with worse drilling yield, with de-specification still “under debate.”
• Power and thermals jump to 2.3 kW per Rubin GPU (vs 1.4 kW Grace Blackwell Ultra), handled by evolution, not revolution. The microchannel lid wasn’t time-to-market ready; instead a two-piece heat spreader plus stiffener fights warpage, electroplated gold guards against corrosive liquid-metal TIM, and higher coolant flow rate enables chiller-less 45°C-inlet designs — an option, not a mandate, since “not everything is optimized for Rubin.” A liquid-cooled busbar and 50V-direct-to-Strata-board conversion round out the rack changes.
• NVIDIA has taken back SoCAMM procurement from customers amid DRAM tightness — a potential pricing-stability advantage over competitors. Wega estimates NVIDIA contracted perhaps half of current Rubin deployment availability for SoCAMM (“Jensen went to Korea for fried chicken… there’s a purpose there”), will pass it through with a margin, making it a double-digit BOM chunk. Jordan’s read: stable input costs let NVIDIA win marginal deals against AMD and Trainium, which may be “more exposed to changes in the DRAM pricing market” — and a high HBM price hike could still be coming.
• Timeline: rack systems ship 2H26, mass deployment more likely 1H27, Rubin Ultra (1TB HBM4E, 16 stacks) scheduled 2H27. The team expects a smoother ramp than GB200 — supply chain, rack assemblers, and test-site power are already in place — though “with the speed that NVIDIA is pushing… there could always be hiccups.” The next major system challenge is Kyber, not Rubin.

Deep dive

1. Sparse FP4 finally means something: the adaptive compression engine

• The chip itself: two big compute dies, eight stacks of HBM4, and I/O disaggregated into side chiplets for NVLink C2C and NVLink 6. Wega’s headline numbers — 35 PF dense FP4 vs 15 for Blackwell Ultra, plus a “sparse 50 petaflops” that is “a different type of sparsity” from what NVIDIA marketed in Hopper and Blackwell.
• Howie’s mechanism: NVIDIA markets “up to five times” sparse FP4 vs GB200 — but that’s sparse-on-Rubin vs dense-on-Blackwell. The new hardware compression in the Transformer Engine “directly looks at the data stream and dynamically compresses” rather than forcing every other value to zero, delivering up to 50 PF effective while actual processing runs at 35 — “a hybrid approach where you don’t have the accuracy losses of forcing the data to be zero.”
• Why it matters, per Howie: structured sparsity “wasn’t really used at all” at low precisions — “down to FP4… the models wouldn’t converge” — so this generation can “effectively reclaim your sparse performance.” Jordan’s framing of the history: after four or five generations of “Jensen math” doubling marketed FLOPs, “we expect this sparsity to work in a way that previously it wasn’t.”

2. HBM4 at 22 TB/s: a spec above JEDEC that will bin the suppliers

• Wega’s math: HBM4 doubles I/Os per stack to 2,048 from 1,024, and NVIDIA is pushing suppliers to keep the same pin speed — targeting 22 TB/s per Rubin chip vs 8 TB/s per Blackwell chip, ~2.5x. The catch: the original JEDEC spec “was much lower” than what NVIDIA wants, and suppliers have “different levels of preparedness.”
• The base-die divergence is the tell: Micron stayed on a DRAM process with relatively poor-quality transistors, SK hynix adopted TSMC’s N12 logic process, and Samsung went “all out” with its own 4nm. Result: Micron in particular is “having trouble hitting the speeds,” so “it’s quite likely there’s gonna be a lot of Rubin that ships with lower than the advertised memory bandwidth” — Jordan’s summary: GPUs binned by HBM supplier.
• Capacity stays at 288GB because density can’t move: 16-high stacks haven’t shipped in volume, and doubling TSV wiring eats die area, so per-layer density is flat from HBM3 to HBM4. Rubin Ultra gets to 1TB by doubling stacks to 16 and going 16-high HBM4E (54GB per stack vs 32GB for a 12-high stack). Copper’s caveat: DRAM is “very tight,” so XPU designers may end up “rationing or skewing… with perhaps lower capacity than initially planned.” Copper also corrects the terminology: it’s not “R200” — “drop the 200. It’s Rubin.”

3. NVLink 6 doubles bandwidth by cheating physics honestly — bidirectional SerDes

• Wega’s core point: NVSwitch per-switch bandwidth did not increase, so switch count doubles from 18 to 36 per rack (nine trays, four packages each) — and the marketed “400G SerDes” is really 200G running in both directions on each physical wire, doubling bandwidth “with the same amount of physical wiring” and the same backplane hardware as Blackwell. His hedge stands: 400G unidirectional “is the best case scenario, but it’s unlikely they’re gonna hit that.”
• Howie’s addition on why they doubled chips instead of chip bandwidth: the previous switch die was already at the reticle limit, over 800mm², so doubling bandwidth in one package “will be really challenging” — they’d have had to go chiplet. Each Rubin-to-switch link is now a single 400G link instead of multiple ports.

4. Cableless is a yield story, and PCB is the new BOM battleground

• The key correction: the cables removed are not the NVLink protocol cables; they are the Ethernet and PCIe cables, while cables remain in the backplane. The motive is GB200’s ramp trauma: workers routing 10–20 cables in tight spaces, connectors “scratched during assembly,” then painful debugging of which cable failed. Jensen’s CES claim: automated assembly cuts the compute tray from “two hours to five minutes” — with only three vendors able to do the automation or working with NVIDIA on it. “It’s kind of like building Legos.”
• The architecture: two Strada modules (each housing two Rubin GPUs and one Vera CPU) at the rear, a midplane bridge, and four Orchid modules housing CX-9 at the front — all signals on PCB, driving PCB area to “two times or more” versus Blackwell plus major material upgrades. The midplane’s quartz fiberglass is state-of-the-art but so hard that it can worsen drilling yields; whether NVIDIA de-specs to high-end glass-fiber cloth is “under debate currently.”
• Asked whether NVIDIA pioneered this, Wega credits Trainium — the first cableless design, advocated at re:Invent for two years, but as “one massive baseboard” spanning the chassis (a time-to-market decision, per Amazon) versus NVIDIA’s modular take. The trade-off isn’t serviceability (“serviceability is better here”) — it’s cost. On packaging proper, Myron calls NVIDIA “more conservative”: AMD is pushing hybrid bonding/SoIC, “doing that since MI300X.”

5. 2.3 kilowatts per GPU, and the thermal/power tricks that make it survivable

• TDP jumps to 2.3kW per Rubin GPU vs 1.4kW for Grace Blackwell Ultra. The microchannel lid — a cold plate integrated into the package — wasn’t mature enough for first deployment; instead: a two-piece heat spreader plus stiffener to fight warpage on the bigger package, electroplated gold on both lid and cold plate because the liquid-metal TIM “is corrosive to copper,” and tighter cold-plate channel pitch for more heat-exchange surface.
• The 45°C inlet everyone talked about is an infrastructure play, not a chip trick: by raising flow rate and shrinking the inlet-outlet delta (45°C in, 65°C out), you enable chiller-less data centers. Wega’s hedge: “the chiller-less design is not a one-for-all solution… but it’s an option that helps with your TCO” — data centers host other hardware and roadmaps change. NVIDIA’s reference design suggests double Blackwell’s thermal performance.
• Power delivery: a liquid-cooled busbar (no fans in the system), and 50V-to-12V conversion moved onto the Strada board for the compute modules themselves — each Strada module takes 5kW, and stepping down closer to the package is “the right design choice” for efficiency. The networking modules still receive the step-down from the power-delivery board. HVDC power racks are optional, but the timeline is “being pulled forward” and some Rubin deployment could use them.
• Overall, the architecture remains broadly similar to Grace Blackwell; the biggest system change is PCB content, while much of the performance improvement comes from the semiconductor side.

6. The SoCAMM land-grab, and a two-year rhythm that says 1H27

• Wega’s most tradeable observation: SoCAMM — NVIDIA’s socketed LPDDR module for the Vera CPU, replacing soldered-down LPDDR — was previously customer-procured, but amid memory tightness NVIDIA “has taken the procurement responsibility back,” contracting “maybe half” of current Rubin deployment availability. “Jensen went to Korea for fried chicken, that is definitely for something.” Customers pay for it with NVIDIA’s margin — making SoCAMM a double-digit BOM chunk (single-digit without the margin) — while supply and pricing are secured for the contracted portion.
• Jordan draws the competitive conclusion: with stable input costs, NVIDIA “can make deals on the margin work” against AMD or Trainium, which may be “more exposed to changes in the DRAM pricing market.” Wega adds the kicker: on HBM, “there could be high price hike coming.”
• Timeline consensus: racks ship in the second half of 2026, but Myron’s caution from Blackwell applies — “out of the box performance straight away… is never really super optimized” — so mass deployment lands in the first half of 2027, with Rubin Ultra scheduled for 2H27. Jordan’s Blackwell benchmark: first CoreWeave rack in November 2024, real usage in February–March, at-scale deployment in May–June 2025. Wega argues this ramp will be smoother — rack assemblers have capacity and test-site power secured, unlike early GB200 — but “there could always be hiccups,” and “the next big challenge will be Kyber.”