Why next-gen AI scale-up needs CPO

Summary

• CPO’s first target is scale-out, where removing the DSP attacks AI data centers’ power and utilization constraints. In pluggable transceivers, the DSP consumes up to 60% of power and accounts for over 90% of added latency; the transceiver takes 150–200 ns to translate the signal. CPO moves optics close enough to the switch or ASIC to eliminate that DSP tax.
• The architecture roadmap moves from compromise products toward increasingly integrated—and harder-to-repair—optics. LPO simply drops the DSP and “hopes for the best,” OBO delivers the “worst of both worlds,” and NPO may be either a durable middle ground or an intermediate step; true interposer-based CPO eliminates both the DSP and SerDes, becoming the “final boss of CPO.”
• CPO’s scale-out benefits do not guarantee uniform adoption because it changes supplier relationships and the failure domain. Standard OSFP and QSFP-DD pluggables are standardized, multi-sourced, and replaceable by any technician; CPO can tie buyers to NVIDIA or Broadcom hardware and turn one failed optical interface into an entire-switch replacement. Hyperscalers therefore prize repairability, vendor diversity, and price control, while new clouds are more attracted to turnkey NVIDIA systems.
• Copper remains the superior scale-up medium while it can still reach across the rack. It “speaks the native language of semiconductors,” adds roughly 5 ns per meter—perhaps 10 ns over 2 m—and requires no electrical-optical conversion, leaving even DSP-free CPO slower and more power-hungry at current rack-scale distances.
• The forcing function arrives with 448G links because copper’s reach will shrink. Current-generation 224G copper can reach about 2 m with PAM4; next-generation 448G may require PAM6, PAM8, or a higher baud rate, each worsening signal-to-noise and shrinking reach. The revised rule is “use copper as long as you can,” while NVIDIA still plans copper-based intra-rack networks for Rubin, Feynman, and beyond.
• The ultimate CPO prize is a larger scale-up world size, which can overwhelm its latency and energy penalties. Blackwell expanded the domain from eight GPUs to 72 with NVL72; announced at GTC 2026, Vera Rubin Ultra NVL576 combines eight copper-based NVL72 racks with optical links, while NVIDIA Kyber NVL 1,152 is already “on the horizon.” Once optics connect far more GPUs, “no one can outcompete a larger scale-up world size.”

Deep dive

1. Copper owns the rack; optics owns the data center

• The episode begins with copper’s physical bargain: it connects everything from transistor metal layers to the spine of a single NVIDIA NVL72 rack, but a 200-gigabits-per-second-per-lane signal reaches only about 2 m. Hence the rule: “use copper when you can and optical when you must.”

• Scale-up links make GPUs inside one or maybe a double-wide rack behave almost like one GPU, demanding extreme bandwidth and latency; scale-out connects racks across the data center. Scale-out needs eight to 10 times front-end bandwidth, while scale-up needs another 10 times more.

2. The DSP is the tax CPO was invented to remove

• A standard OSFP or QSFP-DD pluggable contains its electrical interface, DSP, laser-bearing TOSA, and light-sensing ROSA. Contrary to the intuitive answer—“It’s a freaking laser, right?”—the laser consumes only about 15% of power.

• The DSP consumes up to 60% of a pluggable’s energy. A pluggable transceiver takes 150–200 ns to translate the signal, with over 90% of the added latency attributed to the DSP.

• The electrical signal can degrade along a path of up to roughly 30 cm through the chip’s metal layers, package, motherboard, and copper before reaching the transceiver. CPO’s premise is to place the optical engine close enough to the source that the DSP is no longer needed.

3. Every step toward full CPO trades serviceability for integration

• Linear pluggable optics keeps the familiar module but discards its DSP—effectively saying “Screw it.” It works, but translating an already-distorted signal into light sharply reduces optical reach.

• On-board optics moved closer yet retained the DSP while losing easy replacement, combining the “worst of both worlds.” NPO moves onto a high-performance substrate and is already being deployed, though it “could become a real middle ground” or merely precede CPO.

• Basic CPO puts optics and the ASIC on one package, removing the DSP but retaining SerDes over the shared packaging substrate. A silicon or organic interposer enables denser, fully parallel links and removes SerDes too—the “final boss of CPO.” Integrating optics directly with the GPU would be an entirely different level rather than another tier; current examples such as NVIDIA Quantum and Spectrum-X are CPO for networking switches.

4. Hyperscalers and new clouds value different sides of the trade

• The episode’s pushback on inevitable CPO adoption: pluggables are standardized, familiar, multi-sourced, and quickly replaceable by any data center technician. With packaged optics, NVIDIA hardware implies NVIDIA’s solution, Broadcom hardware implies Broadcom’s, and one optical-interface failure can require replacing the switch.

• CPO nevertheless attacks the largest AI-data-center pain points: power and system utilization. Lower scale-out latency can help keep billions of dollars of accelerators from sitting idle, while reduced networking energy limits the power spent on everything outside AI compute.

• Many hyperscalers still prioritize repairability, vendor diversity, and price control, seeking to avoid “vendor lock-in at all costs.” New clouds are keener on turnkey CPO systems; hybrids combining NPO and pluggable transceivers are also under exploration. Because scale-out was already optical, the surrounding infrastructure is in place and AI data centers are beginning to adopt CPO there.

5. Copper’s approaching wall turns world size into CPO’s killer feature

• Copper remains faster because no translation is required: about 5 ns per meter, or perhaps 10 ns over 2 m, versus 150–200 ns for a pluggable transceiver, most of whose added latency comes from the DSP. Even DSP-free CPO still adds latency and consumes more energy than copper because it must translate between electrical and optical signals.

• Copper’s roughly 2 m reach at 224G with PAM4 may contract at 448G. PAM6, PAM8, or a higher baud rate may be needed, but higher modulation levels and signaling speeds worsen signal-to-noise and shrink copper’s reach. NVIDIA therefore still plans copper-based intra-rack networks for Rubin, Feynman, and beyond.

• Blackwell’s defining scale-up jump, in the episode’s framing, was from eight GPUs to 72 through NVL72—a 9× expansion that let 72 GPUs operate like one and unlocked the generation’s “true performance advantage.”

• Vera Rubin Ultra NVL576 extends that logic across eight NVL72 racks: copper remains inside each rack, while optics joins the racks. NVIDIA Kyber NVL 1,152 follows; once copper reaches its literal limit, the narrator expects CPO to take over scale-up “in the blink of an eye.” The narrator cautions that CPO is not a holy grail: copper will be pushed to its limit, and scale-out adoption is clearer than scale-up.