Topics Pioneers Insight Method Author
Back

Fractile’s Walter Goodwin on NVIDIA, Capex and Neo Clouds

·20VC

Rendering the long image…

Topics AI Chip Memory Wall and Bandwidth Bottlenecks AI Inference Cost Optimization AI Data Center Power Bottlenecks GPU Depreciation and Cloud Market Shakeup Next-Gen Disruptive Computing Architectures
Pioneers Walter Goodwin

Summary

  • Fractile’s bet is that faster inference requires a different memory architecture. Goodwin says new platforms believe they can gain more than 10X on cost or speed in selected workloads; Fractile targets gains in both by stacking memory above logic. The aim is SRAM-like bandwidth with DRAM capacity for long-running agents. Fractile has received fabricated silicon, but full-product tape-outs remain “a few months” away.
  • Goodwin expects inference demand to keep expanding even as intelligence becomes cheaper to deliver. Training is the factory’s fixed cost; deployment must ultimately justify building it. Reasoning models multiply consumption through both longer execution and parallel agents, leaving him expecting “thousands of times more compute” and far higher model-running expenditure in 10 years.
  • Power constraints strengthen the case for more efficient chips. Goodwin says land and power are fundamental limits on data-center capacity, and that energy is the most fundamental long-term AI exposure: “buy energy.” Within a fixed power envelope, fast inference that sacrifices aggregate throughput becomes costly. He points to China’s large hydro projects and says the UK should bring more renewables, nuclear and other power online.
  • Frontier models retain pricing power, while competent neo clouds face weaker differentiation. DeepSeek’s optimized serving stack illustrates why even open-model inference has not become a complete race to the bottom. Goodwin prefers flexible inference providers with enterprise distribution to capacity-heavy neo clouds, and predicts “a couple of multi-hundred-billion-dollar companies” in that space.
  • Chip orchestration could be a temporary opportunity rather than a durable control point. Goodwin argues that splitting workloads across specialized processors creates a fixed hardware mix that can become wrong as models and contexts change. Stranded capacity is “completely unacceptable” when operators must maximize revenue from finite megawatts; Fractile aims to combine speed, capacity and compute on one platform.
  • Goodwin predicts Nvidia will become a $10 trillion business within five years. His argument combines incumbent supply-chain advantages with replacement demand: every chip installed today will leave its data center within six or seven years at most, he says. “We’re so traumatized by the dot-com bubble that we’ve priced in exuberance right from the outset.”
  • Fractile’s $720 million Series C finances supply-chain commitments and a broader roadmap. Introduced at a $7 billion valuation, the round supports future generations, systems, networking and internal ML experiments. Goodwin emphasizes “multiple shots on goal”: predicting workloads and securing components years ahead matter alongside getting the first product manufactured.

Deep dive

1. Full-stack expertise does not require competing at every layer

  • Stebbings asks whether vertical integration makes a standalone chip company untenable. Goodwin separates internal expertise from monetization: Fractile needs ML researchers, manufacturing specialists and rack engineering to choose hardware bets, but sells chips and compute racks. “We never intend to compete with our customers”; it will never monetize its internal models.

  • Goodwin accepts an intelligence “waterline” beyond which an open model satisfies particular tasks. But “frontier intelligence is not yet commoditizing”: capability gaps still support high margins, while frontier research also produces better, smaller models. Adequate intelligence does not eliminate competition over its cost.

  • His concrete counterexample is DeepSeek’s inference stack, released “a few days ago.” Goodwin says its combination of model design and serving software delivered much higher throughput, better interactivity and better speeds than other DeepSeek servers. An open model therefore did not make its execution interchangeable; vertical understanding preserved room for “really healthy margins.”

  • Cloud compute looks different: “One competent neocloud is much like another competent neocloud.” The opportunity today partly reflects uncertainty about which operators are competent. Stebbings points to Nebius moving into routing and inference; Goodwin agrees that neoclouds are seeking differentiation through software and silicon.

2. Specialized chips can address an enormous market

  • Goodwin credits Nvidia with outstanding systems, scale and data-center inference work. Its strength is delivering relatively “unopinionated” general-purpose processors at extraordinary volumes. Fractile instead makes explicit predictions about the evolving frontier and organizes its technology around them; inference speed has driven its emphasis on memory bandwidth.

  • Specialization no longer necessarily means a niche. Five or 10 years ago, Goodwin says, no single workload represented more than roughly 10% of compute demand; frontier LLM inference now supports enormous dedicated build-outs. New platforms believe they can gain more than 10X on cost or, in Fractile’s case, on both cost and speed in selected workloads. Long design cycles and useful lifetimes can let a successful architectural bet retain value.

  • Those remain contrarian bets, with few gigawatts of alternative silicon deployed and financing harder than for conventional platforms. Goodwin cites Fluidstack deploying Google TPUs as an example of a neocloud stepping away from Nvidia and benefiting. Accepting platform risk can change a provider’s economics rather than merely improve its software.

3. The memory wall is a bandwidth problem with a capacity trap

  • Goodwin’s numerical framing: chip arithmetic capability has increased about a millionfold in 25 years, versus roughly 40-fold for memory bandwidth—a 25,000-fold divergence. During autoregressive generation, model weights and context must repeatedly reach the processor. A frontier model producing perhaps 80 tokens per second may therefore be limited by memory movement rather than mathematical capability.

  • His analogy puts DRAM in a warehouse and computation in an office: boxes must travel between buildings because the office cannot hold them all. SRAM keeps boxes inside the processor, enabling fast access, but consumes expensive logic silicon. DRAM’s separate manufacturing process delivers much greater density and lower capacity costs.

  • Fractile’s approach stacks memory above the logic die. Conventional memory access competes for limited edge “beachfront”; stacking creates a two-dimensional interface with “millions of them coming straight down”—the copper connections. Goodwin describes the objective as SRAM-like bandwidth with DRAM’s capacity and scalability, rather than choosing between speed and memory size.

  • That combination matters because fast SRAM-based systems face a capacity mismatch with long-context agents. Millions of sequential tokens can mean days of execution, while larger contexts consume more memory. Goodwin wants simultaneous speed and capacity, but resists defining Fractile permanently as “the company that cracked 3D DRAM”: its response must change if workloads change.

4. Tape-out is a handoff; reliable racks are the milestone

  • Stebbings challenges the industry’s marketing against what is actually in production. Goodwin says Fractile has received silicon from foundries and is running further trials, with full-product tape-outs coming “in a few months.” He distinguishes those milestones from the harder question of reaching meaningful manufacturing volume.

  • Direct manufacturing relationships are central to his answer. Goodwin says several competing platforms are rumored to use Broadcom and share process nodes and HBM supply. Recalling a Broadcom presentation promising six customers a common next-generation platform in 2028, he argues that different architectures still inherit important shared building blocks: “You get to change the color of the paint.”

  • Fractile keeps physical design and foundry interaction in-house, allowing more customized manufacturing work. Different logic and DRAM choices may also access resources other frontier platforms have moved beyond. Goodwin’s phrase is finding gigawatts of capacity “down the back of the sofa”—provided the company can coordinate larger supply-chain partners around its custom approach.

  • Tape-out sends the final GDSII design to the foundry; months of fabrication, bring-up and integration follow. Fractile instead tracks “task zero”: a reference workload running across an entire rack. Goodwin stresses repair, redundancy, yield, reliability and software readiness. A design file is tangible, but manufacturing an economical system at volume is the objective.

5. Land and power constrain deployment and reward efficiency

  • Goodwin describes a moving bottleneck: Nvidia GPU shortages became broader silicon and memory shortages, followed by acute land and power constraints. Easily converted industrial sites and rapid gas-turbine deployments were often “one-time tricks”; the next sites are harder to bring online. HBM and logic supply constraints nevertheless persist alongside those limits.

  • Fixed power makes chip choice more consequential. On HBM-based systems, Goodwin says, faster generation can sharply reduce aggregate throughput per megawatt. His example is a multi-trillion-parameter model running at thousands of tokens per second inside a 500-megawatt facility: insufficient total throughput means the operator cannot serve its users, however attractive the individual response speed.

  • Goodwin calls data-center capacity the fundamental limit on how much compute can be built out. He says energy is the most fundamental long-term AI exposure: “buy energy.” The constraint also makes chips that deliver more throughput at high speeds especially valuable.

  • Stebbings challenges Western permitting against China’s energy build-out. Goodwin agrees China has “stolen the march,” citing the fit between a remote 15-gigawatt hydro project and colocated AI compute. Scotland’s wind resources offer a similar location argument. His UK policy priority is more renewables, nuclear and other power that enables industrial growth.

6. Reasoning turns inference into sustained computational work

  • Goodwin’s original inference thesis begins with fixed-cost economics: foundation-model training must eventually be justified by deployment. Spending more to build a factory than its output ever returns would invalidate the investment. His second bet was test-time scaling; reasoning models arrived two years later, with credit to Noam Brown and OpenAI’s team.

  • Capability now scales through both longer execution and parallel copies. Goodwin cites what he describes as a Navier–Stokes solution involving, “I think,” 80 hours and 10,000 agents. The point is the multiplication of both axes: a swarm generating extended chains of thought consumes something fundamentally different from a chatbot’s immediate response.

  • Inference is “the knob that we have to turn to get greater capability.” A rare-disease question may require hundreds of candidate approaches, literature searches and agents debating one another—not an instant answer. Goodwin imagines returning after a week and perhaps billions of tokens. This creates a chance of solving the problem; it does not make the answer automatic.

7. Efficiency is Goodwin’s answer to the financing objection

  • Stebbings’s pushback is financial: debt, credit constraints and the additional demands of robotics make the required spending look impossible. Goodwin answers with innovation across the stack. He rejects the dark-fiber analogy to 2000–2001, asserting that powered-on GPU fleets are fully utilized: “We don’t have dark GPU fleets.” That utilization underpins his belief that the build-out is justified.

  • Algorithms can deliver more intelligence without changing hardware, through improvements including context compression and sparse attention. Better chips add another efficiency gain. Goodwin thinks the world needs “thousands of times more compute,” but will not get thousands of times more data centers; the route is dramatically more intelligence from existing land, power and racks.

  • Against Mike Cannon-Brookes’s forecast of sharply cheaper inference over three to five years, Goodwin questions the unit being priced. A token in 10 years may come from an orders-of-magnitude larger, more capable model. His categorical forecast concerns total expenditure: “We will be spending far more on running models in 10 years’ time than we are today.”

  • Stebbings also tests Satya’s vision of company-specific models. Goodwin accepts that fine-tuning helps today, but calls the advantage “transient”: the next frontier release or open model can erase it. Specialization can work when layered onto leading capability; the unresolved trade-off is whether adapting an open model beats using a more efficient frontier service directly.

8. Heterogeneous chips can strand expensive capacity

  • Goodwin questions software businesses built around orchestrating a permanent proliferation of specialized chips. In the Nvidia/Groq configuration he describes, fast SRAM holds shared model weights, amortizing expensive memory across users. GPUs handle attention over users’ separate contexts, where memory capacity and cheaper DRAM matter. The split answers current hardware limitations rather than proving that seven platforms are economically necessary.

  • A fixed deployment mix becomes dangerous as workloads evolve. Longer contexts could require more GPUs; larger models could require more Groq capacity. Either change can strand the other resource. Stebbings calls that unacceptable, and Goodwin agrees: finite megawatts must generate revenue, while the installed blend remains fixed for years as capital expenditure is amortized.

  • SRAM also occupies die area that could contain arithmetic units, weakening compute density for prompt processing. Fractile’s proposed stacked-memory architecture frees the logic die for tensor cores while adding bandwidth and capacity above it. Goodwin calls the combination a “holy trinity,” arguing that it could remove the need for complex orchestration across separately constrained processors.

  • Stebbings asks whether Nvidia/Groq was insurance or a reaction to OpenAI/Cerebras. Goodwin calls it an “acknowledgement” that fast inference needed another solution. He says, “I think,” both deals were signed on December 24. His objection today is SRAM’s cost and context limits: “snappier chatbots” do not deliver orders-of-magnitude acceleration for days-long frontier agents.

9. Flexible inference providers have a stronger position than neo clouds

  • Asked to choose among inference providers, Goodwin declines to name a favorite. Their common advantage is the ability to “dodge and weave”: they have not locked themselves into enormous owned capacity and can adopt new chips or specialize in multimodal models, text or fast inference.

  • He would rather operate an inference provider than a neocloud because enterprise customers consuming tokens can create a stronger business flywheel. The field will consolidate: he does not expect 10 major players, but does predict “a couple of multi-hundred-billion-dollar companies.” Specialization leaves room for several businesses without guaranteeing survival for today’s long tail.

10. The funding round buys supply-chain reach and more future bets

  • The episode introduces Fractile’s $720 million Series C at a $7 billion valuation, co-led by Thrive, Founders Fund, Lightspeed and Redpoint. Goodwin says getting an initial chip and systems back has remained a relatively fixed cost; much of the new capital serves purposes beyond that first manufacturing milestone.

  • Priorities include long-term component agreements, multiple future generations, networking, systems and internal experiments on model architectures and long contexts. “Multiple shots on goal” means exploring agent swarms versus serial agents and improving predictive judgment. Goodwin also highlights expansion in the US and the arrival of Meta’s Dan Rabinovich to drive systems work spanning chips, racks, networking and data-center integration.

  • Asked to invest in a supplier, Goodwin chooses TSMC: most frontier chip production ultimately traces back to it. “It is cheap at the price”; he says he would buy the stock “every day of the week.” His rationale is its central manufacturing position and extraordinary team.

  • For a customer-side investment, he favors scrappy neo-labs and names London’s Inherent, whose work includes automating science and research. Such systems need not pursue the single best general model. Among competitors, he chooses d-Matrix, which he says pursues a different architectural direction in the same space; he does not know its latest valuation.

11. Nvidia’s replacement market supports Goodwin’s bullish forecast

  • Asked whether Nvidia will be worth $10 trillion in five years, Goodwin answers “Yes.” He says Nvidia trades at a lower forward multiple than Cisco did after the dot-com crash, adding that the comparison “may be unsourced.” His explanation is that investors anticipated a bubble before the boom and underestimate how sustained spending will be.

  • The replacement cycle does not require continually adding facilities. Within finite megawatts, operators must install better chips; Goodwin says every chip deployed today will be gone within six or seven years at most, many within a couple. Nvidia’s supply-chain relationships favor the incumbent, while each replacement also opens a market-entry opportunity for Fractile.

  • Goodwin says the underlying pace of model capability advance has been extraordinary, and recent maths results have significantly affected the timelines people discuss. “Anything verifiable,” including maths and coding, “will be solved in the next 12 months.” His broader optimism concerns deployment: cures for diseases people could not previously tackle and high-quality education for everyone are “unequivocally going to come.” He sees the technology as available now, with real-world applications only beginning to emerge.