Summary

The core thesis is stated flat-out in the opening seconds: “inference is going to be the biggest market in the world. Whoever produces the most tokens is going to be the most viable company in the world.” Etched’s founders argue the entire semiconductor stack is “built on buffer” — general-purpose assumptions (like EDA defaults that assume chips run at freezing temperatures) that a pure inference company can strip out, compounding “20% here, 50% there, 2x here” into a system they claim is not 10% but “10x better” than incumbent AI chips.
Two technical bets carry the whole design. For prefill: low-voltage inference — exploiting Dennard scaling (power is quadratic in voltage) to run at “under half the voltage of any other AI chip,” solving thermal throttling before adding flops. For decode: cluster-scale memory — a fully custom interconnect above layer-2 ethernet that cuts chip-to-chip latency “by more than a factor of 5x” versus Blackwell’s ~4,000 ns hops, letting the whole scale-up cluster’s HBM and SRAM act as one pool.
Execution style is the moat as much as the architecture: extreme vertical integration (“the best vendor is no vendor”), a self-built rack, a Taiwan factory, and “pre-fetching” — 700 FPGAs running the full chip, racks pre-shipped to customer data centers without chips, thermal mock chips validating cold plates. Result: silicon back to inference-in-a-rack in 40 days, versus a “very famous AI chip company” that publicly took 10 months.
The near-death moment was capital: in early 2024, with $15M in the bank and a $100M twelve-month spending need, “every major investor in the valley passed immediately.” They scraped to a $103M Series A of soft commits, with support from a debt provider, Synopsys emulators on multi-year terms, and TSMC — won over at a conference dinner by a then-22-year-old Gavin talking tensor math with a senior VP. Patrick, an early investor, breaks the fourth wall: “you kind of have to damn the base rate.”
The structural edge over hyperscaler silicon is existential focus: “Google won’t fail if TPUs fail… OpenAI won’t fail if Jalapeño fails” — and Rob says lab/hyperscaler chips carry lower “FB8 × FB8” FLOP density than the “Blackwood B300” (as heard) because “they don’t have to go take the risk.” Supply-wise Etched claims it’s additive, not zero-sum: 4nm and different HBM than Rubens’ 3nm, so customer conversations are “2 gigawatts,” not either/or.
The futurist calls are aggressive and dated: Rob predicts 2027 is when agents outnumber humans in knowledge work, that productivity will be measured in “agents per megawatt,” and that a trillion-dollar single data center is “a matter of time.” Gavin’s model thesis: math gets cheaper faster than memory, so future models should burn compute lavishly — billion-token context, “every book ever written in its short-term memory,” giant distributed MoE “brains.”

Deep dive

1. The consensus said 21-year-olds can’t build chips — the answer was that the industry is “built on buffer”

Patrick opens with the diligence verdict: semis companies are built by 40-50-year-olds who’ve shipped multiple chips; “two 21-year-olds are not going to do this.” Gavin’s response is that the objection encoded stale constraints: “there’s a certain level of naivety required to think that you could build a chip better than every other AI chip ever built… and we had the naivety.”
The load-bearing insight: the entire semiconductor and data-center stack is “built on buffer” — every layer from EDA tools to power modules is general-purpose. His concrete example: timing sign-off “corners” default to chips running at freezing temperatures — “I’ve never seen an AI data center with ice in it” — when inference chips never run below 80°C. Killing dead constraints compounds: “20% here, 50% there, 2x here.”
Rob’s filter for early believers: heuristics people versus truth-seekers. The specimen is Mark Ross, ex-CTO of Cypress Semi (sold for $9B), who said “No, you can’t. It will not work” — then demanded a white paper and a functional simulation, was surprised twice (“Huh, this works”), advised them to raise at least $3M (they raised five), and progressed from advisor to half-time advisor to full-time CTO.

2. Two bets, one per phase of inference: low-voltage prefill and thermal-first flops

The product framing: not a chip but a full rack-scale inference solution — chip, power delivery, boards, interconnect, and “really, the production is the product.” Inference splits into prefill (reading text to set the model’s KV cache) and decode (generating tokens), which they disaggregate across separate clusters — Patrick’s gloss, “loading the gun and then firing it,” which Gavin accepts.
On prefill, the metric that matters is real flops, not headline flops: MFU on GPUs runs 20-50%, and you can’t reach 100% anyway because chips thermally self-throttle. So “if I just add more flops to a GPU today… it’s just going to thermal throttle” — the thermal problem must be solved before adding flops.
The mechanism is Dennard scaling — power is quadratic in voltage (“if I 2x my voltage, my power goes up by 4x”). When they asked dozens of semiconductor veterans how to run below GPU voltages, the answer was “you can’t” — dissatisfying, since “Bitcoin miners run at under a quarter of the voltage of GPUs.” Their new power-delivery mechanism, low-voltage inference, runs “at under half the voltage of any other AI chip,” and Gavin says: “we think all AI chips in the future are going to be low-voltage chips.”

3. Decode is a memory game — and the right question is cluster bandwidth, not chip bandwidth

Gavin’s reframe: people ask “how much memory bandwidth is on your chip?” when they should ask how much bandwidth is on your full scale-up cluster. On Blackwell, chip-to-chip hops run about 4,000 nanoseconds point-to-point, so an 8x tensor-parallel setup delivers “way, way less than an 8x improvement” in tokens per second per user.
Etched built a totally custom interconnect stack — “everything above the second-layer ethernet, built it full custom” — cutting latency “by more than a factor of 5x,” so the SRAM and HBM of the whole cluster works as a single pool (“cluster-scale memory”). As world size scales, time per token drops proportionally.
The unsentimental diagnosis of incumbents: “all these architectures were built before ChatGPT” — flop organization, voltage domains, power planes, packaging, board design, interconnect all look different when designed for modern workloads. And the physics headroom is enormous: the mathematical latency limit is speed of light, “two, three nanoseconds” — against 4,000 today. “There’s a lot of room at the bottom.”

4. Why this is the decade’s bottleneck: tokens haven’t had their economies-of-scale moment

Gavin’s macro case: real AI exists, and the constraint is now concurrency and speed — “it’s just not possible for a billion people to use these models concurrently,” and today paid AI plans reach only a few million users, “1/1000 of the global population.” Faster decode also compresses wall-clock time: an agent task that takes a year of inference-time compute becomes a month.
Patrick’s signature analogy: with iPhones, “more money does not really buy a better iPhone” — billionaire and average American buy the same one. Tokens aren’t there yet: “a general-purpose system, relatively small one, is kind of handcrafting these tokens… like they made screws back in the Renaissance.” He wants iPhone-grade economies of scale for token-making.
The practical stakes: certain products (coding models) are unusable below a tokens-per-second threshold, so the choice today is “shut off a bunch of the world from using this stuff, or everyone’s going to get a worse experience” — hence the pressure for new hardware, “from the wafer to the watt… from the transistor to the token.”

5. Origin stories: a tumor GPT-4V caught instantly, and a 17-year-old kernels engineer

Rob’s motivation is visceral: stage-four bone cancer at the end of sophomore year of high school, under 30% survival odds, two years of chemo and learning to walk again. When GPT-4V shipped, he uploaded a pre-diagnosis photo of the bump on his back; the model immediately said “this could be a tumor, get an MRI” — “that took me six months” with real doctors. Then the kicker: a “you’re all out of image credits today” notification. “Holy crap… we clearly don’t have the infrastructure to serve it.”
His second angle: running the Prod incubator (early companies included Cursor/AnySphere), he watched every startup spend its raise on compute and concluded software COGS “is not going to be zero anymore… it’s going to be a function of inference” — a “decade march for inference to become the biggest market in the world.”
Gavin’s path: kernels development at Xnor at 17 (too young to sign a contract), then ExaNous (bought by Altera for $200M) and OctoML (bought by Nvidia for hundreds of millions — “Octo” as heard). The lesson from kernels work: “the math is relatively easy… what matters is data movement” — which is exactly what cluster-scale memory attacks.

6. The robotics template: two people, no outreach, just win

In FTC robotics, Gavin and partner Sanford abandoned the standard 20-person team model and its documentation/outreach culture: “we did nothing else besides build a robot that scored the most points,” redesigning every 3 months. They held the world-record high score at one point and ranked third in the world by OPR.
The translation to Etched, teased out by Patrick (“you chose to have no communications” — “exactly like the robotics team,” Gavin admits): velocity, velocity, velocity — “you win by shipping” — and the conviction that “you can do the best product in the world with far fewer people” than incumbents’ 20,000.

7. “The best vendor is no vendor”: building the rack, the factory, and the chip simultaneously

Rob’s extension of “the best part is no part”: vertical integration from chips to boards, cold plates, interconnects, and production itself — “we’re the only startup right now that’s building its own rack as well as its own chips… at the same time.” They hired the rack lead who “built all of Nvidia’s HGX and DGX systems, which is like 80% of the revenue” (Bryan Loyler, as heard).
Before silicon returned they built thermal mock chips with the exact expected hot spots, over-pressurized cold plates until they blew up — “we haven’t had a single leak since” — and ran a Taiwan factory with cloned test stations in the office, a 2 MW data center on the floor, and 24/7 day/night development shifts.
On where integration stops: economies of scale draw the boundaries — “natural boundaries are on the chip side, on the bottom, and the model layer at the top. And we’ll fill the whole gap between.” They’re deliberately not building data centers today: “that doesn’t actually help us get more capacity online.”

8. Talent: “legends” plus “chips on shoulders” — and you have to be “sick in the head” to join

The bimodal philosophy: for unsolved problems, find the literal best person in the world via “project-based recruiting” — map every hardest technical problem ever solved, find who did the zero-to-one, and persist: “the amount of people who say yes after the first conversation is pretty low, but after the 20th conversation is surprisingly high.” Brian’s value once landed: “that’s a billion-dollar lesson I learned” — pointing to lessons learned before they’re repeated.
The other mode — “chips on shoulders put chips in data centers” — is exemplified by Sanford, asked to build a cold plate in a week (“any thermal engineer would think you’re totally naive… these things take months”) and de-risking a key power question anyway. The magic is pairing: naive first-principles risk-takers who “don’t know where the bodies are buried” plus scale veterans, working together.
Rob’s self-aware recruiting pitch, verbatim: “You kind of have to be sick in the head to join our company… move to San Jose for the semiconductor company run by two, what, 24-year-olds now… against the biggest companies in the world… with a design that they’re saying is not going to be like 10% better, but 10x better. Something must be wrong with you to do that.” His worry: as specs go public and the company gets consensus, the contrarian self-selection filter may weaken.

9. Spending money to go faster: Bangalore, pre-fetching, and 40 days versus 10 months

When a physical-design vendor fell a year behind near tape-out, both obvious options cost a year — so option three: ship dozens of top engineers to Bangalore for 6 months (Gavin lived there 4.5 months), first in at morning, out at 1 a.m., with 12-hour US handoffs at 8 a.m. and 8 p.m. daily. Other chips at the same stage with the same vendor “still haven’t taped out today.”
The economic logic for burning cash: “the biggest risk is not taking risk” — with over a billion dollars of daily category revenue, “every day we don’t ship, we’re just leaving tons of opportunity on the table.” Hence “pre-fetching”: 700 FPGAs running the full chip and a dozen models on the real inference stack, racks shipped to customer data centers without chips to bring up software, the production line ready before silicon landed.
The payoff, as told: a “very famous AI chip company” took 10 months from silicon back to inference in a rack — publicly disclosed to investors. Etched did it in 40 days, “because by the time the chip came back, everything was boring.” Over half the company lives next to the office; on the shift pay question, Rob answers: “the invisible hand does wonders.”

10. The two darkest weeks: 50 picoseconds, quitters, and “the puzzle begins”

FPGAs verify digital but not analog logic — so when silicon returned, a back-pressure failure across a clock-domain crossing produced wrong attention results. The only fix: align two clock signals on-chip to within 50 picoseconds — “50 trillionths of a second” — on every chip, 2 billion times a second. “We had people quit… people literally were like, this problem is unsolvable, and best of luck, guys.”
The method: “step one is, okay, let’s assume the problem is solvable” — a drifting mechanism to phase-shift clocks by picoseconds, then lock them. It worked in about 2 weeks — “a very scary 2 weeks” — and Gavin’s generalization: the moment things feel hopeless “is the most important time to be investing effort.”
The companion war story: first wafer sort at 2-3 a.m. with TSMC on the phone, every die on the screen turning red instead of green. The silicon-validation legend leans back: “The puzzle begins.” And the experimental philosophy behind it all — 30 board experiments, three worked, all “worth their weight in gold”: “people come to me and say, Gavin, almost none of your experiments works. And I’ll say, I only got to get lucky once.”

11. The near-death fundraise: “every major investor in the valley passed immediately”

Early 2024, pre-Series A: architecture proven, but the physical-design stage alone costs at least $40-50M, giant MoE models meant building the whole cluster, and the bank held $15M against a $100M need over the next 12 months. Gavin’s honest memory: “you’re sitting in that moment and you think, holy crap, we can’t afford this… how hard is it to go back to Harvard?” A 30-page technical memo took 100 hours; every major investor passed — “two kids that just finished Harvard, haven’t taped out a chip… everything’s going to be training… this could all be a bubble.” The biggest semiconductor Series A fundraises were around ~$40-50M.
Survival mode: a $30M “ramen to tape-out” floor plan backed by a debt provider, then calling everyone — “we need a hundred million dollars… do you know somebody that wants to take an aggressive bet?” — until a board meeting where a spreadsheet read $103 million in soft commits: “we all look at each other and we say, we’re going to take it.” Nearly half a dozen rounds since, “many of them from those investors just doubling and tripling down.”
Suppliers believed before the money did: Synopsys extended emulators on multi-year terms (“basically a big loan”), and TSMC signed on after a SEMI conference dinner where 22-year-old Gavin — the only speaker under 30 — sat by chance next to a senior TSMC VP, both math majors, talking per-tensor model mechanics on paper. Next day’s email: “Gavin, want to work with Etched? Find a way to make it happen.” Rob’s broader TSMC verdict: the tech is the best, but “the real value is all in the service” — they ran a yield experiment on their own dime at Etched’s suggestion, then rolled it across the line. “If I go to the steel works plant and say change the composition of the steel, they’ll say screw you. Not TSMC.”
Patrick breaks the fourth wall — “I’m a big Etched investor… I’m incredibly biased” — and reflects on contrarian investing: experts “laid out in very logical terms why this wasn’t going to work,” and the lesson is “you kind of have to damn the base rate… there’s always the index fund.” Gavin’s observation on who did believe: pure market/team believers, or the ultra-technical — high-frequency trading firms who audited everything down to the RTL; “if you were anywhere in the middle, you just wouldn’t understand it.”

12. The software bet: kernels-first, under 100 models, and an existential edge over hyperscaler silicon

Three years ago the choice was graph compilers (work out of the box, poor performance) or kernels-first programming; Etched went kernels-first, betting “there was going to be under 100 models that actually mattered” — no arbitrary PyTorch, CUDA, or ONNX support. Skipping the compiler “saved us a tremendous amount of time”; the only early believers were HFT firms — “they all hate compilers too” — dozens of whose engineers have since joined.
The bet is aging well as AI eats kernel-writing: tooling is designed for how models will use it, and in one internal experiment “Codex actually got GPT-OSS running from scratch just based off of our docs, completely by itself… overnight.”
The market-structure argument comes from a hire poached mid-recruitment (an architect at a frontier lab’s chip project “Uno reverse card”-recruited into Etched within a week): “It fundamentally is not existential for my company for this product to win. Google won’t fail if TPUs fail. Meta won’t fail if MTIA fails. Microsoft won’t fail if Maya fails. And OpenAI won’t fail if Jalapeño fails.” Rob’s corroborating data point: lab and hyperscaler chips show lower “FB8 × FB8” FLOP density than the “Blackwood B300” (as heard) — “they don’t have to take the risk. They just have to build a similar enough product and not pay the Nvidia tax.”
Supply is framed as positive-sum by design: first-gen sits on 4-nanometer with different HBM, versus Rubens on 3-nanometer, so scale deployers see “2 gigawatts,” not a swap. The closing discipline: think supply chain at design time, because “if you have the most performant product and you can’t produce it, you’re just a podcast.”

13. The future: giant distributed brains, agents per megawatt, and trillion-dollar token factories

Gavin’s model thesis starts from “machines don’t think like people think” — airplanes don’t fly like birds. For chips, loading data is expensive and math is cheap, and “math gets cheaper at a rate that is faster than memory gets cheaper” — so future models should burn compute: many parallel copies, gigantic experts spanning racks, billion-token context. “I would love to talk to a machine that was able to attend to every book ever written in its short-term memory.” Rob adds the near-term architectural theme of dynamism — per-token, per-user control of compute and memory — which pre-ChatGPT hardware handles with “blunt force.”
The wall-clock argument for speed, via Noam Brown (now an angel): 6-month agent tasks can’t even be evaluated before the next model ships; and just as no single human can build a rocket, agent work will need teams — “maybe that’s 10, maybe that’s a million” — demanding colossal shared memory and flops. Cursor’s agents built a browser from scratch in a week; “that will soon happen in under an hour.”
Rob’s boldest dated calls: inference on a “global march” to a majority of global GDP (may take more than 10 years), productivity re-denominated as “agents per megawatt,” and “this is the second-to-last year a majority of the workforce is going to be human — in 2027 there’s going to be more agents doing knowledge work than humans.” On a trillion-dollar single data center: “Absolutely. It is a matter of time” — economies of scale don’t stop at $40B fabs, and the same holds for “plants that make tokens.”
The closing framings for Patrick’s “smart alien”: Gavin — thinking is valuable, every company runs on it, and someone must build the roadmap for “the future quadrillion-parameter models for a billion people all at the same time.” Rob — the cost of producing intelligence is so far below its value that “we are in a many-year, probably many-decade supply shortage of these tokens.” The episode ends with Rob’s answer to the kindest-thing question: at 16, asked to choose surgery (live, maybe never walk) over radiation, then needing radiation anyway from one of the few machines in the world, in Boston — and both parents dropping everything to move there with him.

Grant Sanderson (@3blue1brown) – AI disproved a famous math conjecture. Now what?Why Hardware-Software Co-Design Is AI's Real 100x: Dylan Patel of SemiAnalysis