Why AI Labs With Unlimited GPUs Still Fail — Anjney Midha, AMP
Summary
Anjney Midha argues that money and compute do not guarantee shipping; aligned execution and culture are the constraints. He says Google treated roughly 95% node utilization as outage territory, while best-in-class MFU is 60–70%; poorly coordinated labs let small planning errors compound across organizational layers. “AI scaling should be putting a premium on the value of common sense.”
Amp’s proposed answer is a neutral compute grid that pools fragmented supply across clouds and silicon. The goal is to make “megaflops flow like megawatts,” guaranteeing participants base capacity while dynamically allocating spikes. Amp has started securing demand at a stated 1.3-gigawatt scale—roughly $40 billion of cloud spend—but Midha estimates its teams could need 6 gigawatts of spike capacity over four years.
Data centers need an explicit bargain with their host communities or permitting risk becomes infrastructure risk. Swyx relays an estimate that up to 20% of U.S. data centers this year may lack sufficient community support, though he cautions it could be overstated. Scott Nolan’s proposed “new AI deal” would raise compute from $4 to $4.50 an hour and return the incremental $0.50 directly to locals through cash; Swyx also suggests cheaper electricity. As a compute customer, Swyx says he would gladly pay.
Alternative AI chips can expand supply without fragmenting every layer of the stack. Matrox chose NVIDIA’s reference architecture and rack footprint, then concentrated innovation on the logic die and systems co-design: “You just can’t fight on every front.” The deeper constraint is trust—chips take roughly two years to tape out, so designers need early visibility into changing model architectures.
Research hoarding inside vertically integrated labs creates an opening for independent capital and infrastructure. Midha says DeepMind’s six-month internal embargo can become permanent when work appears commercially useful, producing an adverse-selection problem in what gets published. Amp’s Foundry arm therefore backs frontier teams—including a stated few hundred million dollars invested in Anthropic earlier this year—while Amp also donates excess compute to nonprofits and university labs.
End-of-life prediction is Midha’s clearest example of outputmaxxing with public value. He says Stanford’s longitudinal dataset covered at least 12 million patients, while more than 30% of Medicare and Medicaid spending at the time went toward end-of-life care; even regression and simple neural nets appeared technically useful. Regulation, not modeling, remained the block because liability could not shift from physicians to an AI system: “I haven’t been able to get this out of my mind a single day for the last 14 years.”
Culture is the fragile compounding advantage behind Anthropic’s coding breakout. Midha rejects the “lucky dice roll” explanation: Anthropic spent four years becoming prepared under scarcity, made coding its P0 because computer-use capability could advance AGI, and endured “21 noes” that forced clarity. “Culture is not a set of beliefs. It’s a set of actions”—a garden requiring daily reinforcement, not a permanent moat.
Deep dive
1. Underutilized clusters expose an alignment failure
Midha separates node allocation from model-flop utilization: roughly 96% node use should be standard, with Google reportedly treating 95% as an outage, while today’s best MFU sits around 60–70%. Most clusters he discusses, he says, fail even that first test.
His diagnosis is organizational: the funder, cluster deployer, scheduler, and person measuring output sit too many degrees apart. Like two lines starting only slightly off-angle, small misalignment becomes enormous wastage at scale.
The remedy is iterative bring-up borrowed from semiconductor practice, not an AI-specific invention. New capabilities do not justify abandoning operational discipline: “This time is different” may hold for capability, but not for infrastructure common sense.
2. Compute pooling can outperform full-stack ownership
Swyx’s challenge—worth keeping: wouldn’t vertically integrated operators such as xAI or OpenAI have the strongest incentives? Midha’s systems answer is that utilization can rise through either integration or pooling; Amp deliberately chooses the second architecture.
The Amp Grid is multi-cloud and multi-silicon, coordinating stranded compute at the scheduling and economic layers. Its ambition is to make “megaflops flow like megawatts,” turning otherwise non-fungible capacity into a shared resource.
Midha models Amp as an independent system operator, not a neocloud. Historical grids worked when neutral coordinators pooled generators and uncorrelated customers—a steel factory peaking at night and a shoe mill by day—without owning the underlying assets; PJM Interconnection is his mature analogue.
Amp says it is assembling supply and demand around a 1.3-gigawatt, four-year scale, while emphasizing that all capacity is not yet secured. The desired steady state is 1.3 gigawatts of base-load capacity plus roughly 6 gigawatts of spikes over four years.
3. Dynamic allocation still needs command authority
Amp’s scheduling model echoes Google’s Borg/GQM implementation and Midha’s a16z Oxygen program: teams receive guaranteed base capacity, queue interruptible jobs, and spend credits to reprioritize them. Midha illustrates Nano Banana 2 receiving 10 credits against Genie 3’s five, allowing the former to interrupt the latter; he labels this a made-up example.
Swyx supplies the critical counterexample: the Brain Marketplace was real, yet Midha says the internal Google credit approach led to Google missing GPT. Midha’s broader holding-company structure pairs pooled infrastructure with a Foundry capital business that can incubate and back frontier labs.
4. Social license is becoming a compute input
Swyx relays an estimate that up to 20% of U.S. data centers this year may struggle to obtain community support, while explicitly warning that the figure may be overstated. Jobs alone are insufficient when residents bear grid, environmental, and permitting costs.
General Matter founder Scott Nolan’s proposal makes the bargain concrete: if compute costs $4 an hour, charge $4.50 and transfer the extra $0.50 to the host community. Swyx says he would gladly pay as a compute customer because cheaper local electricity or direct cash makes residents partners—and the resulting compute more reliable.
Midha expects audits and investigations to catch up with operators who “move fast and break things.” His preferred partners are 20-plus-year data-center veterans with land, power, credit histories, and experience surviving internet boom-bust cycles: they may not sponsor NeurIPS happy hours, “but they’re adults.”
5. Independent infrastructure can unlock hoarded research
Midha sees a market failure inside major laboratories: commercially promising DeepMind research can enter a six-month embargo and remain unpublished indefinitely. That creates adverse selection—the public may see precisely the work the business considers least valuable.
Foundry, Amp’s capital arm, incubates and finances independent frontier laboratories; Midha says it invested a few hundred million dollars in Anthropic earlier this year. Amp also donates excess compute to nonprofits and university labs.
His deliberately comic scale check—“We only have 1.3 gigawatts of compute. That’s nothing”—comes with an estimated $40 billion cloud-spend equivalent. Even that pool falls short if teams simultaneously pursue superconductors, healthcare models, and other frontier programs.
6. End-of-life AI shows where outputmaxxing becomes public policy
As a Stanford bioinformatics graduate student, Midha joined Nigam Shah’s end-of-life prediction work because Stanford’s longitudinal dataset covered, in his estimate, at least 12 million patient lives; he says the only larger set was at the VA. At the time, more than 30% of Medicare and Medicaid spending reportedly went toward end-of-life care.
The practical failure begins with uncertainty: a terminal patient might hear “six months to six years,” while malpractice risk discourages physicians from giving sharper recommendations. Patients then “try it all,” spending their final weeks in hospitals rather than choosing knowingly how to use their remaining time.
Midha’s cultural contrast is personal, not merely technical: growing up in Chennai, he saw death framed as another stage and marked by celebratory processions. In the U.S. system, he found that uncertainty defaulted toward postponing death.
The modeling itself appeared tractable—“even regression models work,” alongside simple neural nets, with RL offering more today. The unresolved barrier is regulatory liability: the burden of a wrong clinical diagnosis cannot be shifted from the physician to an AI system. After 14 years, Midha still calls patient empowerment “the hill I would like to die on.”
7. Outputmaxxing means scaling without losing alignment
Midha calls the proposed discipline the “Department of Outputmaxxing,” or says he would probably call it a Department of Alignment in a more formal setting. Throwing 500 or 500,000 GB300s at a suboptimal scaling plan is not obedience to the Bitter Lesson; Anthropic’s decision to standardize on the Transformer architecture illustrates how focus can unlock velocity.
Scaling organizations add specialization, interfaces, and abstractions, each introducing communication loss. His engineering question is whether systems can scale up and out without lossy transmission between layers.
He sees two escape routes: standardize protocols until coordination approaches losslessness, or create enough new abundance that the loss matters less. A room-temperature superconductor is his maximal example—lossless energy transmission and, in his telling, “flying cars” within years.
8. Compatible silicon moves innovation to the real bottleneck
Swyx asks about non-NVIDIA accelerators. Midha’s counterexample is Matrox: it adopted NVIDIA’s reference architecture and physical I/O footprint so its chips can plug into sites that already have an NVIDIA bring-up plan.
That choice leaves the company free to pursue logic-die and systems co-design gains. NVIDIA’s published standard becomes enabling infrastructure rather than merely a competitive moat, especially because, in Midha’s view, NVIDIA cannot meet total production demand.
The hard dependency shifts to trust. A chip needs about two years to tape out; if the model architecture changes before launch, the design may be stranded. Google’s internal teams could co-design beside Gemini or PaLM researchers, while an independent founder loses that privileged visibility.
Midha sees Amp’s cross-lab relationships as a bridge across this trust boundary. The grid could also connect exchanges such as SF Compute bidirectionally, but near-term scarcity is acute: he says the last six weeks eliminated Amp’s expected year-end excess capacity, while founders who had raised billions were texting for “15 nodes.”
9. Scientist-founders win through precision and maintained culture
Midha rejects the investor habit of treating people as static categories. Researchers who reach the top of their discipline at places like Berkeley are “athletes of the mind,” already practiced at securing resources, earning organizational trust, confronting technical convictions, publishing work, and putting it out.
The additional CEO requirement is confrontation beyond science—across hiring, the team, recruiting, and customers. He cites Anastasios of LM Arena, Robin of Black Forest Labs, Guillaume of Mistral, and Dario of Anthropic as evidence that elite research performance can transfer into company leadership.
Precision also dissolves false competition. “World models” hides several distinct problems; “real-time action prediction models” reveals the actual work. Researchers operating at that resolution often seek collaboration, while business abstractions manufacture simplistic stories about who is winning.
Anthropic’s coding breakout is Midha’s culminating specimen. It may have benefited from luck, but “luck favors a prepared mind”: four years of preparation under scarcity, efficiency, and safety conviction, plus “21 noes,” forced coding into the P0 slot as a path toward AGI.
Culture therefore is not a static moat but “a very brittle, fragile thing” tended daily through action. Midha says cash-rich labs that never face hardship may never define their hill; at Periodic Labs, he opposed rehiring someone who left for more money and returned only after a SOTA breakthrough because the decision itself would rewrite the culture.