田渊栋

田渊栋’s startup was triggered by a hands-on experiment last August: after co-writing a Grokking paper with GPT-5, he realized the model was already capable enough that he might not need to hire interns. Ideas, research directions, and theorem proofs could be worked out through conversation, with the coding handed to AI; the research validation loop went from days or weeks in the advisor-student model to minutes—“Rather than be replaced, I might as well start a company.” Recursive Super Intelligence emerged from 4 months of stealth in early June with a $650M Series A at a $4.65B valuation.

The central debate is over Silicon Valley’s popular assumption that the first company to reach RSI will pull irreversibly ahead during a window of opportunity. 田渊栋 thinks the logic is “still broadly correct,” but the decisive variable is the shape of the curve: intelligence gains will follow an S-curve with plateaus, not a smooth progression. If scaling laws hold perfectly and the curve is smooth, startups have no chance; but scaling requires 10x compute and 10x data for linear gains, and “you can’t have all the electricity on Earth supplied to one person for self-evolution.” The real bottlenecks are already visible, leaving room for breakthrough-driven paths.

RSI is “much bigger than a coding agent”: coding agents are merely executors, while open source is flattening that capability. He feels the gap between GLM-5.2 and Opus 4.6/4.7 is narrowing—“4.6, 4.7, and 4.8 aren’t that different”—while RSI’s real deficit is research taste: the abstraction and creativity to “see precise, detailed connections from a small number of examples.” Large labs may prioritize coding businesses as they push toward IPOs, creating an opening for new labs; “everyone I talked to has left.”

田渊栋’s read on Anthropic’s When AI Builds Itself is deliberately cooler: Claude’s 800-hour effort narrowed the weak-to-strong gap by 97%, but it is still productivity enhancement rather than recursive self-improvement. Humans completed 23% of the work in a week; the method remains conventional and score-driven, making it only the first rung of self-evolution. OpenAI’s timeline is an AI research intern by September this year and a genuinely automated AI researcher by March 2028.

Recursive Super Intelligence’s first 3 results all reached SOTA using the same general-purpose system. NanoChat’s 5-minute training run beat the community on BPB, the speed run moved from the community’s 79-second SOTA after 2 years of work to 77 seconds, and operator optimization beat Double AI—a specialist GPU team led by former Mobile AI CEO Sasha—by 10% on Nvidia’s new SOL Exact Bench. The team had no GPU specialist; the starting point was “1 person plus AI reaching SOTA.”

Beyond scaling, he is betting on interpretability and considers himself a minority voice. While “most people may already feel hopeless,” he believes there must be a good principle that makes models work; AI will eventually become a science, as alchemy became chemistry, through a Tycho-Kepler-Newton progression. The next Newton “could be AI, could be humans plus AI.” His organizational view is equally blunt: “this wave of AI is anti-big-tech,” teams start breaking down after 150 people, and Llama 4 is the example.

🔗 Original source & video: 178: Talking RSI with 田渊栋: How Will Model Self-Evolution Arrive?

Coding agents could create a data flywheel, a path 田渊栋 sees as capable of accelerating in specific areas. Early on, humans wrote the code and data accumulated slowly; once AI-generated code becomes good enough to produce large volumes of training samples, “suddenly, there will be a takeoff.” After leaving Meta, he caught the Claude Code and Codex wave and realized, “I don’t need to write code anymore.” He rejects the idea of an AI pullback in 2026 and expects many applications to break out: 2 months of coding work could shrink to 2 weeks or less. “For VCs, that is the bigger story—they want to know whether it can make money.”

“I don’t think following scaling laws alone can get us to AGI,” but scaling law itself will not top out—the constraint is resources. Data centers may need small nuclear plants for power, then 10 times more, then another 10 times more, until the Earth’s entire energy supply is serving data centers. The question is which path produces the greatest impact under finite resources; large and small companies may take different routes. His conclusion: “One wave is not enough to take us to AGI.” The ability to jump between waves matters more.

Against the backdrop the host described of Llama 4 falling out of the lead, 田渊栋 primarily blamed organizational pressure and an excessive need for speed. The team grew from dozens of people to 200-300 or more while rushing to catch DeepSeek and other reasoning models. Multiple layers of VPs passed good news upward, so senior leadership may have heard only positive reports until the model shipped and the problems surfaced. He also warned that optimizing only for benchmark scores encourages teams to stuff things into datasets so models look good on tests but perform poorly in practice.

The large-language-model race has squeezed the diversity out of research, one of the major reasons he chose to start a company rather than join another big tech platform. He received offers from Amazon, Apple, xAI and others; the VP who recruited him to Amazon left 2 months later, and Apple went through a similar episode. The FAIR-style bottom-up betting culture has been crowded out by competition for resources. He is now the founder of a stealth company; some of his post-departure research was completed on a single 4090 at home, and he enjoys the speed of a one-person loop.

He partly agrees with LeCun that the core problem with today’s approach is learning efficiency, not an inability to learn anything at all. The human brain runs at roughly 20 watts, perhaps 20-30 watts, and can learn from very little data. He uses his daughter as an example: before she reaches a certain developmental stage, flooding her with data produces memorization; then one day she grasps the concept and learns quickly. But he rejects the “parrot” characterization: LLM representations are not purely memorized, touch some underlying truths and reflect the real world, but are inefficient to learn. The real problems for 2026 include data efficiency, inference efficiency, continual learning and architectures beyond Transformer; the probability of a paradigm-level breakthrough is currently not high.

The host raised the experiment involving nearly 300T tokens; 田渊栋 says its biggest payoff was cognitive. We now know that code can be written without a human writing it, and that realization alone was worth the cost. OpenAI, Trae and other companies spend substantial sums on R&D; if they were satisfied with today’s models and focused only on serving and inference, they could make money, but the next generation requires continued investment.

The AI flood is a prisoner’s dilemma; the individual response is to choose higher ground, not build a dam. Many people signed the 2023 call for a 6-month pause to address safety, but it went nowhere because whoever stopped risked falling behind. The flood will keep rising, so people should find a good position before it arrives. Around what he calls “the Fermi level of human society,” the minimum requirement is that human plus AI be greater than AI alone, using judgment and vertical expertise to identify and correct AI’s mistakes. The episode extends the logic further: as AI supplies repetitive labor without limit, scarce human value lies increasingly in choosing what to bet on and articulating what one wants.

🔗 Original source & video: 69. Interview with 田渊栋: The Real Problems with Large Models, the Shifting Landscape, the AI Flood, and The Path Not Taken

田渊栋 sees data as the most important moat, infrastructure as important too, with algorithms and talent secondary; compute mainly separates Big Tech from startups. His reasoning: AI coding has made him “at least 10x” more efficient than 3 months ago, suggesting the infrastructure moat may erode; algorithms are in a phase where “either everyone keeps tweaking without getting anywhere, or someone comes up with something completely different”; and rapid talent mobility means “it’s hard for any secret to stay secret for long in Silicon Valley.” New approaches will be known more broadly within 1-2 months or 2-3 months.

Memory and storage demand is, in his view, a hard physical bottleneck that is difficult to solve. The host noted that the supply chain is already undersupplied; 田渊栋 attributed incremental demand to longer contexts, larger training runs, and intermediate outputs for multimodal 4K images. Model sizes have gone from open-source models of roughly 70B parameters to around 1T for Kimi K2 and 600B+ for DeepSeek; with its larger memory, H200 can deliver the same performance with fewer cards. On potential solutions, he said, “It’s genuinely hard to see a good solution right now.”

The ceiling for reinforcement learning and test-time scaling may be locked in by pretraining. Pretraining provides the raw material for the one correct path among 100 possible ways of thinking, while RL searches for and amplifies it; if “the knowledge simply isn’t in pretraining,” reinforcement learning cannot come up with a solution either. Test-time scaling may therefore gradually approach its ceiling, which is why the industry is turning toward continuous learning and trying to change weights during both post-training and inference.

The central dispute in memory research is between storing everything (expensive, slow, accurate) and selective forgetting (cheap, fast, forgetful); he believes true AGI should have “fixed brain capacity while continuously consolidating memories and actively forgetting.” Unlimited data accumulation looks more like Internet-style retrieval and does not automatically become a system with deeper understanding of a problem. The generational contribution of large models is integrating knowledge into weights. The hardest problem is the leap from rote memorization to “insight”—like a child who suddenly develops an intuition for the relative size of numbers after age 4, a mechanism that remains unknown.

Distillation makes it hard to hold a lead for long, while the labs’ race maps to different survival strategies. Big Tech companies such as Google use Gemini 3.1 Pro to demonstrate technical strength and talent depth; startups prove they are strong enough to raise money, then need to find a business model before the cash runs out. OpenAI is also considering ads in ChatGPT to create cash flow.

Agents could upend e-commerce and transactional human interaction, while creating a “herding effect.” For an Agent, “every webpage is one connection,” and ads have no desire to exploit. Agents on both sides could coordinate meals, meetings, and other routine tasks, and may evolve into personal assistants covering food, clothing, housing, and transportation. A plumber who does not use a bot that can take orders 24 hours a day and automatically plan routes could be less efficient than peers and be forced out. After using Clawdbot for only 2 hours, he felt “more uneasy the more I used it” because it demanded various API keys, comparing the security risk to “a child who holds all your secrets but isn’t smart enough.”

He warned of AI’s social shock and disclosed his next move. “The flood is coming, but many people still haven’t felt it”: future layoffs could reflect an industry-wide change in logic, leaving existing skills useless everywhere. He has left Meta after nearly 11 years and will join a startup with an undisclosed direction and team as co-founder; the company is raising a Series A that is “almost done, with many people willing to invest.”

🔗 Original source & video: Silicon Valley Coordinates x Former Meta AI Director 田渊栋: Unpacking Foundation-Model Moats, the Memory and Storage Bottleneck, and Agents’ Social Impact

The most industry-relevant “prophecy” in Within the Dawn is that the compute race will shift from the performance of individual GPUs to interconnects, communication bandwidth, and information compression. In 2020, 田渊栋 cast the consciousness-bearing “Spirit Realm Cubes” as an allusion to NVIDIA GPUs: superintelligence emerges when the cubes are linked at high speed, while the Galactic Alliance imposes a bandwidth blockade. Reality then produced NVLink, NVL72, CloudMatrix 384, and DeepSeek V3’s use of FP8 on H800 to reduce communication pressure; he had imagined this as science fiction from 10 or even 50 years in the future, only to see “science fiction become reality two years later.”

Virtual worlds and interstellar civilization are not mutually exclusive; humanity is more likely to digitize its existence while sending its computational substrates onward to explore the universe. Human bodies are fragile and lifespans finite, while sending astronauts into deep space requires food and biological support; once consciousness enters a virtual world, its substrate can travel to Proxima Centauri and beyond. 田渊栋 believes humanity entering virtual worlds is “unavoidable,” but curiosity will not disappear: the final form may be “entering the virtual world on one hand, and continuing to wander among the stars on the other.”

Once technology reaches “whatever you think of, you get,” the scarcest resource may no longer be productive capacity but genuinely original ideas. The Galactic Alliance can instantly materialize factories, spacecraft, and any object it imagines; what it fears is taking the wrong path, because a remote civilization with a better solution could immediately become a replacement force using the Alliance’s own technology. The Alliance therefore has to preserve Earth’s diversity of thought like a nature reserve while keeping that exploration under control—it does not want its own ideas to contaminate the reserve, nor allow it to grow strong enough to turn against the system.

AI’s impact on employment may unfold in 3 stages: displacement and emptiness, competition for uniqueness, and finally an explosion of occupational diversity. In the short term, people will find the jobs they depend on replaced by AI, or discover that no amount of effort can make them better than AI; in the middle stage, they will be forced into attention contests built around novelty and eccentricity; only in the long term might they leave the “education—skills—work—wages—raising the next generation” loop and turn writing, painting, or singing into life itself. 田渊栋 is optimistic about the end state but stresses that “what twists and turns happen in the middle is hard to say,” while AI’s descent into industry-specific workflows will continue over the next 2-3 years.

Today’s frontier models can assist with research but have yet to cross the threshold at which top researchers infer fundamental rules from a handful of clues. Deep Search is already good at finding material, but it remains “an outsider who looks like an insider, while insiders see an outsider”; a model may need 1,000 or 10,000 samples, while an excellent researcher can infer the real minor factor from 1 or 2 anomalies. 田渊栋 does not believe more pretraining data alone will close the gap, is “not that optimistic” about replacing researchers within 5-10 years, and estimates that a breakthrough in the training paradigm may still take 10-20 years.

Generative AI is pushing technical organizations toward smaller, sharper teams, hands-on managers, and core staff amplified by multiple AIs. 田渊栋 went from arguing in 2021 that leaders should not bury themselves in details to requiring himself in 2024 to read and write code again, because purely managerial leaders lose technical sensitivity and “one person plus a lot of AI” may outperform a small team. The OpenAI, Hugging Face, and DeepSeek examples discussed on the show all point to the same trend: leaders who can quickly judge frontier shifts such as o1 Preview while implementing them personally can accelerate organizational pivots and execution.

AI has entered the novel-production chain, but the most valuable ideas, structures, and turns still come from humans. DeepSeek is good for brainstorming but prone to logical breakdowns; Claude 3.5 is better at sensing relationships among supporting characters, Gemini 2.0 at detailed description, and 4o relatively flat; all share problems with long-context forgetting, formulaic plots, and “the prince and princess living happily ever after.” 田渊栋 uses Cursor to build his own collaboration tool: humans write the outline and key passages, while models fill the gaps. His view is that procedural writing will be standardized, while personal experiences that have “never been explored before” will become the fresh knowledge AI needs most.

🔗 Original source & video: 116: When an AI Researcher Writes Science Fiction: Talking with 田渊栋 of Meta About His Vision of Intelligence—We Will Eventually Get Whatever We Think Of