Pioneers Insight Method Research Author
‘Hard Fork’ Live, Part 3: Differing Visions of an A.I. Future
Back to Episodes

‘Hard Fork’ Live, Part 3: Differing Visions of an A.I. Future

Summary

  • Daniel Kokotajlo now puts a 50% chance on AI that can conduct its own AI R&D by late 2028, probably slightly later than Anthropic expects. He expects no intelligence explosion in 2026, but thinks coding agents could fully automate coding within one or two years, shifting bottlenecks to research taste and management; even “99% automation” might compress “a decade or two decades worth of research in a year perhaps.”
  • Sayash Kapoor’s counter-thesis is that progress in coding does not prove that every economically important domain can be automated on the same curve. Code offers instant, objective feedback, while law retains unreliable outputs even as models improve because “even the right answer is not obvious to a domain expert.” The investment crux is whether compute can remove the remaining bottlenecks or whether real-world learning, reliability, and sample efficiency require unknown breakthroughs.
  • The two forecasters agree on more of the near term than their rival labels suggest. Kapoor found AI 2027 plausible through the end of 2026, while Kokotajlo accepts that AIs short of “humans in the cloud” remain normal technologies; both say that once AI matches the best professionals across computer-based cognitive work, the normal-technology framework stops helping. Their policy overlap includes transparency and external scrutiny, but they differ over whether more aggressive scenarios warrant a conditional slowdown; Kapoor says his normal-technology view gives near-term diffusion benefits more weight.
  • Dwarkesh Patel sees an unsettling capability overhang: current models are already powerful despite remaining far from human intelligence and learning efficiency. Digital minds can think “thousands of times faster” and absorb knowledge across domains, while humans may learn new things “literally a million times faster,” retain knowledge across sessions, and improve on the job. The consequential question is what happens when models retain their digital advantages while acquiring ours.
  • Actual workplace evidence supports productivity gains more strongly than full-job automation. Patel said most tokens he reads are AI-generated, and Casey Newton can obtain a podcast briefing in roughly four minutes that could once have been hired out; yet Patel’s one-hour sponsor negotiation or the coordination required to book a live show remains beyond reliable automation. His blunt AGI check: “We all have jobs.”
  • Continuous learning may separate enormous commercial value from genuine superintelligence. One camp expects sufficiently long contexts and varied RL environments to substitute for updating model weights; the other notes that employees can take six months to become net productive because experience is distilled into abstractions, not merely stored as an ever-growing transcript. Patel thinks the former route might still support “a trillion dollars in revenue” without producing a system that can acquire real-world political judgment on the fly.
  • Humanoid robotics remains a data-acquisition market, while nearer-term industrial deployment appears more credible in controlled settings and quadruped inspection. A Unitree humanoid with dexterous hands costs roughly $50,000-$70,000; George Ekas expects factory tasks within the next couple of years but household chores “a few more years” out. Unitree’s logging traffic to China and proposed US import restrictions add security and import-policy concerns before the household use case is proven.
  • The labor and corporate narrative may turn before the technology does. Kevin Roose expects companies to tout AI restructuring only while markets reward it, then rename AI-related layoffs once backlash outweighs the premium; audience concerns centered on vanishing entry-level paths, privacy, and education for an unknowable labor market. The hosts’ upside case was concentrated in accelerated science and medicine, personalized learning, and making software-building accessible and enjoyable.

Deep dive

1. Full AI research automation moves to late 2028

  • Kokotajlo’s updated estimate is “probably 50% by late 2028” for systems capable of conducting their own AI R&D. He thought that was slightly later than Anthropic’s expectation and offered the forecasting lesson behind his revision: “Things take longer than you plan for.”

  • His scenario contains no intelligence explosion in 2026; that arrives the following year. By “intelligence explosion,” he means fully automating AI research so that an already fast research process accelerates further—not an instantaneous jump to a system that can do everything.

  • The proposed sequence starts with coding agents becoming steadily stronger and perhaps “fully” automating coding within one or two years. Research taste, management, and other non-coding skills then become the constraints; companies train models against those bottlenecks, and only after the complete research loop is automated does broader superintelligence become “probably not far off.”

2. Reliability, not raw capability, limits real-world adoption

  • Kapoor locates the disagreement in whether every obstacle to automated R&D is computational. Programming supplies simulators, virtual environments, and abundant feedback; achieving human-like sample efficiency “across the board” requires progress in domains where experiments interact with the slower, messier real world.

  • His best specimen was a bullish lawyer who expanded the size of tasks delegated to AI as models improved, only to find hallucination and unreliability persisting. In law, unlike code, an expert cannot always run the answer and inspect a definitive output: qualified professionals may reasonably disagree about the correct approach.

  • Kapoor called for evaluations of a much higher standard than those available today before drawing broad conclusions about “humans in the cloud.” His own evaluations used by Anthropic were saturated by Opus 4.5—“Look, this is solved now”—but solving well-specified tasks does not establish that researchers know the bottlenecks, architectures, or breakthroughs required for the next level of generality.

  • Kevin Roose’s pushback — worth keeping: successive models keep blowing through benchmarks and forcing new evaluations, which makes lab confidence easier to believe. Kapoor accepts that this progress will continue “as long as we can specify things well enough”; he disputes only the assumed endpoint.

3. Recursive improvement can happen without ending in ASI

  • Kapoor argues that recursive self-improvement began roughly six decades ago: compilers, frameworks, systems, and libraries let humans build better tools with prior tools. Compilers alone made programming about “two orders of magnitude better,” while modern libraries compress work that could take years or decades in assembly.

  • That history makes better AI research models entirely plausible without proving artificial superintelligence. Teams of humans using AI might continue outperforming AI alone, leaving the endpoint as vastly more capable models rather than systems superior to the best humans at every task.

  • Kokotajlo’s reply is that none of the candidate barriers to Anthropic’s stated plans looks strong. Models may remain less data-efficient than humans, but companies could improve that rapidly—or bypass the requirement: “99%” automation of AI research could still produce a decade or two of research in a year.

  • Their shared boundary is “strong AGI,” also described as “humans in the cloud”: systems performing all computer-based cognitive tasks as well as professional humans, perhaps the best professionals. Kokotajlo agrees weaker systems remain normal technologies; Kapoor agrees that at this threshold, the normal-technology thesis “stops being accurate or helpful.”

4. Both camps distrust long-range forecasting—and their own readership

  • Kokotajlo’s meta-case rests on repeated failed declarations about what deep learning cannot do: alleged walls are “smashed through almost as soon as people make the claims.” Data efficiency could become the next barrier to fall, even if the method is not currently visible.

  • Kapoor distinguishes near-term forecasting inside the field’s “event horizon” from predicting paradigm changes. The community can extrapolate current work but has been poor at anticipating transformative shifts, including its long dismissal of neural networks before a small group of researchers and large datasets helped overturn the consensus.

  • That history also cuts against today’s consensus. Kapoor worries that the community is herding around transformers, potentially sidelining architectures needed for more data-efficient systems; even discovering such architectures would not guarantee human-level sample efficiency.

  • Both saw their arguments conscripted by political and ideological camps. Kokotajlo calls publication a “leap of faith in humanity” that open reasoning may improve decisions; Kapoor’s shock was “how few people read things in depth,” despite his essay’s opening comparison of AI with the internet, the electrical revolution, and the First Industrial Revolution.

5. Near-term policy converges even when tail risks do not

  • Kapoor defends “normal” as a relative, not dismissive, label: he expects impact comparable to the internet, while Kokotajlo may see “the most important invention in the history of humanity.” Saying “today’s AI is normal technology” still allows for profound social consequences.

  • Both prioritize transparency and the ability of external third parties to see what is happening inside companies. Discussing Anthropic’s release of Claude Fable-5, Kapoor called purposeful degradation on AI-R&D tasks “a very dangerous precedent” and argued that companies should not fine-tune models to mislead customers; he presented this as a point of agreement with Kokotajlo.

  • Their main policy divergence concerns conditional slowdown. Kapoor said that in more aggressive scenarios one might want a slowdown or a pause, but from his normal-technology view the benefits of diffusion and more capable systems outweigh the risks somewhat more in the near term. Strikingly, Kapoor and AI 2027 co-author Thomas spent hours searching for near-term differences and found none through the end of 2026.

  • Kokotajlo ranks loss of control first and concentration of power second. Kapoor is more alarmed by military AI: “kill bots” require no further breakthrough and can be built from off-the-shelf computer-vision libraries, making nation-state choices “pretty damn alarming.”

6. Humanoid robots are collecting data before doing chores

  • Toby the Unitree robot danced, fell hard, temporarily stopped responding, then recovered—a live illustration of both durability and immaturity. Ekas said humanoid demand currently comes primarily from researchers collecting task data and training control policies for different verticals.

  • The more practical industrial product today is the quadruped, or “dog robot.” Customers can attach LiDAR and other sensors for inspections or security patrols, making it easier to deploy than humanoids operating at the research frontier.

  • Pricing rises with manipulation capability: a humanoid equipped with dexterous hands for task-data collection runs about $50,000-$70,000, “like a mid-range sports car.” Ekas expects controlled factory work such as loading parts into equipment within a couple of years, citing early work by Figure, Unitree, and BMW; household deployment comes later.

  • On security, Ekas acknowledged that Unitree sends logging data to China but said no camera feed or joint-telemetry transfer had been established. Proposed US restrictions on Unitree imports would be “problematic”; he offered no further plan if Chinese humanoids were broadly banned.

7. Powerful assistants still leave the whole job intact

  • Patel’s framing: models already think thousands of times faster and absorb broad knowledge, yet humans learn new things perhaps a million times faster, retain information across sessions, and learn on the job. “What happens” when digital minds gain those human advantages while retaining their native ones is the scary question.

  • AI nevertheless dominates Patel’s information workflow: most tokens he sees each day are machine-generated. Newton similarly uses models to turn recent public material about an unfamiliar guest into a briefing in about four minutes—a discrete research job he could previously have hired someone to perform—without working less or spending less time at a computer.

  • Patel’s bearishness is explicitly relative to an “absurd timeline” where friends discuss a singularity in two years. A model still cannot reliably conduct the contextual back-and-forth of a one-hour sponsor negotiation or coordinate an event in another city: “People really underrate the range of human, even white-collar, work.”

8. Computer use and continuous learning remain separate bottlenecks

  • Computer use shows that verifiability alone is insufficient. Training requires many deterministic parallel rollouts, but a live service will resist that load—“if you try to do that on Amazon, Andy Jassy will just shut your ass down”—so labs must build labor-intensive clones of sites such as Amazon and Slack.

  • Continuous learning poses the deeper challenge. A strong model may outperform an intern on day one, while the intern pulls ahead after two weeks; employees can take six months to become net productive because experience is distilled into higher-level abstractions, not accumulated as perfect episodic recall.

  • One camp therefore expects user-specific learning to require weight updates between sessions. The opposing view is that models can spend the equivalent of six months inside a vast context and, after training across sufficiently varied RL environments, learn to adapt to whatever situation appears.

  • Patel’s honest uncertainty: long-context adaptation might generate “a trillion dollars in revenue” and other “truly ludicrous outcomes” without reaching superintelligence. A model as politically capable as Henry Kissinger—or LBJ, since “the example doesn’t matter”—cannot train inside an obvious data-center environment; it may need to learn directly from the world.

9. Markets reward AI restructuring before society absorbs the costs

  • Roose expects disclosure incentives to reverse. Companies currently receive a “weird market premium” for claiming AI productivity and layoffs, sometimes masking earlier over-hiring; once backlash grows, they may continue restructuring while calling the resulting job cuts something else and sweeping AI’s role “under the rug.”

  • An audience software engineer saw hiring concentrated among senior engineers who can architect systems and fact-check models, raising a missing-entry-rung problem. Newton cited labor economists who say current conditions remain well short of the Great Financial Crisis, while conceding they could worsen and that telling graduates their first job may simply be bad is hardly reassuring.

  • Education faces an even longer-duration mismatch: schools prepare children for fixed occupational targets, while credible AI forecasts barely extend two years. On privacy, Newton favored privilege-like protection for some chatbot conversations and systems that keep sensitive data from large corporations; Roose’s sharper prescription was, “outlaw data brokers.”

  • The closing upside case centered on faster science and medicine, including more breakthrough therapies, alongside AI as a tool for learning and building. Newton imagined infinite personalized quizzes for students and celebrated the pleasure of making projects through vibe coding—even when the result is “pure slop.”