Helen Toner

The clearest near-term AI-biology business is not inventing more molecules but rescuing value at the clinical bottleneck: Abhi Mahajan says 97% of oncology trials fail even though some patients often respond. Noetik combines pathology, 16-plex spatial proteomics, a 19,000-gene spatial transcriptome, and exome sequencing to find potentially “non-human-legible” response biomarkers. Mahajan expects human-simulation companies to improve at least a few trials within several years, while remaining much less certain that AI will rapidly discover wholly new targets.

Biology is unlikely to repeat software’s intelligence explosion because its most valuable rewards are slow, ambiguous, and expensive rather than cheaply verifiable. Toxicity can emerge in seconds or years, vary by dose and species, or cause cognitive and cardiac damage without killing an animal; a clean hepatocyte assay may save months while still missing the decisive in-vivo question. Even a model doubling “Alpha 3” on difficult preclinical benchmarks does not prove better patient outcomes: “The field is already awash with many really good preclinical assets.”

China’s biotechnology advantage looks more like an operating-system advantage than a demonstrated AI-model lead. Mahajan traces it from generics through a strong CRO ecosystem into indigenous drug development, with lower trial costs and tighter feedback between designers and wet-lab workers; he has not yet seen a “DeepSeek thing” in Chinese bio-AI. Jeremie Harris separately argues that chip export controls are visibly binding, citing DeepSeek’s pre-R1 complaints and pent-up H200 demand, while rejecting the idea that Nvidia sales would make China abandon its strategic domestic stack.

Automated AI R&D is a strategic-surprise machine because informed experts agree about the near term while disagreeing on whether it ends in recursion, jagged acceleration, or a plateau. Helen Toner’s workshop participants agreed substantially about what they might see in 2026–2027 but not what comes next; the decisive questions are whether AI replaces every human contribution and how quickly physical, organizational, and adoption bottlenecks “bite.” An underexplored case is a superhuman but bounded plateau: transformative enough to reorder the economy without becoming an incomprehensible singularity.

The frontier-lab race is outrunning both evaluation and governance, yet inevitability does not erase meaningful design choices. The discussion pointed to Anthropic and OpenAI releasing models despite acknowledged difficulty evaluating awareness or long-horizon autonomy, while distinguishing AI-led research under human direction from setting millions of agents in motion with “no clue what’s going on.” Toner’s policy prescription shifts from release-day paperwork toward continuous internal-risk measurement, independent audits, and societal hardening across cyber, bio, and epistemic security.

The overlooked AI trade is the physical substrate—and Harris thinks compromising it could nullify every model-level advantage. TSMC is an exceptionally fragile concentration point; China-linked components and personnel create “one-way doors” in data-center construction; and grid transformers may offer an adversary leverage far beyond model theft. Harris treats secure builds, supply-chain scrutiny, grid redundancy, and credible offensive options as purchases of optionality, while assigning loss of control only a deliberately broad “10 to 90%” range and treating 2027, 2030, but less so 2035 as plausible timelines.

Adoption may look discontinuous even when capability curves look smooth because a final 1% improvement can turn a toy into a production system. Harris says podcast clipping failed six months earlier but worked end to end three or four weeks before the show; Nathan Labenz sees a similar threshold in agents, though current models still overuse code when they should “just read the document.” The emerging stack—deep personal context, 300,000-to-10,000-token monthly compression, possible continual learning, and models aimed at “judgment transfer”—was discussed as a way to preserve more individual economic leverage.

🔗 Original source & video: Approaching the AI Event Horizon? Part 2, w/ Abhi Mahajan, Helen Toner, Jeremie Harris, @8teAPi

Toner’s base case is not imminent superintelligence; it is that a civilization-scale transition is plausible enough to prepare for now. In 2016, “short timelines” meant advanced AI within a couple of decades or a lifetime; by 2025, it can mean superintelligence before the 2020s end, so even “long timelines to advanced AI have gotten crazy short.” For investors, the discussion points toward sustained demand for evaluation, resilience, and government capacity—not confidence in a five-year countdown.

The clearest new OpenAI disclosure is that Q did not trigger the board’s 2023 decision.* Toner said the board knew reasoning research later released as o1 and o3 was underway, but received no breakthrough letter and did not act on one: “That whole Reuters story was totally false.” The broader governance discount remains harder to quantify because confidentiality and legal obligations prevent a complete public record.

Frontier-AI oversight works better when whistleblowers can point to a violated rule than when protection depends on subjective concern. Toner favors disclosed safety-and-security plans, capability and risk evaluations, and internal processes that create a crisp standard employees can invoke. Those employees may have their greatest leverage now because they are “actively working to replace themselves,” while institutional failure may arrive “boiling frog style” without one obvious crisis.

Adaptation buffers are the strategic answer to an AI market where frontier development gets costlier while yesterday’s frontier rapidly commoditizes. DeepSeek matched reasoning capabilities only one or two months behind some US systems—or base-model capabilities roughly six to nine months behind—at lower cost, making permanent nonproliferation increasingly invasive and brittle. The practical response shifts toward vaccine capacity, outbreak detection, cyber remediation, and distribution of defenses before capabilities diffuse.

Iterative deployment remains useful only while releases are unlikely to cause severe, irreversible harm. Toner prefers conditional “if-then” gates—do not advance until specified understanding or mitigation exists—over fixed pauses or input-based speed limits that would require unavailable legislation and immediately encounter the China objection. With no comprehensive regime in sight, transparency, measurement science, interpretability, alignment work, and technical government staffing are the practical building blocks.

“Beat China” is functioning simultaneously as a geopolitical argument and the AI industry’s path of least resistance in Washington. Toner grounds the rivalry in power transitions, maritime access, and international rules; the conversation also covers Taiwan. She says the framing conveniently supports “funding,” “government contracts,” “no regulation,” and protection from liability. That makes China rhetoric material to AI company economics even when it does not resolve the underlying strategic question.

Military AI should be evaluated use case by use case, not sold as a general-purpose battle buddy. Bounded tools for viewshed mapping, medical triage, ship-movement anomalies, or database retrieval differ fundamentally from an LLM asked to generate “three courses of action that are non-escalatory.” Toner and Amelia Probasco’s framework—scope, training data, and human-machine interaction—puts reliability and adversarial robustness ahead of demo fluency.

An “AlphaGo for the army” is not a credible near-term equilibrium because real war cannot be enclosed in a clean simulation. Battlefield, logistics, economic, political, and public-attitude dynamics interact while an adversary deliberately attacks the model’s assumptions. Toner’s honest conclusion is that she does not know what nation-states, democracy, or the Chinese Communist Party look like under superintelligence; strategic uncertainty, not a settled doctrine, is the central fact.

🔗 Original source & video: Helen Toner: OpenAI Reflections, Adaptation Buffers, and AI in Warfare