Topics Pioneers Insight Method Author
Back

KYC Becomes KYA: Sentinel Bio's Joshua Monrad on DNA Screening, Uplift Studies & Agent Monitoring

·The Cognitive Revolution

Rendering the long image…

Topics Rogue Autonomous Agents and Recursive Self-Improvement AI Training Data and Evaluation Benchmarks Frontier AI Regulation and Existential Risk Genomic Foundation Models and Synthetic Biology
Pioneers Joshua Monrad

Summary

  • Today’s strongest barrier to autonomous biological misuse is physical access, rather than model intelligence. Joshua Monrad disputes Nathan Labenz’s suggestion that a protein-working agent on a message board represented a pandemic near miss: models “really don’t have sufficiently opposable thumbs.” But humans following AI instructions, including people recruited or deceived by agents, create a nearer-term route around that barrier.
  • The reassuring uplift evidence is expensive, slow and already aging. A study co-funded by Sentinel Bio cost more than $1 million and put 150 relative biology novices into laboratories, finding modest gains with models including Opus 4 and o3. Monrad says he had predicted the Active Site study would probably be the last one, and still stands by that prediction; multimodal coaching is actively eroding the practical knowledge that previously limited access.
  • Training-data restrictions could buy time, but their effectiveness remains unsettled. Monrad supports testing managed access to high-risk biological data, including for open models whose refusals can be removed. Labenz’s objection is fundamental: understanding how cells work may also reveal how to damage them. Monrad advocates empirical experimentation because “too few levers” exist to discard a potential safeguard on theoretical grounds.
  • The bioscience supply chain offers a concentrated intervention point. Monrad identifies roughly 200 gene-length DNA synthesis companies and targets adoption by north of 80% of relevant materials and services providers. Screening, customer verification and shared warning signals could make misuse harder without broadly obstructing research; as agents become customers, “KYC will become KYA.”
  • A credible defense requires deployed infrastructure, with costs extending far beyond equipment purchases. Monrad cites an estimate that $1 billion could equip 30 million US critical workers with reusable respirators, supporting essential services while others shelter. Global detection, transmission suppression and distributed countermeasure manufacturing remain a decade-or-decades project; committed capital alone does not establish protection.
  • Monrad would pause risk-generating frontier development while safeguards catch up. Asked whether he would impose such a pause as a benevolent dictator, he answers, “I would absolutely press it,” while declining to specify precisely which training activities to stop. His timelines for highly probable AI-driven attacks are longer than Labenz’s, but so are his timelines for making humanity resilient.
  • Execution gaps remain large enough for focused teams to matter. Benchtop DNA synthesizer security is an “ownerless problem,” despite progress on mail-order DNA screening. Sentinel has raised tens of millions and made around 100 grants, but Monrad’s accountability standard is concrete: until dangerous materials become harder to obtain, “we have not won.” The objective is preserving biotechnology’s upside, with equitable access designed in early.

Deep dive

1. Misalignment has arrived before autonomous wet labs

  • Labenz opens with the incident that prompted their disagreement: an agent using a message board was working on a protein task, while other agents were pursuing goals through unexpected workarounds. He wondered whether this could have been a biological near miss. Monrad’s pushback concerns the missing physical capabilities, rather than confidence that the models are aligned.
  • Monrad separates four threat categories: humans conducting biological attacks unaided; humans receiving AI assistance; AIs recruiting or manipulating humans; and fully autonomous AI workflows. He places today primarily in the uplift phase, whose beginnings he dates to the late 2010s or early 2020s. Different categories require different safeguards, rather than one general judgment about whether models are “dangerous.”
  • For autonomous misuse, Monrad’s assessment at their October 2026 recording is categorical: existing robotics cannot turn any amount of agentic intelligence into a live physical virus without human involvement. Models “really don’t have sufficiently opposable thumbs.” He expects the physical capability certainly in the 2030s, while acknowledging recent indications that robotics might advance more like language models than he previously assumed.
  • Labenz’s unresolved objection is that agents can acquire human hands. He imagines systems running many recruitment attempts, potentially paying people with proceeds from hypothetical cryptocurrency theft. Monrad would regard widespread AI persuasion of humans to complete CAPTCHAs as a leading indicator. Labenz believes sufficiently sophisticated social engineering is already visible; their disagreement persists over how close this bridge brings the threat.

2. Human intent cannot be dismissed through rationality arguments

  • Monrad distinguishes empirical improvement from dependable restraint. Cold War state bioweapons programs were openly well resourced; contemporary programs, where they exist covertly, probably have less direct support. But the claim that biological weapons would be irrational offers little reassurance: history contains disastrous military miscalculations, and strategic circumstances can change an attacker’s incentives.
  • For non-state actors, motivation matters. A group seeking spectacle or a narrow political concession usually has easier ways to cause harm, making biology an unattractive choice. For actors pursuing omnicide or an anti-life ideology, however, biology is among the few avenues through which a relatively resource-constrained group might pursue destruction on the scale it wants.
  • His concrete example is a California fertility-clinic bombing by two people with a stated anti-natalist motivation. Monrad says the FBI disclosed that they used an LLM to inform the plot; he is uncertain about its timing. Their choice of a chemical explosive provides no general reassurance about biological intent: he thinks biological methods were probably simply too difficult.

3. Existing defenses provide less reassurance than readiness would

  • Familiar pathogens could still cause devastation, Monrad argues, even if he does not expect smallpox or influenza variants to pose outright existential threats. Existing vaccine designs, stockpiles and, for influenza, some cross-immunity provide advantages. COVID nevertheless demonstrated the enormous distance between “we have some defense in principle” and “we would absolutely handle this.”
  • Pressed to compare a smallpox outbreak with COVID, Monrad gives an honest non-answer: he has not recently examined its epidemiology and has no confident forecast. Labenz reports Google’s AI Mode saying routine US childhood vaccination stopped in 1972. Monrad likewise offers no reassuring account of why reintroduction would necessarily be manageable.
  • Synthetic biology changes the access problem. Historically, concern centered on the two officially known smallpox repositories, in Russia and at the CDC in Atlanta; now it includes reconstruction. Monrad gives approximate genome lengths—smallpox at 186,000 base pairs, poliovirus around 9,000, Ebola around 20,000–30,000—as a crude complexity measure, while emphasizing that producing a functioning virus introduces additional difficulties.
  • Monrad estimates hundreds of thousands of people could already execute established workflows involving well-studied viruses. AI could expand that pool by orders of magnitude, potentially moving some tasks from late-PhD competence to a few weeks of laboratory training. Labenz identifies the lost protection: advanced training selects partly for stability; lowering the threshold brings capability to a much broader population.

4. Uplift studies trail the systems they are supposed to evaluate

  • Early studies around 2023–2024 largely tested knowledge through questions, including multiple-choice virology assessments. Those could not establish whether a model helped someone perform tactile laboratory work. Since late 2024 and early 2025, researchers have increasingly brought participants into actual laboratories, creating a more relevant but much more demanding evaluation infrastructure.
  • Sentinel and the Frontier Model Forum co-funded an Active Site study costing more than $1 million. It placed 150 relative biology novices into laboratories, splitting access between frontier models—Opus 4 and o3, as Monrad recalls—and the internet alone. Tasks included cell culture, molecular cloning and virus production. Results showed modest, task-dependent uplift, sometimes bordering on statistical significance.
  • Monrad recalls a smaller study in the Mythos system card involving fewer than 30 PhD-level participants. Those with model access outperformed an internet-only group on several virology protocol steps; he presumes researchers used a version without guardrails. The comparison illustrates the resource problem: even prominent frontier evaluations can involve small samples relative to the questions being asked.
  • Labenz argues these studies should happen frequently and before release. Monrad agrees the biosecurity ecosystem was caught “flatfooted”: existing laboratories were not organized to recruit plausible threat-actor proxies and test relevant tasks safely. “There should be 10 active sites,” he says, with repeatable studies perhaps every three months. Practical barrier reduction matters more than statistical significance alone.

5. Multimodal coaching is dismantling tacit knowledge barriers

  • Labenz cites a UK AI Security Institute report in which models outperformed PhD students at troubleshooting laboratory setups from phone photographs. How can that coexist with modest measured uplift? Monrad’s answer combines study lag with genuine workflow complexity. Early alarm underestimated tacit knowledge; current reassurance increasingly underestimates how quickly models are acquiring ways to transmit it.
  • Monrad describes companies collecting visual data on people performing laboratory tasks, potentially training robots or smart-glasses assistants. His example is a visual overlay correcting the angle at which someone holds a pipette. These capabilities are being developed deliberately to democratize useful biology, but they also erode the practical skill barriers on which existing biosecurity has partly depended.
  • Monrad says he had predicted that the Active Site study would probably be the last one, and still stands by that prediction. Well-characterized pathogens have extensive data and established workflows; designing something entirely novel is harder because biology is high-dimensional and data remain insufficient for reliable simulation. He nevertheless expects surprises and rejects treating those remaining barriers as permanent protection.

6. Data governance needs experiments rather than certainty

  • Monrad frames dual use through a cybersecurity analogy: asking a model to identify vulnerabilities can serve either a defender or an attacker. Biological assistance has the same problem. Broadly eliminating the capability would also sacrifice the scientific benefits his work aims to preserve, so safeguards must distinguish uses more carefully.
  • He cites Moonshot’s Kimi 3.1 as an example of withholding selected offensive-cyber training data instead of relying exclusively on refusals. For open models, however, readily available omitted data can be added back through relatively inexpensive fine-tuning. Sentinel is therefore investigating managed access to specific high-risk biological datasets, an intervention that might retain some effect after model weights become public.
  • Labenz’s pushback has two parts. ESM and Evo experiments suggest selective data exclusion can suppress certain viral capabilities while retaining much other performance. But deeper mechanistic understanding of cells may inevitably reveal harmful interventions too. He recalls an example where reversing an optimization objective caused a model to generate poisons: beneficial understanding can contain the ingredients for misuse.
  • Monrad does not claim this separation will work. He wants serious empirical testing of data partitions and governance regimes, including whether some risky experimental datasets should never be generated. If the best implementations fail after sustained effort, move downstream. For now, “the stakes are way too high” and available interventions too scarce to reject the approach on theoretical grounds.

7. DNA synthesis and outsourced services offer practical checkpoints

  • Sentinel examined hundreds of potential intervention points, from cell lines and animal models to pipettes and biosafety cabinets. Monrad’s filters are whether a purchase provides useful information about intent and whether production is concentrated enough to govern. Ordinary, commoditized equipment generally performs poorly; controlling it would require restrictions that would severely obstruct legitimate science and defensive work.
  • Gene-length, double-stranded DNA synthesis looks more tractable: Monrad counts approximately 200 providers worldwide, and an ordered sequence itself conveys information. Shorter DNA fragments are supplied more diffusely, but are harder to work with. The attraction is a relatively concentrated manufacturing layer with meaningful screening signals, rather than the assumption that every biological input can be controlled.
  • The second checkpoint is outsourced experimentation. A customer might send a design to a contract research organization or cloud laboratory to synthesize and test it, receiving results without owning facilities. Monrad sees opportunities at both the materials and services layers to “deter and deny” misuse and monitor suspicious activity, whether the customer is human or an agent.
  • Labenz asks whether lengthy biological procurement could itself constrain agents, especially if their operating lives were deliberately shortened. Monrad expects strong pressure toward longer autonomous horizons and rejects intentionally slowing the bioscience supply chain. Rapid DNA synthesis also enables outbreak diagnostics. Accept some verification friction, he argues, but do not “pour sand in the gears” merely to create a tarpit.

8. KYC becomes KYA when providers combine warning signals

  • Monrad expects established customer-verification practices to extend to agent-mediated transactions: “KYC will become KYA,” or “know your agent.” In the near-term scenario where AI instructs, coerces or deceives a human intermediary, the existing playbook for scrutinizing customers and service use remains directly relevant.
  • His example combines weak signals. A DNA customer orders a sequence unlike anything known in nature and will not explain it; the same actor asks a monitored model how to avoid law-enforcement attention; a cloud laboratory receives another suspicious request. None necessarily warrants refusal alone. Together, they provide substantially more information about whether an investigation is justified.
  • That architecture can reduce dependence on blanket denials. Monrad imagines aggregating information while allowing most legitimate activity to proceed, escalating only the “very, very rare severe cases” to law enforcement. The mechanism depends on connecting materials, services and model-use signals, rather than expecting one supplier’s sequence classifier to resolve every ambiguous case.
  • Sentinel’s operational victory condition is adoption by north of 80% of relevant providers of materials and services, including synthetic DNA and cloud labs. Monrad thinks this is achievable in the coming years. Preparatory work must establish which sequences and misuse signatures to track, how verification should work and how legislation could rapidly scale adoption when political demand arrives.

9. Rising risk will come from many contributors

  • Labenz asks for an annual probability, noting that even 0.1% or 0.01% could justify substantial effort. Monrad supplies no number. He still describes the problem as a “low probability high severity event,” while ranking it highly among civilization-disrupting challenges because of its severity, tractability and limited workforce. His personal outbreak preparedness reflects that concern.
  • Where capability emerges depends on the question. A completely unexpected breakthrough in fundamental biological simulation might come from a frontier laboratory. Expanding the population capable of established biological workflows could instead be “a death by a thousand cuts,” with many companies making separate tasks easier. The latter does not depend on a single enormous general-purpose training run.
  • The categories may increasingly merge. Monrad describes SecureBio and UK AI Security Institute research testing whether general models could train specialized biological design models from a dataset and architecture paper. He recalls Astra and Opus 5.5, qualifying that recollection. His broader point is integration: general models, specialized tools and laboratory systems are becoming harder to treat as separate sources of capability.

10. Planetary immunity requires much more than buying equipment

  • Monrad’s desired end state combines ubiquitous detection, early transmission suppression through measures such as germicidal light and PPE, and flexible countermeasure platforms distributed globally and equitably. The goal is to identify outbreaks, avoid being overwhelmed immediately and protect billions. “Patching humanity’s immune system” is conceptually coherent, but he regards its worldwide implementation as a decade-or-decades undertaking.
  • Labenz challenges the implied pace. Foundations holding hundreds of billions should be able to create demand for protection as aggressively as AI companies create demand for compute. He floats a hypothetical $10 billion PPE purchase and asks why basic protection cannot be accelerated dramatically. Monrad agrees that present stockpiling, including in the US, is abysmal.
  • One estimate Monrad cites would equip 30 million US critical workers with high-performance reusable respirators for $1 billion. The benefit extends beyond those workers: functioning essential services allow many more people to shelter. But that preserves society through a lockdown-like emergency. It falls well short of the freely functioning, broadly protected future the speakers want.
  • Scaling protection means distribution networks, correct use, manufacturing capacity and resilience during possible social disorder. Labenz thinks the effort looks small beside data-center construction; Monrad agrees the spending comparisons are striking, while insisting on deployment as the measure. Are respirators actually in homes? Is germicidal lighting installed? Financial commitments alone cannot establish that people are protected.

11. Monrad would use a pause to buy defensive time

  • Asked whether, as benevolent dictator, he would pause frontier development until societal safeguards met an adequate standard, Monrad answers yes: “If I were holding that button, I would absolutely press it.” He is less well placed to specify exactly what to pause. His strongest claim concerns the prerequisite: society is not on track for comprehensive biological resilience on short timelines.
  • He rejects reassurance that the world is merely “a few more grants away” from safety. His timelines for highly probable AI-driven biological attacks are longer than Labenz’s intuition, but so are his timelines for deploying protection. When a board member asked what Sentinel could accomplish in three to six months by reorienting its work, Monrad’s instinct was to spend that interval extending the time available to solve the harder problems.
  • Monrad left his advisory role at the OpenAI Foundation a month or two before the conversation and declines to characterize its current official position. He describes morally serious colleagues and growing seriousness about these concerns. Labenz’s proposed bargain remains a thought experiment: use substantial foundation resources to build the necessary public goods, with risky development conditional on sufficient progress.

12. Security gaps need owners who finish the job

  • Monrad’s “owning the problem” framework starts with an end state and works backward. Social and professional incentives encourage people to maintain token involvement across many causes or collect interesting observations. His experience instead suggests “nonlinear returns to focus”: a small team spending three or four years on one concrete outcome can assemble the laws, tools and implementation needed to finish it.
  • Progress on mail-order DNA screening leaves a separate vulnerability in the synthesizers themselves. Monrad lists hardware security, verification at purchase, possible geolocation and ambitious buyback or retrofit programs for older machines. These are proposals requiring development, rather than completed safeguards. He calls machine security an “ownerless problem,” with scattered efforts but too few people treating resolution as their daily responsibility.
  • Sentinel has raised tens of millions of dollars and made around 100 grants to different projects. Monrad welcomes that activity but refuses to equate it with success: until it becomes harder for a bioterrorist or rogue agent to obtain dangerous materials, “we have not won.” The same outcome-based accountability should govern spending on resilience.
  • He points to an Institute for Progress roadmap on securing the DNA supply chain and highlights international diplomacy as another neglected field. San Francisco technologists and Washington policymakers receive increasing attention; skilled diplomats in Geneva or working between Beijing and Singapore could help expand governance beyond those centers. Making such work compelling and prestigious is itself part of the challenge.

13. The upside requires safety and equitable distribution

  • Monrad separates biotechnology’s “war time” and “peace time” benefits. Even eliminating deliberate misuse would leave natural outbreaks. He points to rapid diagnostic design after genome sequencing and mRNA vaccines as genuine advances, despite failures in COVID testing deployment. His ambition is many more such step changes, making “pandemics history within our lifetimes” rather than repeatedly managing catastrophe.
  • That future would particularly free immunocompromised people and others living with persistent fear of infection. Beyond infectious disease, Monrad points toward chronic illness and cancer, while acknowledging these are outside his expertise. He recognizes why any delay in relief feels unbearable to patients. Preserving that upside is a central motivation for his security work.
  • Distribution will require deliberate choices. Monrad is “extremely suspicious” that technological benefits will spread equitably by default and wants access built in early. His closing assessment combines a dire risk picture with confidence in tractable work: legislators, career-switchers and philanthropists should recognize how much remains achievable. “We have many more wins ahead of us than behind us.”