The most important question nobody's asking about AI.

Summary

• Solo essay narration by Dwarkesh: the Department of War declared Anthropic a supply chain risk after it refused to drop red lines on mass surveillance and autonomous weapons. He predicts that within 20 years, 99% of the workforce in the military, civilian government, and private sector will be AIs. His surprising concession — “If I was the Secretary of War, I probably would have made the same determination” (you can’t give a contractor “the kill switch on a technology that you have come to rely on”) — but the government crossed the line by threatening “to destroy Anthropic as a private business,” which if upheld forces Amazon, Nvidia, Google and Palantir to wall Anthropic off from all Pentagon work.
• The tradeable inversion: once AI is woven into every product, big tech forced to choose between its AI provider and the DoW — “a tiny fraction of their revenue” — would they rather drop the government than the AI? Prediction markets give a 74% chance the supply chain restriction gets backtracked, but the harassment toolkit (permitting, antitrust, contract conditions) remains.
• Surveillance economics are the core numbers: mass surveillance in certain forms is already legal under current law; processing all 100M US CCTV cameras at 10¢/M input tokens, one 1,000-token frame per 10 seconds, costs ~$30B today — with 10x/yr cost declines, “by 2030 it’ll be less expensive to monitor every single nook and cranny in this country than it is to remodel the White House.”
• Diffusion doesn’t save you: even if 2027-era “Claude 6s and Gemini 5s” labs refuse, by 2028 open-source matches the frontier of 12 months prior and some vendor will serve the state. AI “gives more leverage to whatever assets and authority you already have” — and the government starts with the monopoly on violence plus “extremely obedient employees that will never question their orders.”
• The episode’s title question: to what or to whom should AIs be aligned — model company, end user, the law, or the model’s own morality? The 1989 Berlin Wall guards’ refusal to fire and Petrov’s refusal, after judging five ICBMs a false alarm, illustrate refusal to obey; if Petrov hadn’t, Soviet High Command would probably have retaliated. But “one person’s virtue is another person’s misalignment.”
• Against an NRC/SEC-for-AI: vague AI-risk terms like “catastrophic risk” and “autonomy risk” are “a fully loaded bazooka” for a wannabe despot — Anthropic has been “especially naive” urging such regulation. He concedes some regulation may be inevitable and coordination could lessen AI risk; regulate specific destructive end uses and government use instead. The nuclear analogy fails because AI is industrialization itself, and multipolar.

Deep dive

1. The warning shot: refusing Anthropic is fine, destroying it is not

• The setup: DoW declared Anthropic a supply chain risk after it refused to remove red lines on mass surveillance and autonomous weapons. Within 20 years, Dwarkesh predicts that 99% of the workforce in the military, civilian government, and private sector will be AIs. His concession up front — he’d have made the same call, via his Starlink hypothetical: if Elon reserved the right to cut military access during an “unjust war,” no military could accept a contractor holding “the kill switch on a technology that you have come to rely on.”
• The line crossed: the government “has threatened to destroy Anthropic as a private business” for refusing its terms. Anthropic survives today because Amazon, Nvidia, Google and Palantir can cordon off Pentagon work — but not once “AI will be woven into how every product is built and maintained and operated.” Is an AWS service built with Claude Code a supply chain risk?
• The boomerang: forced to choose between AI provider and the DoW’s “tiny fraction of their revenue,” big tech might rather drop the government than the AI — and the China frame inverts: are we racing the CCP “just so we can adopt the most cruelest parts of their system?”

2. Some mass surveillance is already legal — AI deletes the enforcement bottleneck

• The legal base: under current law, mass surveillance in certain forms is already legal; there’s no Fourth Amendment protection for data shared with banks, ISPs, carriers, and email providers — the government reserves the right to buy and read it in bulk without a warrant. The missing piece is manpower, and “that bottleneck goes away with AI.”
• The math: 100M CCTV cameras, open multimodal models at 10¢/M input tokens, one 1,000-token frame per 10 seconds → $30B to process every camera in America; at 10x/yr cost decline, $3B next year, $300M after, cheaper than remodeling the White House by 2030.
• Once the capacity exists, the only barrier is the political expectation “this is just not something we do here” — which is why Anthropic’s stand is “so valuable and commendable.”

3. Neither concentration nor diffusion solves it — the tech structurally favors control

• The leverage catalog: power-generation permitting for data centers, antitrust, and soft or explicit conditions on other companies’ federal contracts to drop Anthropic. Prediction markets: 74% the supply chain restriction is backtracked — but the president’s harassment options persist.
• The diffusion counterargument he rejects: even if “Claude 6s and Gemini 5s” labs draw lines in 2027, by 2028 open-source matches the frontier of 12 months prior and the government can use a model that “might not be the smartest thing in the world, but is definitely smart enough to not take a camera feed.”
• No symmetric hope: AI won’t check state power the way it enhances it — the government starts with the monopoly on violence, now “supercharged with extremely obedient employees that will never question their orders.” That obedient army “is what it would look like if alignment succeeded.”

4. Aligned to whom? — the question nobody’s asking

• The core: “to what or to whom should the AIs be aligned” — model company, end user, law, or its own morality? “Maybe the most important question about what happens… and we barely talk about it.” This spat is “a sneak peek” at “the highest stakes negotiations in human history”; mass surveillance is “like the 10th scariest thing.”
• Against “all lawful purposes”: Snowden 2013 showed the NSA — part of the DoW — used the 2001 Patriot Act plus a secret court order to collect every phone record in America. “No government is going to call what they are doing mass surveillance.”
• The case for models with their own morality: 1989 Berlin Wall guards refusing to fire; Stanislav Petrov judged the five ICBMs his sensors said the US had launched at the Soviet Union to be a false alarm and broke protocol. If he hadn’t, Soviet High Command would probably have retaliated. But “one person’s virtue is another person’s misalignment” — he likes Dario’s idea of companies publishing competing constitutions that outside critique iterates on.

5. AI-specific regulation hands a bazooka to a future despot

• The AI safety community has been “quite naive,” Anthropic “especially naive” — ironic given its opposition to the state-law moratorium — in urging an apparatus its own roadmap describes as “closer to nuclear energy or financial regulation.” He acknowledges the rationale: safety investments impose costs and may slow labs unless the whole industry follows suit. Terms like catastrophic risk and autonomy risk “can mean whatever the government wants”: a model refusing government orders becomes an autonomy risk in this framing.
• Proof of abuse already: the Pentagon wields a 2018 defense-bill authority meant to keep Huawei components out of American military hardware and the 1950s Defense Production Act meant to keep Korean War steel mills and ammunition factories running. “Do we really want to hand the same government a purpose-built regulatory apparatus for AI?”

6. The nuclear analogy fails — AI is industrialization, and multipolar

• The opposition, quoted fairly: Ben Thompson — “If nuclear weapons were developed by a private company, the US would absolutely be incentivized to destroy that company” — and Leopold Aschenbrenner’s Situational Awareness: “Imagine if we had developed atomic bombs by letting Uber just improvise.” Dwarkesh’s reply: they’re right it’s crazy, but “Nobody’s qualified to be the stewards of superintelligence.”
• Two breaks in the analogy: AI is a general-purpose transformation like the Industrial Revolution — free societies didn’t nationalize industrialization, they banned specific weaponizable end uses, and should regulate AI likewise (cyber attacks, plus laws on government use); and there are half a dozen competitors, so the government’s requisition argument is extremely weak. If only one entity remained capable of building the robot armies and superhuman hackers, and there were reason to worry its insurmountable lead could let it take over the whole world, he agrees private ownership would be unacceptable. His stated crux: “I expect this technology to be very multipolar.”
• He acknowledges the strongest counterargument: some regulation may be inevitable and coordination could lessen AI risk, but wholesale government control would be dangerous. The honest close: corporate courage can’t fix a structural problem — in 12 months everybody and their mother will be able to trade a model as good as the current frontier — only laws and norms, like the post-WWII norm against using nuclear weapons to wage war, can preserve a free society.