Confronting the Intelligence Curse, w/ Luke Drago of Workshop Labs, from the FLI Podcast
Summary
Drago’s “intelligence curse” is a bargaining-power thesis: once non-human intelligence becomes the dominant factor of production, capital owners will rationally invest in AI rather than people. The risk does not require evil executives or misaligned machines; saving 50% of a wage bill while improving speed and reliability is enough. Political leverage then erodes with economic leverage because “your ability to produce value is a core part of your bargaining chip in society.”
The earliest warning signs should appear at the bottom of white-collar pyramids, where AI can remove the entry-level pipeline before replacing senior decision-makers. Drago would watch employment among 22-to-25-year-olds in automatable fields, job postings, income inequality, social mobility and any moment when capital starts compounding without talent. His endpoint: “One day you wake up to find that all of your colleagues are AI and the next knock at the door is booting you out too.”
Proprietary human know-how may become the decisive data moat—and surrendering it could finance one’s own replacement. Workshop Labs is betting that tacit skill and real-time local information remain bottlenecks, especially across small and medium-sized businesses that employ 50% of Americans. Drago calls data “the new social security number”: within 10 years, handing a lab your working life for moderately better results could leave it “one button push away” from selling the knowledge needed to automate you.
The bearish macro scenario is a shrinking labor-tax base paired with record corporate output, strained transfers and a B2B economy that no longer needs mass consumer purchasing power. Drago imagines a 2030 graduate unable to enter software, companies halving employee expenditure while doubling output, and a government losing revenue when income tax supplies roughly 50% of federal tax receipts. By 2040, unstable UBI and political unrest coexist with “way fewer Starbucks” and “a whole lot more data centers.”
Commodifying intelligence is necessary to prevent model owners from extracting monopoly rents, but open access requires safety techniques that survive release. Drago rejects the assumption that open-weight models must remain far behind, citing Chinese models as roughly six months from the frontier and Kimikatu as potentially state-of-the-art for English writing. His preferred investment frontier is tamper-resistant models and layered defenses; “aligned superintelligence in the hands of one person makes that person a de facto dictator unless they choose not to be.”
The durable consumer-AI opportunity is an agent whose architecture and revenue model make it loyal to the user rather than advertisers, employers or governments. It should “either answer in your interest or tell you when it’s not,” protect intimate data through something like AI privilege, and label commercial influence explicitly. Drago accepts that covertly ad-supported AI is the default trajectory, but sees Apple’s device-funded privacy model as evidence that trusted infrastructure can become a huge market.
For workers and founders, Drago’s call is to leave standardized prestige ladders before those ladders disappear. An “n equals one” role where nobody else performs the same important job is safer than being one of 1,000 interchangeable employees, while startups and locally specialized businesses can exploit AI without donating their advantage upstream. Jane Street or McKinsey may still offer large checks, but he calls that “the last chopper out of Saigon”: “You are the last breed of consultants. That industry is dying.”
Deep dive
1. Human labor’s loss of leverage—not misalignment—creates the curse
The Cognitive Revolution’s framing raises the episode’s central complication: even if humanity solves AI alignment and avoids losing control, the transition can still go badly if machines replace humans as economic actors. Drago’s failure mode echoes the resource curse, where rulers invest in oil rather than citizens because oil offers the higher return.
Drago’s mechanism is impersonal rather than conspiratorial. If capital can purchase systems that perform human work “better, faster, and cheaper,” a company offered better results plus a 50% reduction in its wage bill will usually automate; technology designed to replace rather than augment consequently moves society toward a world “where people just don’t matter.”
Docker’s pensioner-and-UBI pushback—worth keeping—is that societies already protect people who produce little current output. Drago answers that pensioners generally built leverage through roughly 40 working years before receiving a 10-, 20- or 30-year exemption; a society in which “we all are pensioners forever” leaves everyone dependent on elections without an independent bargaining chip.
Drago links democracy to dispersed economic power: property-owning lords helped force mechanisms such as the Magna Carta, courts and Parliament because the king could not ignore them. Economic liberalization is often a precondition for the democracies we care about, and rights-respecting, prosperous governments strongly correlate with democracy; removing the underlying leverage makes continued benevolence a precarious assumption.
2. Automation starts at the pyramid’s bottom and shows up in mobility
The confirming indicators would be widening income inequality, declining mobility and a sudden acceleration in capital accumulation once dollars can generate returns without scarce talent. Drago especially worries if the American promise that someone can “start from nothing and win”—never a guarantee, but at least a pathway—begins closing structurally.
His “pyramid replacement” begins inside multinational white-collar firms that recruit large annual analyst classes to replenish a narrow leadership tier. AI first absorbs entry-level tasks, then moves upward as models gain agency, longer-horizon planning and access to institutional knowledge, eventually replacing each layer from the bottom rather than attacking management first.
Drago cites a paper released, he believes, the previous day showing reduced postings, offers and employment among 22-to-25-year-olds in some AI-exposed fields, including software engineering. He repeatedly limits the claim: he had only skimmed the findings and would test causality by asking whether declines cluster in tasks current and projected systems can actually automate.
Blue-collar displacement may be more “zero to one”: many similarly situated workers remain necessary until capable robots exist, then become automatable together. Docker adds that invoicing, scheduling and other managerial functions could disappear before physical labor itself—a different ordering from the white-collar pyramid and potentially a precursor to a sharper robotics shock.
3. Protected titles can conceal automated judgment
Legal restrictions may preserve judges and senior lawyers formally, but Drago warns of shadow automation. If every judge consults GPT-7 with effectively the same prompt and accepts its output, the office remains human while its judgment becomes centralized around “whatever flaw exists in GPT-7.”
Law illustrates the split: partners may retain protected status, while paralegals and first-year lawyers doing grunt work are easier to remove. The consequential fork is whether incumbent firms compound profits without replenishing their workforce or cheap intelligence produces “an abundance of new law firms” and more diverse economic output; Drago prefers the latter but says it is not the default.
Taste could remain a durable advantage. The artist Nomads and Vagabonds fine-tunes Stable Diffusion on his own work, generates hundreds of outputs and releases a highly selective few; Drago sees unmistakably personal authorship in that loop. Yet this differs from OpenAI’s stated target of AGI doing most economically valuable work—“to do all of it versus to do some of it.”
4. Tacit data is both the moat and the extraction target
Workshop Labs’ two-part thesis is that long-run AI progress bottlenecks on high-quality data about tacit skill and local information. Tacit knowledge accumulates through practice and is difficult to locate because people simply possess it; local knowledge comes from being embodied, noticing changing conditions and seeing opportunities within one’s immediate environment.
Labs’ rush into browsers and bespoke reinforcement-learning environments signals demand for that material. Workshop’s proposition is to tune an existing model to an individual’s private work, restrict it to that user and ensure the resulting advantage cannot be converted into a larger system that replaces them: “You should reap the benefits of the data that already exists in your world.”
Docker raises a labor conflict in which management wants workers’ process knowledge to reduce costs, while Drago says that important competitive information should not simply be given away. Drago broadens the opportunity to small and medium-sized businesses, which employ 50% of Americans and depend on employees whose absence actually breaks something; personal models could offset incumbents’ scale advantages and trigger an “explosion of small companies.”
Individual incentives remain difficult: a poorly paid maths PhD student may accept hundreds of dollars per step-by-step proof. Drago says privacy cannot win with “worse tools” and no payment; the loyal product must outperform an off-the-shelf model. Workshop planned September and October disclosures around encrypted transit, NVIDIA secure enclaves, attested code and encrypted model weights.
5. The bad macro loop can bypass consumers entirely
Drago’s scenario begins with a 2030 computer-science graduate who cannot obtain an internship or entry-level job because in 2026 it was not obvious what would happen. Unemployment support strains as the cohort expands, while a company such as Microsoft—used illustratively, not singled out—posts record earnings after halving employee expenditure and doubling output.
The fiscal mismatch matters because Drago says income tax supplies roughly 50% of US federal tax revenue, while corporate taxes are a small share and large companies can minimize them. A shrinking wage base therefore meets an increasingly strained safety net until austerity, lower payments and social unrest reinforce one another.
By 2040, many people remain unemployed and a politically contested UBI has proved insufficient and unstable. Concentrated companies then face weakened institutions and incentives to remove governmental constraints; Drago invokes Tom Davidson’s coup scenario as one route through which economic precarity becomes lost political rights, democratically or otherwise.
Docker’s demand-side objection is that giant companies still need customers. Drago’s answer is an increasingly closed B2B loop among labs, AIs, governments and providers of land, compute, energy and intelligence. Commodifying the intelligence layer limits rents—“If you are a rentier around a commodity, you’re a landlord; and if you are a rentier around a monopoly, you are a renter”—but does not alone restore human leverage.
6. Institutions—not resource windfalls—separate Norway from rentier states
Norway shows that the resource curse is breakable: it discovered oil after developing a capable civil service, low corruption, stable democracy and an economy able to absorb sovereign-wealth investment. Drago’s uncomfortable question is whether contemporary America possesses institutions comparably resilient; his answer is no, and total labor automation would impose stronger pressures than oil ever did.
Oil is only an analogy because it never replaces every human contribution. Saudi Arabia and Dubai can reinvest petrodollars into more dynamic economies as renewable energy threatens oil’s primacy, but the benefits historically accrue to people economically important to the state; Saudi Arabia and the UAE also rely on underclasses and have not afforded equal freedoms to everyone.
Drago sees Saudi gender liberalization under MBS occurring alongside diversification—not as complete emancipation, but as evidence that economic usefulness and political treatment move together. Oman supplies another mechanism: a credible threat of revolution can force rentiers to distribute enough wealth to remain in power rather than risk losing all rents.
AI may neutralize even that bargaining chip by automating surveillance and repression. States can look weak while institutions destabilize, then become “suddenly very strong” once dissent is legible and machine-enforced; at either stage, democratic processes can fail—first through incapacity, later through an overwhelming coercive advantage.
7. Defensive technology must make diffuse intelligence survivable
Incentives are powerful but not destiny. Drago pairs Washington’s Cincinnatus-like decision to relinquish power with Brexit, where sovereignty arguments defeated a strong economic case, to show individuals and cultures can reject material incentives. Still, relying repeatedly on exceptional character is fragile: “Show me the incentives and I’ll show you the outcome.”
His differential-development agenda has three buckets: technology that makes powerful AI safe enough to remain diffuse, tools that keep humans in control of their own data and productivity, and systems that strengthen democratic institutions. Audrey Tang’s vision informs the third category; Workshop aims at the second.
Catastrophic risk itself supplies governments and companies with credible arguments for monopoly: advanced systems might enable bioweapons or other harms, so authorities centralize access. Drago therefore treats safety as an anti-monopoly prerequisite—if society builds powerful AI, it needs technical defenses strong enough that “one guy” need not control intelligence indefinitely.
His social-media analogy combines regulation with alternatives. Age gates and feature restrictions address addictive products from above, while apps such as Opal help users reclaim focus from below: when “a whole lot of algorithms are pointed at you,” people need “something pointed outwards.” Workshop wants to occupy that defensive layer for employment—a potentially enormous market if users pay to remain economically involved.
8. Open weights turn safety from gatekeeping into engineering
Drago is more pro-open-source than he expects the podcast’s average guest to be because closed weights permit “wild rents.” He accepts that open models lose under a hard “foom” to superintelligence, but says nearly every slower scenario contradicts claims that they cannot catch up: Chinese open-weight models are around six months behind the frontier, with some possibly ahead on particular tasks.
As one example, he wagers that Kimikatu may be state-of-the-art for English writing. The broader point is that data and training methods still offset unequal compute access, so powerful open weights are likely a reality; serious safety work must address them rather than assume they remain permanently second-rate.
Docker’s objection is irreversibility: an open release cannot be recalled after testing exposes a dangerous capability. Drago points to Kyle O’Brien’s work with A.C., where removing biological-material information during pre-training reportedly produced models somewhat resistant to later reintroduction through fine-tuning; the “holy grail” is a model that breaks or stops working when someone restores prohibited capabilities.
Evaluation alone is insufficient because a warning shot might accelerate development when frightened actors demand defenses faster. Drago prefers a pandemic-style “Swiss cheese” of safeguards and rejects a controlled explosion where 12 people become one monitored winner: “Aligned superintelligence in the hands of one person makes that person a de facto dictator unless they choose not to be.” Between one holder and many, he chooses many.
9. A loyal agent needs architecture and aligned revenue
Docker notes that OpenAI’s founding anti-monopoly vision appears to have degraded and asks how Workshop avoids the same path. Drago’s safeguards include public-benefit-corporation status, a fiduciary mission to enhance economic opportunity rather than automate people, and mission-focused hiring because “personnel is policy”—while conceding that “the road to hell is paved with good intentions.”
The stronger commitment is technical: users should not need to trust Drago as a benevolent steward. Workshop wants verifiable controls making it impossible to train a larger model on their data, sell it to an employer or use it against them; every contribution should improve a model “loyal to you alone.”
Personal agents sharpen the conflict. A hotel-booking assistant may serve its user or quietly prefer a property tied to a corporate agreement; Drago’s rule is that it should “either answer in your interest or tell you when it’s not.” His Black Mirror analogy—a cloud-dependent brain interrupted by ads and escalating subscription tiers—shows how indispensable augmentation becomes rent extraction under monopoly control.
Drago endorses “AI privilege”: an agent organizing someone’s life should not become an interrogation record without even a Fifth Amendment-like protection. Docker argues consumers prefer free, covertly ad-supported products; Drago agrees that is the default future, but cites Apple’s “privacy second” model—privacy embedded in paid infrastructure rather than sold alone—as proof that aligned revenue can support trusted personal technology.
10. The safest career is becoming hard to average
Drago says the default paths are closing regardless of whether Workshop succeeds. Large prestigious employers are obvious automation targets because reducing a payroll of 500,000 delivers immediate returns; startups, think tanks and unknown small companies increasingly offer safer terrain because their workers must interpret local conditions and perform consequential, nonstandard work.
An “n equals one” employee with a role nobody else performs is safer than one of 1,000 people doing the same job. Automation can culminate in total pyramid replacement, or it can support a localized, specialized economy where outsiders gain unprecedented leverage—but capturing the second outcome requires changing paths before incumbents absorb the tools and data.
His advice to high-achieving young people is categorical: take the moonshot while the window remains open and the move is easier than ever. The large Jane Street or McKinsey paycheck may represent “the last chopper out of Saigon,” not safety; anyone taking it could become “the last breed of consultants” or entry-level professionals in an industry already dying.