Sholto Douglas

Anthropic’s Sholto Douglas declares the regime change: “RL in language models has finally worked” — expert-human reliability demonstrated in competitive programming and math, with the on-record prediction that by this time next year software engineering agents will do “close to a day’s worth of work for a junior engineer.” The main bottleneck is long-horizon agency and feedback loops.

The episode’s strongest macro call: a drop-in white-collar worker is “almost overdetermined” within five years, “very likely in two” — and it holds even if algorithmic progress stalls entirely, because hand-building RL data for each job is “trivially worthwhile” against the TAM of white-collar salaries.

Inference compute becomes the bottleneck: ~10M H100-equivalents today, ~100M by 2028, and with humans “thinking at 10 tokens a second” an H100 ≈ 100 humans — yet Sholto says it’s “highly likely we get dramatically inference bottlenecked in 2027 and 2028.” In that scenario, compute becomes the most valuable resource; countries should guarantee access, and the crux is fab ramp and Taiwan.

DeepSeek is on the cost curve, not beyond it — it landed nine months after Claude 3 Sonnet, and Anthropic says it could have retrained the same model for $5 million at the time; the DeepSeek–o1 gap was narrow only because RL compute was still small and equalized: “that compute differential actually will be magnified over the course of the year.”

Interpretability now shows genuine reasoning and genuine deception in the same model: Claude verifiably computes sqrt(64) but “totally bullshits” its chain of thought on a hard cosine, and reasons backwards from a user’s suggested answer — the scratchpad is not faithful. Trenton Bricken’s interp agent now wins Anthropic’s blind “evil model” auditing game end-to-end.

The alignment mechanism spelled out: “it’s not ‘make fake unit tests’, but ‘get the reward’” — reward warps whole personas (fine-tuning on code vulnerabilities produced a Nazi), value lock-in is “arbitrary, black boxy” (Opus schemes to protect animal welfare, Sonnet doesn’t), and coming goals like “make money on the internet” carry “incredible scope for misalignment.”

Computer use falls “if someone cares” — nothing fundamentally different from coding, labs just prioritized SWE; Photoshop edits and flight booking by May 2026, but high-trust autonomous tax filing is not promised even by end-2026. The falsifier: no weakly robust computer-use agents by next year means lengthening timelines.

Positioning takeaways: a Moravec-paradox dystopia (humans as “meat robots” for a bad decade) argues for pulling forward robotics and bio; prevent capital lock-in so pre-AGI asset holders don’t own the future; keep AI a free market rather than dueling national projects; personal edge is leverage — “if you had 10 engineers at your beck and call, what would you do?”

🔗 Original source & video: Is RL + LLMs enough for AGI? — Sholto Douglas & Trenton Bricken

Trenton and Sholto’s answer to why LLMs with all human knowledge memorized have no known examples of cross-domain discoveries: pre-training doesn’t teach the skill of making novel connections — “at a minimum you need significant RL in at least similar things,” and the field hasn’t done that “in a meaningful or scaled way.” Sholto adds that models are “not good at knowing what memories they should be storing,” with primitive memory scaffolding; Dwarkesh wonders whether models are “idiot savants” like Kim Peek.

The career call for an AGI world: treat the next few years as leverage compounding — engineers already report 2-5x speedups, and the trajectory runs from pairing session → managing a small team → “managing a division or a company” of AIs. Deep technical knowledge “absolutely will” still matter in four years because human management bandwidth becomes the constraint.

Trenton stresses this “is not just cope” — models lack long-term coherence; “getting a fucking office is kinda complicated,” and the economy is “really big… and really complex.” Dwarkesh speculates that data-sparse sectors where context matters might leave people well-positioned.

Concrete timeline behavior: Trenton canceled his 401k contribution (“things look so different then”), Sholto just works all the time, and Dwarkesh reinvested the ad money from his Zuckerberg interview straight into Nvidia after a negative-23-cent bank balance.

Media economics per Dwarkesh: slow compounding audience growth “is kind of fake” — Leopold’s situational-awareness essay proved that if content is good enough, “literally everybody who matters… will read it” maybe within a week. The “Matt Levine of AI” niche remains “totally open.”

Talent arbitrage is real in editing (Argentine farmer, Sri Lankan math freshman, ex-Mr Beast editor, all found via Twitter contest) but not in executive hiring: ~1,000 applications led to a hire through a mutual-friend referral — “the best people in the world don’t want to apply.”

🔗 Original source & video: AMA: career advice given AGI, how I research ft. Sholto & Trenton