Can Open Models Keep Up With OpenAI and Anthropic? | Nathan Lambert

Summary

• Nathan Lambert’s organizing framework is that model progress reduces to three forms of capital — compute, data, and talent — and China is compute-starved with a comparable talent pool. He allows there might be a slight talent edge depending on who was where when AI got going. His compute anecdote: tell a top Chinese pre-training researcher that “an average OpenAI researcher has 5,000 H100 equivalents that they can access at any time” and “their jaw just falls to the floor.” Distillation can help compensate because “outputs from a language model are a function of compute” — it can be a way to get something compute-equivalent.
• Lambert’s literature-based estimate is that distillation helps Chinese labs by a few months, but its role in today’s RL-heavy post-training is hard to establish. His evidence: open post-training recipes are “stuck using a really outdated Qwen model” because it is hard to get the teacher right, and he doubts Claude Opus reasoning traces would have made OLMo much better. He says 20–30 Chinese companies are building good models, so the idea that “only we are good at building models” is “a bit of cope.”
• He opposes government action on distillation, while supporting labs securing their own products if they choose. He questions government involvement in what he calls a terms-of-service violation; labs can cancel access, and restricting models to products or enterprise agreements would make distillation harder. Jensen Huang’s quoted line anchors the segment: “People distill my products every single day… That’s called competition.” Lambert argues the Chinese labs “are not gonna stop,” and that action against distillation could do more harm than good.
• The open-model economy runs on a slower exponential than closed labs but can end in real ownership — “model weights, no terms and conditions,” and the ability to run them on your own hardware. Closed labs have integrated inference and reach high-value markets like coding first; the revenue there has gone from zero to $100 billion and is likely to grow. Open models involve builders, inference companies, and post-training companies, so customer information and revenue do not flow back to model builders easily. Lambert is “less concerned about this being kind of bubbly”: open models could gain ground as buyers figure out the cheapest model that gets a task done.
• A lot of frontier progress in less-verifiable domains comes from high-quality data and environments, not simply “poof generalization.” Lambert says it is probably some of both, but a lot comes down to data. He balks at the scale of research budgets for environments, while arguing labs buy them because models improve from them. The gold rush also brings corner-cutting and reward hacking downstream of the data supply chain. Investable wrinkle: a Chinese company may buy an RL environment and use it six months later.
• Lambert is skeptical of fast takeoff on current evidence: “all the scaling laws are an exponential increase in compute for a linear increase in performance.” He cites Claude system cards as saying, roughly, that it would take “10× more progress to make this a fast takeoff.” His stated worry is the capability curve alongside “pretty much nonexistent preparation” for public cybersecurity — “not fearing fast takeoff does not mean that you don’t worry about risks.”
• His new nonprofit Trillium will build open-source environments and share its research and model-use process — tasks, configs, trajectories, and failures — to demystify RSI. The name is the thesis: a trillium is “a spring ephemeral flower” that blooms before the canopy and helps spread nutrients. Lambert wants open infrastructure in an industry where “all CS research… the benefit accrues to the tech industry in about 10 years,” while researchers can make “nine-figure returns… within five months of founding a startup.”
• Entry barriers are lower than the discourse implies: “it’s not that you need $10 billion to have a relevant model,” and frontier-lab ideas can travel over “one beer” (with SSI raised as an apparent exception). The durable closed-lab advantage, in Lambert’s view, is serving “the cheapest intelligence per watt.” He poses an illustrative price-drop scenario — roughly 2X every eight months — and asks whether the result would be higher margins from the Jevons paradox or a market that fills up. The open question is whether coding is the main profitable market (favoring frontier integration) or whether the market is all knowledge work (“they can’t do everything”).

Deep dive

1. Model progress is capital allocation — and the org chart is the hard part

• Lambert’s opening framework: progress comes down to “different forms of capital that you put into them, and that is compute, data and talent” — and the differentiator is each lab’s ability to organize them. “Making the org chart work for building models is really hard,” which is why companies do complete bottom-up resets; Meta “has totally reset, and they’re on a positive trajectory again.”
• His frontier shortlist: anyone wanting to work at the true frontier should go to Anthropic or OpenAI — “the easiest bet” — with Meta, Google, and SpaceX AI harder to judge from outside. The two leaders have distinct cultures: OpenAI kickstarted the reasoning revolution — “that beautiful plot with the inference scaling laws is, like, peak OpenAI” — while Anthropic’s long coding focus paid off in “this Claude Code explosion.”
• What unites them: willingness “to spend egregious amounts to make a step towards a direction that people think is impossible” — chasing “true RSI… so that their company wins with the best models forever if you get there first.”

2. Chinese open models are a distribution strategy, not altruism — or state policy

• Lambert’s demystification: if Moonshot AI or Zhipu tried to sell an API from China, “nobody would know about them.” Releasing open models is “a way to shortcut this and get into the conversation” — “a very practical decision,” not government-directed. He says government involvement was not the reason: “They’re just trying to build a startup.”
• The structural prediction: tiers will emerge — the best closed models commanding the best margins, and “near-frontier models, which are a step behind,” open because they access a different, potentially less profitable market (private enterprise data, for instance). Fast-following is eased by the data economy itself: OpenAI and Anthropic fund data vendors who “a year later might sell to everybody.”
• A change of mind worth noting: “I thought this was gonna happen two years ago, and the open model companies have lasted much longer than I thought” — consolidation and M&A are only now starting.

3. Distillation: real, murky, and smaller than the caricature

• The compute asymmetry is the context: China has a comparable talent pool — “so many people being trained at such a high level” — though Lambert allows there might be a slight edge depending on who was where when AI really got going. The 5,000-H100-equivalents anecdote shows the compute gap, and distillation “is a way to effectively spend more money to get what is compute equivalent.”
• His literature-grounded skepticism: SFT-era distillation (generate completions from the best model, train on them, with “prompts are the key unit of post-training” as he and Hannah said at AI2) doesn’t map cleanly onto RL-heavy reasoning training. Fully open recipes are stuck on “a really outdated Qwen model” because it’s hard to get the teacher right — his hypothesis is that something about common open pre-training data makes it difficult. Even Claude Opus traces, he suspects, wouldn’t have made OLMo “a lot better.”
• He still flags genuine question marks that “make me support the labs a bit”: Claude proxies in China burning tons of tokens through intermediary accounts, “even cases where Kimi was serving Claude in their products to users,” and Chinese teams that were ahead of the AI safety community on stealing reasoning traces from thinking-token APIs. His bottom line is that “there’s very big error bars”; he says his “few months” estimate comes from the scientific literature.
• On policy, he sides with Jensen’s “that’s called competition”: Lambert questions government involvement in what he calls a terms-of-service violation, noting that labs can cancel access. If labs want to secure their models, putting them behind products or enterprise agreements would make distillation harder; as long as reasoning models expose thinking tokens through APIs, he says there’s no way to stop that type of distillation. He argues government action would do “more harm than good” because the Chinese labs “are not gonna stop.”

4. Two flywheels on different clocks: closed integration vs. open ownership

• Lisha Li’s steelman of the frontier labs: the compute-park-revenue-raise flywheel is “unprecedented” and bottlenecked until 2028 on chips and power, and anything threatening it — including distillation-driven catch-up — threatens the IPO path. Lambert agrees that a bad IPO would be bad “for the entire AI industry, realistically.”
• Lambert’s counter-structure: closed labs are integrated — efficient inference, scale serving, first access to high-value markets like coding, “where the revenue has just gone zero, 100 billion.” The open stack splits across builders, inference companies, and post-training companies, so customer signal “doesn’t just flow back to the model builder very easily.”
• But the open endgame is stickier: enterprises with data they “just cannot send over an API” eventually build internal evals that map to revenue, then figure out fine-tuning or go to Thinking Machines or Prime Intellect. They finish with “real ownership: model weights, no terms and conditions, and the ability to run them on their hardware.” It “takes way longer to build,” which is exactly why he’s “less concerned about this being kind of bubbly.” His conclusion remains hedged: “I don’t really know what’s gonna happen. It’s too early to know.”

5. “Poof generalization” is not the whole story — purchased data and the environment gold rush

• When Li raises robotics capability as potentially emerging without much investment, Lambert says labs started hiring and buying robotics data six to eight months earlier. Li offers an alternative account involving computer use, spatial reasoning, and Blender demos, so the example is not settled. Lambert’s broader view is that a lot of progress in less-verifiable domains comes from buying high-quality data and giving models worlds to act in and hill-climb on. Labs may describe it as “generalization, magic, poof,” but he says it is “probably some of A, some of B” — and a lot comes down to data, sector by sector.
• The environment market is real but flawed: he admits “I still have denial about it” when he sees the billions-of-dollars research budgets for environments, but says labs buy them because “the models are getting better from them.” Vendors “buy en masse and figure out what works”; corners get cut (his example: a test file that’s actually empty, so the model decides “I should still figure out how to get a high reward anyways”), and reward hacking is “downstream of this supply chain.” Data companies spend on “world building and revenue” while labs burn researcher time killing reward hacking — “somehow these two parties need to come together.”
• The OPSEC rot is concrete: overseas contractors who “don’t know they’re selling it” and “just upload it to GitHub.” Lambert’s price example is a scenario, not a forecast: if prices for bigger models fell by 2X every eight months for three years, his rough calculation suggests about 1% of the cost in four years. That raises whether Jevons paradox preserves margins or the market just fills up.

6. Trillium: the spring-ephemeral theory of open research

• The lab’s core product will be “open-source environments for tasks that are very high leverage” — modeled on the T-MaX terminal-task work he advises Hamish Iveson, a UW and AI2 student, on: fully synthetic data where a frontier model generates problems, varied prompts create diversity, and model rollouts help find bugs and eliminate bad examples. “This process isn’t magic. You read every data point, your data gets better” — it costs thousands or tens of thousands of dollars, but not many institutions invest in making these environments well and releasing them “because the profit motive is so high.” He joked with the author: “you could probably sell this data for a million dollars and then release the paper.”
• The name carries the thesis: trillium “blooms before the canopy has grown its leaves” and spreads nutrients through the growth cycle. The underlying economics: “all CS research, the benefit accrues to the tech industry in about 10 years” — while the alternative is “nine-figure returns for your business within five months of founding a startup with your friends.”
• A new open-community pathology, told as lived experience: AI agents now open “the craziest GitHub issues” with “fake research results,” while getting into AI still requires sitting with “weeks reading and knowing nothing,” writing code by hand for months, then waiting “six to 18 months to get noticed.” “All the AI agents frazzle that.”

7. Demystifying RSI — because even frontier researchers fear their own automation

• The emotional stakes are real: some of Lambert’s best friends — “some of the best people in the world at this” — tell him, “Nathan, I’m struggling. What am I going to do?” His answer: “RSI has so much mystique,” so Trillium will share its open-source stack for SFT and mid-training, including tools such as SkyRL and Miles, along with tasks, configs, compute, and stored trajectories. The aim is to document what the lab did over time and check back every six months to see what changed.
• The payoff is legibility for policymakers: “this is what Nathan asked his model to do, and this is what it got stuck on… that’s the bug it put in its code.” He’s not alone — OpenRSI, a group of academic institutions he’s not affiliated with, launched the prior week, and “it’s good to have different approaches.”
• His scoping of what actually recursively improves: models will be “really good at lowering serving costs” — companies have billions on the line, and models are good at that — but “you’re not gonna be like, ‘Claude, tell yourself to make better PowerPoints.’” Knowledge work will need environments or other data; “it’s gonna take a while.”

8. Pre-training is approachable; post-training is the Wild West

• Against the capital-moat narrative: “it’s not astronomically expensive to make a solid pre-trained model… you don’t have to spend a billion dollars to get a very good base model.” To first approximation, “you train on the internet”; the science ladder is accessible, and new entrants fork Megatron and never push upstream — something even prominent open-model labs have done. So he’s “not betting against new entrants”; the hard part is landing a first model good enough to justify the next raise. Thinking Machines released an initial model with transparent framing — “This wasn’t the best model. We’re proud of it; it has some cool things” — then released a second model and has Tinker. Lambert says they’re “off the ground.” He also mentions rumors that Meta will release another open-weight model; “Mark wrote the best blog post on this” at Llama 3, and “the guy got a little distracted. It seems like he’s coming back to the light.”
• Post-training is the opposite: the mid-training/RL/behavior-seeding cycle “is much more complicated than pre-training… it’s the Wild West.” At AI2, Lambert felt their post-training was falling behind because their cycle time wasn’t high enough — and that’s the opportunity: “it doesn’t take as much capital to make very big, compounding gains on the state of the art in the open source, because the bar is very low.”

9. Harnesses, swarms, and secrets that travel at one beer

• Lambert’s revised prior: he expected closed labs to be Apple — model designed for the harness, integrated economies of scale — but user demand for diverse harnesses “pressured” everyone toward models that work everywhere. It ended up “pretty well balanced, or at equilibrium.” Agent swarms (he gropes for the word: not a harness, “a coordination system”) will follow the same cycle — OpenAI and Anthropic first, then open models copy the shape once it’s known. He says Thinking Machines Lab won’t do it first, in part because it doesn’t have the compute.
• Why secrets don’t hold: “Silicon Valley is famous for ideas traveling over one beer” — people are scientists who want to tell their friends. SSI is raised in the conversation as an apparent exception. What labs can win on is “serving the cheapest intelligence per watt, per score on an eval.” The open question both discuss: if only coding is profitable, frontier integration wins; if the market is all knowledge work, “they can’t do everything” — Li says there will be many winners, which is why a16z invests in both frontier labs and startups.
• The takeoff skepticism, with its stated condition: scaling laws demand “exponential increase in compute for linear increase in performance”; math is “maybe the scientific field that’s best suited to this shape of intelligence,” and Lambert paraphrases Claude system cards as saying it would take, say, “10× more progress to make this a fast takeoff.” His update condition: if an AI discovers something very different from the transformer that unlocks a new type of intelligence, “it is worth updating on this.” Until then, the societal gap is capability progress alongside “pretty much nonexistent preparation” for public cybersecurity. “Not fearing fast takeoff does not mean that you don’t worry about risks.”