[LIVE] Anthropic Distillation & How Models Cheat (SWE-Bench Dead) | Nathan Lambert & Sebastian Raschka

Summary

• Anthropic’s distillation warning turns frontier API access into a geopolitical and capability-control issue. Nathan Lambert argues GPU-constrained Chinese labs “obviously should do this”: buying Claude outputs is easier than generating comparable synthetic data internally, while Anthropic labels distributed collection an “attack.” Terms of service mainly let providers terminate access, but renewed enforcement raises the possibility that providers keep their strongest models in products or restrict premium API access.
• The enforcement problem is epistemic: evaluation and distillation look identical while the API is running. Sebastian Raschka uses essentially the same loop for MATH-500, a 12,000-example math set, and book experiments with DeepSeek V3.2; only later would someone save the outputs and train. Volume, repetition, and topic distribution might expose collection at tens or hundreds of thousands of requests, but legitimate benchmarks run questions three to five times and production chatbots also generate huge traffic—creating an uncomfortable privacy trade-off.
• Anthropic’s evidence looks highly timing-sensitive, particularly the gap between MiniMax and DeepSeek traffic. swyx highlighted MiniMax redirecting nearly half its traffic when Opus 4.6 launched, consistent with active MiniMax 2.5 training; DeepSeek’s roughly 150,000 exchanges could reflect a different training window. Without knowing whether requests span four weeks or six months, Nathan sees Anthropic’s DeepSeek emphasis partly as messaging: it is “the only Chinese AI name that people in the US know.”
• Distilled data is not automatically valuable—the teacher must fit the student. Nathan says Qwen-dense models often teach Qwen-like open models better than stronger frontier systems, while GPT-OSS is also a surprisingly good teacher: “Just because you have this data does not mean it’s actually gonna make your model better.” A lab needs evidence that Claude data produces signal before paying to extract 100 billion tokens, especially when large models generate at roughly 40 tokens per second.
• The API may be a weaker moat than the products built around it. Nathan calls the API market “brutally competitive” and suggests a distillation-worried Anthropic could release its best model directly into Claude Code without offering it broadly; Sebastian counters that chatbots, OpenClaw workloads, and users exceeding subscription caps create a large unsubsidized-token business. swyx thinks Anthropic has “Apple vibes,” while describing OpenAI’s leadership as less likely to lock models behind products out of idealism and principles—even though the last three GPT-5 Codex variants reportedly received two-to-four-week product-exclusive windows.
• SWE-bench Verified has stopped discriminating among frontier coding models because both saturation and task defects overwhelm the signal. Scores cluster in the 80s while run-to-run noise is roughly 0.5–1 point, making tiny release-to-release gains “obviously bullshit.” OpenAI’s later audit found 59% of the stubborn remainder could not be solved as written; one task secretly required the exact string get_annotation, making memorization the only reliable route.
• The next evaluation cycle becomes materially more expensive and more private. GPT-5 used knowledge from later Django versions, while Gemini Flash and Opus could reconstruct a task and solution from its ID—evidence that public GitHub benchmarks contaminate models even without deliberate cheating. SWE-bench Pro adds newer tasks, private/public splits, more repositories and languages, but the panel expects frontier evals to cost millions, then potentially “tens of millions and hundreds of millions of dollars,” advantaging labs and data providers with scale.

Deep dive

1. Anthropic turned ordinary distillation into a strategic-access fight

• Sebastian’s baseline separated classical distillation—training a smaller model on a larger teacher’s logits—from today’s looser LLM practice: collect generated answers, then supervised-fine-tune on them. His example was DeepSeek R1, described in the conversation as 6.71 billion parameters, producing roughly 1- and 3-billion-parameter variants; “everyone does that” when creating compact models.

• Nathan extended the mechanism to frontier systems: labs can use something like Claude Opus to build related models, while internal teams possess richer tools than API customers. The controversy begins when the teacher belongs to a competitor, not when a lab distills its own flagship.

• Nathan’s geopolitical call was categorical: Chinese labs facing a severe GPU shortage “obviously should do this,” because purchasing synthetic data through APIs is easier than generating it themselves. Anthropic’s choice to call the distributed collection an “attack” fits its broader framing of AI competitiveness.

• Frontier providers have long prohibited using API outputs to train competing models. Nathan framed terms of service primarily as access rules—the provider can cut a violator off—not as proof of what happens downstream. Fears around early Alpaca-style research largely faded; ByteDance was a possible prior cutoff, but competitive anxiety has revived the issue.

2. Detection requires surveillance of behavior, not proof of training

• Sebastian’s challenge was practical: his evaluation and distillation scripts are nearly identical. Both submit questions, collect answers, and loop through a dataset; he cited MATH-500, another 12,000-example math set, and an experiment using DeepSeek V3.2 through OpenRouter. The request process itself does not reveal whether those answers later become training data.

• Nathan’s first discriminator was scale. A normal evaluation may run once, though stochastic benchmarks can repeat identical questions three to five times; tens or hundreds of thousands of requests begin to look less plausible as evaluation. Industrial extraction across distributed, rate-limited accounts creates more obvious repetitive patterns.

• Sebastian’s pushback—worth keeping—is that legitimate scale also gets large. An enterprise chatbot can generate enormous traffic, while a serious evaluation suite can span many benchmarks. Providers might distinguish a broad distillation distribution from narrow math or customer-service traffic, but doing so means inspecting what customers generate: “kind of a weird thing,” even if users should not assume internet-hosted LLM traffic is private.

• Nathan corrected the idea that enforcement began with China: Anthropic had already blocked OpenAI and xAI from using its models and may have explicitly accused xAI of distillation. The Chinese-lab post was still the highest-profile escalation because it publicly connected account behavior to geopolitics.

3. Anthropic’s traffic totals say less without a clock

• Nathan called the post “political in a way—maybe not political,” meaning it was designed to make a larger point rather than fully explain the underlying measurements. His sharpest suspicion: DeepSeek appeared because it is “the only Chinese AI name that people in the US know,” despite its detected activity being far smaller.

• swyx’s timing read focused on MiniMax 2.5. While MiniMax was apparently distilling, Anthropic released Opus 4.6 and observed nearly half the traffic switch to the new model—an unusually clean sign that the same pipeline was shopping for the newest teacher.

• The missing timeframe is load-bearing. Whether the reported calls occurred during four weeks or six months changes the interpretation completely, especially if DeepSeek was training V3.1 and V3.2 about a year earlier. Millions of exchanges take substantial time when accounts are rate-limited and continually replaced.

• Sebastian floated MiniMax’s roughly 3×-smaller, faster architecture—he thought it used grouped-query attention rather than MLA or DeepSeek Sparse Attention—as one possible reason for different usage; he also mentioned the possibility of a free tier. Nathan rejected a 10× or 3× efficiency explanation—“I can guarantee it”—while allowing a small chance DeepSeek found the right research idea or data early. His default model is simpler: most labs test available APIs, substitute teachers into one pipeline, and keep whichever ablation improves performance.

4. The strongest model is not necessarily the best teacher

• Nathan’s central technical qualification was that teacher quality is relational. Qwen-dense models often teach open-weight students particularly well because many students are Qwen models or have become Qwen-like; OLMo also learned effectively from Qwen outputs. GPT-OSS, somewhat unexpectedly, is another good teacher.

• Scaling the same prompts through GLM-4.7, a larger DeepSeek model, or a recent Qwen MoE does not guarantee gains. The suspected mechanism is compatibility between the teacher’s token probabilities and the student’s base distribution; sampling settings also matter: “Just because you have this data does not mean it’s actually gonna make your model better.”

• Sebastian independently observed that same-family distillation performs better. A much stronger model may differ too sharply in style or from the Q&A distribution encountered during pretraining. He also preserved the old/new distinction: true logit distillation is practical with open weights, while Claude and OpenAI expose only limited top-token information; Nathan clarified that OLMo used generated tokens, not logits.

• Quantity introduces its own trade-off. Producing 100 billion tokens at roughly 40 tokens per second is slow, so a medium teacher may deliver more usable data sooner than the best frontier model. That makes data quantity versus quality a pipeline ablation in its own right. swyx’s broader maxim was, “The model you deploy is never the model you train”; Nathan agreed labs likely keep larger internal teachers, though he was “not 100% sure” how often they train dense models specifically to deploy MoEs.

5. Product exclusivity could become the answer to API leakage

• Nathan’s provocative business thesis was that frontier APIs are “brutally competitive” and weakly differentiated. APIs fit incumbent clouds such as Google and Azure—and emerging infrastructure providers such as Lambda and Nebius—more naturally than product-led labs. If Anthropic truly fears extraction, it could put a new model into Claude Code immediately and skip broad API release.

• Sebastian’s rebuttal was demand-side: customer-service products, applications exceeding subscription token limits, and OpenClaw workloads can require APIs. At sufficient scale, selling those tokens without subscription subsidies “is actually not a bad business model,” assuming inference pricing remains profitable.

• swyx’s tie-break gave Anthropic the higher probability of closing models because it has “Apple vibes.” He doubted OpenAI would constitutionally lock everything inside products, attributing that reluctance to idealism and principles rather than economic incentives. Yet the economic direction is visible—the last three GPT-5 releases reportedly gave Codex variants a two-to-four-week exclusive window, though Sebastian cautioned that the latest timing may simply have promoted the new Codex app.

6. SWE-bench Verified died from saturation and unsolvable tasks

• swyx traced SWE-bench to a Princeton group that paired thousands of open-source issues with pull requests and tests. It favored a small set of popular repositories, creating selection bias, but became the first widely adopted agentic coding benchmark after Devin reported it. Performance rose from roughly 13% near launch to around 80%.

• Unlike HumanEval-style autocomplete, SWE-bench gives an issue and repository state without prescribing the path to a fix. OpenAI spent what swyx and Nathan guessed was a couple million dollars curating 500 tasks, with three humans vetting each. Ironically, OpenAI initially could not run all 500 on its own infrastructure, so some releases reported a “subset of the subset” with a smaller denominator.

• Saturation made the leaderboard nearly meaningless: models congregated between roughly 81% and 89%, while reruns vary by about 0.5–1 point. swyx called tiny incremental records “bullshit.” The panel noted that 500 binary tasks imply 0.2-point increments, while Nathan added that rubric sub-scores permit finer movements. Sebastian’s practical comparison was sharper: MiniMax 2.5 feels worse than GPT-5.2, yet the benchmark scores them similarly.

• OpenAI then re-audited the stubborn failures using six people per task plus escalation. It found 59% of that remainder could not be solved as written. swyx’s specimen required a hidden test to find the exact name get_annotation, although the prompt never demanded it. The panel’s conclusion was that only memorizing the original answer would reliably pass—making the defect useful as a “honeypot” or canary for contamination.

7. Public benchmarks contaminate themselves, and replacements will cost more

• GPT-5’s chain of thought sometimes invoked information from later Django versions to solve older tasks. The panel stressed that this need not be malicious: GitHub training data can absorb the benchmark through cloned repositories, downstream unit tests, blog posts, conference talks, or documentation of features released after the benchmark snapshot.

• OpenAI also prompted Gemini Flash and Opus with only a task ID; the models could “vomit out” the problem and solution. Sebastian found one-pass memorization remarkable given corpus scale. Sebastian said labs have probably learned an “art” of repetition across training stages: too much reuse at pretraining can cause forgetting, while Nathan added that continued pretraining needs some revisiting of old data and that even roughly 1%–2.5% added data can preserve striking detail.

• SWE-bench Pro starts over with newer dates, private/public splits, broader repositories and languages, and improved tests. Nathan argued Scale AI has effectively unlimited motivation and budget because good data is “literally existential” to it. Sebastian kept the necessary skepticism: SWE-bench Verified also had professional reviewers, so defects in Pro may only become visible after models saturate it.

• Private evaluation may require controlled access—API keys, agreements, or server-side scoring—to reduce the risk that the private tasks leak. swyx’s cost forecast was the investor-relevant endpoint: frontier evals may cost millions, then “tens of millions and hundreds of millions of dollars.” Coding and math remain comparatively objective; computer-control tasks and subjective work, potentially represented by GDPval, are harder and may take another two years to benchmark well.