Really Big Test-Time Compute in AI Changes Benchmarks, Safety and Research with OpenAI's Noam Brown
Summary
- A benchmark score without a test-time-compute budget no longer measures model quality cleanly. Brown says 5.5 looked only modestly better than 5.4 on standard grids, while users quickly found a larger improvement. He attributes the understatement to unmeasured test-time compute and describes 5.5 as more efficient with its thinking; separately, he says controlling thinking time reveals a substantial o3-over-o1 jump. The replacement is a cost, token, or time curve: “There should be an x-axis.” In practice, he says users should iterate quickly when needed and allow longer thinking when the problem warrants it.
- The industry may be releasing models before anyone discovers their capability ceiling. Modern systems can improve for weeks and remain on an upward slope beyond 100 million tokens, while new models arrive every two or three months. Brown proposes extrapolating expensive performance from smaller runs—for example, predicting a $10,000 inference result using experiments capped at $10 or $100.
- Safety frameworks inherit the same measurement failure, with much higher stakes. Policies designed around GPT-3 largely treat capability as intrinsic, but today it is “a function of how much money you put into it”: a model given $10,000 may do far more than at $10, and $10 million may unlock more again. Existing policies do not clearly answer which budget should govern evaluations of cyber, bioweapon, or other dangerous capabilities.
- Brown’s poker-solver test suggests model progress is much larger than benchmark deltas imply. With 5.2 he built a river solver roughly five times faster than working alone, although the models could be unreliable; 5.5 can nearly build a full solver with gentle steering. He would not be surprised if, within six months or a year, one model could reproduce “basically my entire PhD thesis.”
- Existing models may contain valuable scientific capability that remains uneconomic to excavate only briefly. Brown says a scaffolded 5.5 could likely have found the Erdős unit-distance disproof before OpenAI’s internal model, at a ballpark cost of $1,000–$100,000 of inference. If each release cuts such costs by 10x or 100x, and sometimes more, the investment question becomes when to deploy compute versus wait for the next cost curve.
- Recursive self-improvement looks gradual because research taste and elapsed time remain binding constraints. Models can optimize existing algorithms by 10–100x yet still fail to invent a better one, and their strongest results require long inference runs: “Time itself becomes a bottleneck.” The larger upside may come from persistent multi-agent knowledge accumulation, not an overnight intelligence explosion.
- Routing businesses must prove they beat simply letting the strongest model think longer at the same cost. Consensus across models can raise scores, but Brown asks whether it still wins after equalizing test-time compute—and whether benchmark gains survive real-world use. That makes budget-normalized evaluation central to assessing routing, orchestration, and model-choice layers.
Deep dive
1. Model quality is now a curve, not a grid cell
Brown traces the 5.5 launch skepticism to the familiar benchmark grid: it showed only a few percentage points over 5.4, while users quickly found a much larger experiential improvement. He attributes the understated grid results to failing to control test-time compute, describes 5.5 as more efficient with its thinking, and separately says that controlling thinking time reveals a substantial o3-over-o1 jump.
The old instruction—run each model until performance plateaus—worked in “GPT-3 land back in 2022.” Modern models, when reasonably scaffolded, can think productively for weeks, pushing the plateau beyond any practical release evaluation. Every comparison therefore needs a fixed budget or performance plotted against tokens, cost, or time. For actual use, Brown favors flexible thinking time: quick iteration when responsiveness matters and longer runs when the user wants deeper work.
Cyber evaluations Brown referenced show models continuing to improve at 100 million tokens and beyond. His proposed research problem is to estimate the far end cheaply: can runs capped at $10 or $100 predict performance at a $10,000 inference budget?
Benchmark maxing becomes easy without that control: run a model five times, select the best answer—or let another model judge—and the score rises. Brown’s objection is not that scaffolding is invalid, but that the comparison is misleading unless compute is equalized; private held-out sets can help guard against direct benchmark optimization.
2. Poker exposes both the capability jump and the remaining unreliability
Brown uses poker bots as a personal evaluation because little usable open-source code exists, despite abundant published papers. Building one requires sustained reasoning, iteration, and handling “a lot of small gotchas” he already understands well enough to diagnose.
With 5.2, Brown built a river solver—the final stage of poker—about five times faster than alone. It behaved like a graduate student needing gentle steering, especially around blockers, yet its code optimization was striking: he says it made the work 10x faster than he could, and later characterizes the gains over his algorithms as 10–100x.
The weakness was that 5.2 felt like it was “gaslighting me.” In a separate test with another model, a unit test said folding $100 already in the pot lost $92, then defended itself: “It’s close to 100. It’s fine.” With 5.5, Brown says the task is nearly zero-shot; he can now work toward a full-scale solver with gentle steering, although he has not yet simply given it a month-long
/goaland received a completed solver.
3. Safety policy has no agreed price tag for capability
Responsible scaling policies and preparedness frameworks were largely designed when giving GPT-3 $10 million produced little beyond a $10 or $1 run. Today, Brown argues, $10, $10,000, and $10 million can represent materially different capability levels—leaving an unanswered question: “At what budget should you evaluate these models?”
Sarah Guo’s pushback—worth keeping—is that this is the mirror image of useful capability scaling: if large budgets keep improving desirable work, they may also improve tasks society does not want performed. Brown declines to settle whether releases should be blocked, but calls it an “inconvenient truth” that many policies do not account for the budget issue.
Longer horizons compound the mismatch. Models can now execute experiments for weeks or months, yet releases arrive every two or three months; fully testing a six-month task requires six months. Brown’s conclusion is stark: “Nobody actually knows what the ceiling of capabilities is,” while labs face competitive pressure not to delay launches.
Brown says the river solver is already within reach if he gives a model
/goal, but a full poker solver is not yet at the point where a bare/goaland a month of runtime would reliably produce it. He expects that threshold soon.
4. Latent scientific capability creates a wait-versus-compute trade
Brown’s central example is OpenAI’s internal model disproving the Erdős unit-distance conjecture at what he calls a “dirt cheap” budget. The result mattered because mathematicians had spent substantial time on the problem and found the model’s method interesting and useful, not merely correct. Brown says the team had trained a new model, was curious what it could do, and had not invested heavily in this particular search.
Afterward, someone elicited the same disproof from 5.5 by asking for many approaches, identifying a promising path, and repeatedly extending it. A general-purpose scaffold could automate that search; Brown ballparks the expense at $1,000–$100,000, meaning the capability was publicly accessible but insufficiently explored.
The economic complication is that a new release might cut the same discovery’s cost by 10x or 100x, perhaps more in some cases. Brown says OpenAI does not encourage its mathematicians and physicists to spend all their time harvesting current models’ limits; the priority is building safer, more capable models that let scientists worldwide solve those problems more cheaply.
5. Test-time compute slows recursive improvement before it removes researchers
Brown rejects the claim that arbitrary inference spend already yields general superintelligence. Factual recall resembles asking someone for Lincoln’s birth date without Wikipedia: a week of thought cannot recover unknown information. Sudoku is the opposite extreme—randomly test enough assignments and, given sufficient time, any puzzle eventually yields.
Research sits between those poles, and today’s models lack “very good research taste.” Brown watched them radically optimize his existing poker algorithms, then fail—even with more time and published work—to invent a superior algorithm. Better scaffolding might help, and each release improves, but “please come up with a better algorithm” is not yet enough.
His recursive-self-improvement model is bottleneck migration: if one research step accelerates 100x, progress becomes limited by steps that did not. As those constraints shrink, he expects a gradual takeoff that transforms researchers’ work rather than immediately replacing the entire research cycle.
That dependence on lengthy inference argues against an overnight intelligence explosion. “Time itself becomes a bottleneck,” Brown says, because powerful outputs still require models to run. Frontier competition remains intense and AI already amplifies lab researchers, though Brown takes comfort that researchers across the labs understand both the upside and the severe risks.
6. Durable orchestration must outperform longer reasoning at equal cost
Brown sees more headroom in multi-agent systems through civilization’s example: human achievement compounded because billions of people preserved, shared, and extended knowledge. Today’s models are instead “born into a world,” live for a short context window, then “just disappear”; Moltbook and OpenClaw were overhyped when they appeared, but indicated where large-scale coordination could go.
For everyday users, his advice is to revisit assumptions formed around 2022 or 2023 models. He now uses AI for tax questions and condo paperwork and says he can arguably trust its output more than an expert human’s—while his poker experience still illustrates why verification matters when errors become confidently rationalized.
The publishing norm remains a “bad equilibrium”: everybody supplies a grid because everybody expects one. Routing and consensus layers face the same test—compare them against a single model thinking longer at equal cost, then probe fragility and real-world relevance. Otherwise, an apparent orchestration advantage may be benchmark-specific and may not transfer to real-world use.