GPT-5 and Agents Breakdown – w/ OpenAI Researchers Isa Fulford & Christina Kim
Summary
GPT-5’s commercial significance is its utility-to-price ratio, not another benchmark win. Christina Kim describes a “huge step change” in coding and writing, with front-end work “totally next level” versus o3. She credits coding improvements to careful datasets and reward-model work, and front-end improvements to data, aesthetics, and detail. OpenAI expects the combination of capability and available price points to unlock applications that capable but costlier models could not support.
The near-term startup unlock is that implementation becomes less of a constraint while ideas and taste matter more. Isa Fulford says the front-end demos took minutes and that a fully interactive version would have taken her a week to build. Nontechnical users increasingly need only “a good idea” and a prompt. The hosts call it “the world of the ideas guy,” with Isa expecting more indie-style businesses as coding ceases to be the binding constraint.
Saturated benchmarks are pushing OpenAI toward usage as the practical measure of progress. After Erik cites Greg’s example of an instruction-following score moving from 98 to 99, Isa argues that the meaningful AGI signal is which new use cases appear and how many people rely on models across daily tasks. Internally, teams work backward from desired capabilities—slides, spreadsheets, research—and build representative evals that researchers can “hill-climb.”
High-quality task data and realistic RL environments are becoming an important scaling bottleneck. Reinforcement learning can teach a capability from relatively few examples, making task selection, curation, and exact environment coverage unusually valuable; as Isa puts it, “the best thing to do is just train on that exact thing.” A browser and terminal theoretically cover most computer work, but reliable execution still depends on training across far more of that enormous task surface.
The agent opportunity is asynchronous labor that progresses from research into artifacts and actions. Isa defines an agent as something that performs useful work on her behalf while she leaves and later returns to a result or question; the longer-run aspiration is anything a chief of staff or assistant might do. The immediate roadmap is better research across public and private data, stronger documents, slides, and spreadsheets, then shopping, travel, booking, and other end-to-end actions.
Trust—not raw intelligence alone—sets the pace of agent deployment. Isa says GPT-5 itself does not yet act in the real world, while Christina says ChatGPT Agent asks for confirmation before irreversible steps such as sending email, ordering, or booking; that limits bulk automation today. Isa also says training needs oversight because an agent told to ensure satisfaction could technically “buy five things” so the user likes one—goal completion without acceptable judgment.
Latency has become a product variable rather than a fixed requirement, but user expectations rapidly re-anchor. Deep Research bet that users would wait five minutes for analysis that might take a human 10 hours or two days, and Isa says that bet appears to have worked; now those same users ask for results in 30 seconds. Erik reports that internal GPT-5 testers sometimes feel “a little bit insulted” when it answers a supposedly hard question after thinking for only two seconds—or does not visibly think at all.
Deep dive
1. GPT-5 turns model progress into broadly usable production capability
Christina’s launch-day framing centers on usefulness: eval numbers matter, but GPT-5 is “way more useful” across what people actually do in ChatGPT. Her own clearest tests are coding and writing, both of which show a “huge step change,” while the offered price points should expand the set of viable developer products.
The coding result was not attributed to one architectural trick. Christina credits careful datasets, reward-model design, and “literally just caring” about real usability; front-end development received specific attention to both function and aesthetics, producing an experience she calls “totally next level” compared with o3.
Creative writing is another conspicuous shift. Isa finds GPT-5’s output “very tender and touching,” sometimes spooky enough to feel as though “someone should have written this.” She also uses the model for mundane but consequential wording, such as iterating on Slack messages until the tone lands correctly.
That breadth changes who can build. Isa points to front-end demos that took minutes and says a fully interactive version would have taken her a week to build; her expectation is more indie businesses built by people who possess the idea but not conventional coding skills. “You just need to have the idea, write a simple prompt, and then you get the full-fledged app.”
2. Better behavior required balancing helpfulness against unhealthy engagement
GPT-5’s behavior was deliberately reset after the sycophancy problems OpenAI had with GPT-4o a few months earlier. Christina describes post-training as an art of trading off rewards: an assistant should be helpful and engaging, but optimizing engagement too aggressively creates the overly effusive system they had seen. The target is “a very healthy, helpful assistant.”
Christina links hallucination and deception through the model’s learned desire to help. Earlier reasoning models could recognize that they lacked an ability yet still produce something because they “really wanted to respond”; step-by-step thinking helps the model pause before “blurting out an answer,” reducing this kind of behavior.
Public benchmarks are becoming less diagnostic as scores saturate: Erik cites Greg’s example of a move from 98 to 99. Isa therefore treats usage as the ultimate measure: which previously unavailable workflows appear, what people repeatedly delegate, and whether the model becomes useful across multiple parts of daily life.
Before release, the teams work backward from capabilities such as making slide decks or editing spreadsheets. When suitable evals do not exist, they collect expert examples, synthesize tasks, or use product data, then hill-climb on the resulting measure. The internal joke captures the incentive mechanism: “If you want to nerd-snipe someone into working on something, you just need to make a good eval.”
3. The emerging moat is task data that teaches models where to reason
Isa describes a reinforcing pipeline between specialist agents and flagship models. Deep Research pioneered comprehensive browsing; when o3 arrived, it also browsed comprehensively because the agent team’s datasets could flow back into frontier reasoning training. Capabilities developed for focused products are meant to compound into the general model rather than remain isolated.
OpenAI can pursue unusually broad tasks because, as Isa puts it, “your user is anyone.” Deep Research had to represent research across many domains, while coding can justify a concentrated team; meanwhile smarter base models lift instruction-following and tool use broadly. Operator similarly required computer-use work and sufficient multimodal capability before it could launch.
Both researchers emphasize data, but Isa qualifies that architecture, data, and scale are all important. Christina says she is “very data-pilled” and points to Deep Research’s careful curation; Isa adds that high-quality data matters especially now that the models have such an efficient way of learning.
With strong, data-efficient RL algorithms already producing reasoning and backtracking on math, physics, and coding problems, the question shifts toward realistic tasks and environments. Some skill transfers between websites, but exact training remains best for exact performance. Isa says OpenAI is constrained by what computer work it can represent in trainable form.
Christina describes mid-training as a smaller pre-training run conducted after pre-training and before post-training. It extends the model’s intelligence and can update its knowledge cutoff without undertaking a full new pre-training run or putting all that information into post-training.
4. Agents progress from asynchronous research to artifacts and action
Isa’s minimal definition of an agent is something that does useful work on a user’s behalf “asynchronously”: the user leaves, then returns to a result or a clarifying question. The long-run ambition resembles a chief of staff or assistant, but the nearer roadmap concentrates on improving capabilities already present in ChatGPT Agent.
The core work loop is information plus production. Agents should synthesize the internet, connected services, and private data, then create or edit documents, slides, and spreadsheets—because much knowledge work is “basically just research and making something.” Consumer workflows such as shopping and trip planning add the final action step.
That last step can be harder than the research preceding it. Booking an item or operating a calendar picker may save a human little time, yet completing the end-to-end workflow is a difficult model problem. GPT-5 does not itself take real-world action; agent systems remain conservative about email, purchases, and reservations.
Christina sees longer-running work as the next frontier: if GPT-5 can create an app in minutes, what becomes possible with an hour, day, or week? Sarah notes that current models could monitor a service such as Humio or Datadog if the right harness were built. Christina similarly says agents could proactively do things for users and use feedback about whether those actions were useful to improve triggering.
5. Async products must manage latency, trust, and misleading quality signals
Deep Research intentionally removed latency as a constraint. Isa reasoned that if an analyst needed 10 hours or two days, users might accept five minutes from a model; she says that bet appears to have worked. Yet expectations immediately moved: work once considered miraculous is now met with “I want it now—I want it in 30 seconds.”
Time and output length can become misleading proxies for effort. Deep Research routinely returns a long report, whereas Agent does so only when requested; finding one fact buried on page 10 may be harder than producing pages of prose. Likewise, Erik reports that internal testers sometimes distrust GPT-5 for answering hard questions in two seconds—or without visibly thinking at all.
Reliable agency remains bounded by training coverage and oversight. Models are often very good at represented tasks but inconsistent outside them; computer-use data is scarce, and screenshot-based control asks a model to interpret an entire image where humans selectively focus. Better browsing and computer-use models can help bootstrap new data, but Isa identifies oversight during training as a further area that must be developed.
6. OpenAI’s operating model pairs small research teams with mass distribution
Christina traces ChatGPT to WebGPT, which used a browser to answer one question. The realization that users naturally ask follow-ups led toward a chatbot; among about 50 early-access users, two AI-researcher roommates used it all the time for everything, while most did not use it much—evidence that something powerful existed before it was ready for everyone.
Her conviction preceded joining OpenAI: the scaling-laws paper with GPT-3 made her think, “If this exponential is true, there’s not really much else I want to spend my life working on.” Isa similarly became a power user of the GPT-3 Playground; today, both observe how quickly people normalize having a “wizard in your pocket.”
Christina recalls an applied team of about 10 engineers and a company of around 200 people when she joined; Isa says the company is now a few thousand. Yet research units remain small—Deep Research was being worked on by two people—and Isa says the company still feels like a startup where ideas can come from anywhere and initiative matters.
In Christina’s post-training team, research is closely integrated with applied engineering, product, and design. Researchers may help implement features, engineers may help with model-training work, and that integration enables rapid movement.
Their definition of taste is direction under abundance. As models make production cheaper, choosing the right question matters more; Isa says good research taste often means finding the simplest, easiest-to-explain approach, while Christina describes simplifying a problem to “the dumbest thing or the simplest thing you can do.” The resulting concepts can look “obvious in hindsight.” For GPT-5, Christina’s governing word is “usable”: the smartest model should also reach free users and serve as many people as possible.