Extreme Harness Engineering: 1M LOC, 1B toks/day, 0% human code or review — Ryan Lopopolo, OpenAI
Summary
- Ryan Lopopolo’s team produced roughly 1 million lines of code in the total codebase and 1,500 PRs in five months under a deliberate constraint that Ryan write no code himself. The constraint was deliberate: if enterprise agents were to perform his job, “the only way I could do my job was to get the agent to do my job.” Codex Mini initially failed at whole features, forcing the team to build smaller primitives that agents could later assemble reliably.
- The productivity curve first moved sharply backward, then overwhelmed traditional engineering throughput. The first month and a half was “10 times slower” than Ryan coding by hand, but paying that tooling cost ultimately made the system roughly 10x faster than one engineer. The broader call is that agent adoption requires funding an assembly line before expecting assembly-line economics.
- At agent scale, synchronous human attention—not tokens or code generation—is the scarce resource. The team progressed from close code review to mostly post-merge sampling because models are “trivially parallelizable,” while people still need sleep and cannot follow dozens of concurrent trajectories. Autonomy remains bounded: this is a greenfield native application, and a human still cuts the release branch and approves a smoke test before distribution.
- The durable advantage is encoded institutional taste: documentation, tests, lints, observability, and review agents that turn every failure into future context. A missing timeout becomes both a fix and a rule requiring timeouts on every network call; a failed build or PR comment signals that “at some point the agent was missing context.” The advantage therefore shifts from privately held engineering intuition toward a continuously improving, machine-readable operating system.
- Symphony removed terminal babysitting; Shawn described the resulting workflow as producing another 5x productivity increase. Output had already risen from 3.5 PRs per engineer per day before 5.2 to 5–10 after 5.2, but context switching exhausted the humans; Symphony instead drives tickets through completion and escalates a compact merge decision. If a result fails review, it deletes the worktree and PR and starts over because code now carries “close to zero investment in the actual authorship experience.”
- Code abundance may compress parts of the software-dependency and internal-tools markets, although Ryan preserved important limits. He thinks a dependency of a couple thousand lines can already be internalized in an afternoon, stripped to the exact functionality needed, while “ghost libraries” can distribute a reproducible specification instead of source. Shawn’s counterpoint was that internalization resets confidence to zero, forfeiting lessons accumulated through widespread use and “many eyes.”
- In the later Frontier discussion, Peter Steinberger described the same factory model extending from coding into governed enterprise labor. Frontier targets observable agents integrated with IAM, security tooling, workspaces, bespoke safety policies, and revocable authorization. David Luan reported that Codex had passed 2 million weekly active users while growing 25% week over week; Shawn cited roughly 1 billion tokens of intelligence deployed daily. Yet the frontier remains clear: hard, novel product creation and the “gnarliest refactorings” still require sustained human steering.
Deep dive
1. A no-human-code constraint forced the harness to become the engineer
Ryan works in Frontier product exploration, packaging OpenAI models into enterprise products. After six to eight months with coding agents, he judged them sufficiently “isomorphic to me in capability” to impose a hard constraint: he could not write the product’s code himself.
Early Codex Mini could not assemble an entire feature. Ryan’s response became the foundational operating loop: “Whenever the model just cannot, you always pop open that task, double-click into it, and build smaller building blocks” that it can later reassemble into the larger objective.
The cost was severe before it compounded: “The first month and a half was 10 times slower than I would have been.” After five months, however, the greenfield Electron system had roughly 1 million lines in its codebase, about 1,500 PRs, and reported throughput around 10x hand development because the team had built “the tools, the assembly station.”
2. Model upgrades now force infrastructure upgrades
The repository lived through GPT-5, 5.1, 5.2, 5.3, and 5.4, with each generation bringing a different working style. Ryan’s team therefore treated model behavior as an infrastructure constraint: when the model changed, the build and repository sometimes had to change with it.
Under 5.2, Codex lacked background shells, so blocking scripts could perform long-running work. With 5.3, background execution made the agent “less patient, less willing to block,” prompting a migration from a bespoke Makefile through Bazel and Turborepo to Nx until the build completed in under one minute.
The minute was not a magical threshold; it was an invariant the team could enforce. Instead of letting build latency drift upward and scheduling a multiweek platform cleanup, cheap parallel tokens continuously “garden” the repository, reducing dispersion across code and the software-development lifecycle.
3. Human attention became the capacity ceiling
Ryan’s economic framing is blunt: “As many GPUs and tokens as I am willing to spend, I can have the capacity to work on the codebase.” The fundamentally scarce input is synchronous human attention, which cannot be parallelized and is consumed by terminals, reviews, context switching, lunch, and sleep.
The operating question is consequently not whether an agent made one mistake, but “Where is the agent making mistakes? Where am I spending my time? How can I not spend that time going forward?” Each answer should become automation trusted enough to remove that piece of the lifecycle from human supervision.
Code review has largely moved post-merge. Ryan compares his role to group-tech-leading a 500-person organization: sampling representative code to infer systemic struggles is appropriate, while forming detailed opinions on every PR is not.
The hosts pressed on the safety boundary. This is a greenfield native application rather than continuously deployed infrastructure with strict uptime requirements; a human still cuts the release branch, performs a blessed smoke test, and approves promotion to distribution.
4. Every defect is converted into durable, textual context
“The models fundamentally crave text,” Ryan said. The repository supplies it through a short top-level guide, core beliefs, skills, a technical-debt tracker, and quality scores that let Codex inspect business logic, compare it with guardrails, and propose work it can later burn down automatically.
His timeout example captures the mechanism: after a page caused by a missing timeout, Codex can fix the call and update reliability documentation to require timeouts on all network calls. That rule can subsequently produce tests, lints, or focused review behavior, converting a point fix into process knowledge.
The hosts’ pushback—worth keeping—was that a permanent rule may omit legitimate exceptions, while instruction-following agents can obey it too literally. Ryan’s answer was optionality: skills are invoked when relevant, and prompts explicitly permit agents to challenge, defer, or contextualize instructions.
Early review agents “bullied” authoring agents into non-convergent rewrites. The team taught reviewers to bias toward merging and not surface findings more severe than P2, while authors could reject feedback or backlog scope-expanding suggestions rather than treating every comment as an immediate command.
5. Agent-scale teams require organization-scale architecture
Ryan avoids prescribing the detailed shape of business logic, but insists it use leverage-bearing primitives, such as a command class that supplies tracing, metrics, and observability automatically. The governing concern is not aesthetic authorship; it is whether every implementation inherits the capabilities needed for autonomous operation.
Ryan described the repository as roughly 500 npm packages and “10,000-engineer-level architecture”—architecture as deep as what would be normal for a seven-person team. If each person drives 10–50 agents, deep decomposition, strict interfaces, and sharding stop being premature complexity; they prevent concurrent work from trampling shared surfaces.
That scale creates a new human problem: nobody reliably knows the current code state. The team holds a 45-minute daily stand-up to fan out knowledge, even as agents author product code, tests, CI, release tooling, dashboards, documentation, evaluation harnesses, and repository-management scripts.
6. The coding harness is expanding into a general work harness
Ryan’s broader thesis is to collapse a user journey into code whenever possible, then let Codex supply the wiring and execution. The hosts rendered the implication plainly: coding agents may “eat knowledge work,” including tasks for which teams previously assumed they needed a bespoke non-coding agent.
Git’s multi-agent friction did not persuade Ryan to discard it. Worktrees and merge conflicts are tolerable when a “land” skill can push a PR, await reviewers and CI, repair flakes, merge upstream, enter the queue, and stay engaged until the change reaches main.
Ryan is “pretty bearish on MCP” where tool definitions are forcibly injected into context, interfere with compaction, and teach many calls the agent never needs. His Playwright example involved someone replacing the direct MCP setup with a locally built daemon and tiny CLI shim; the system improved without Ryan even knowing the substitution occurred.
CLIs work because they are textual and token-efficient. The ideal command suppresses successful noise and returns only actionable failures; even visual interfaces can be translated into ASCII layout alongside an image because agents do not perceive spatial design exactly as humans do.
7. Disposable code changes what should be built, bought, or distributed
Ryan agreed with Bret Taylor’s suggestion that some dependencies could be vendored away, though he limited current capability to “low to medium” complexity. A dependency of a couple thousand lines could be internalized in an afternoon, with generic functionality stripped away and only the required surface retained.
Codex Security can then inspect and modify that code directly, avoiding upstream patches, release waits, and transitive compatibility work. Shawn countered that scale testing and open-source scrutiny embody accumulated knowledge; his point was that internalization returns confidence to zero and requires rebuilding those assurances.
One of the team’s engineers spent an afternoon building a polished local DevTools Next.js viewer for an exported performance trace. Ryan later realized the human-facing tool was unnecessary: Codex could ingest the tarball and answer the debugging question in five minutes, exposing how old instincts can keep humans in loops they no longer need to occupy.
8. Symphony industrializes asynchronous agent work
Symphony began after output rose from roughly 3.5 PRs per engineer per day at December’s end to 5–10 in early January after 5.2, without other work on the repository. Throughput was welcome, but switching among active tmux panes left engineers “pretty tapped out.”
Ryan explained that the Elixir implementation fits BEAM process supervision and GenServers, which naturally match task orchestration: each ticket becomes a supervised process driven toward completion. He had to learn the ecosystem afterward, but his own language familiarity no longer needed to bias selection of “the right tool for the job.”
Human review is intentionally binary and cheap: mergeable or rework. Rework destroys the entire worktree and PR, starts again from scratch, and asks why the first attempt failed so the missing context can be repaired before rerunning the ticket.
Shawn said the workflow had pushed productivity 5x. The trust discussion centered on compressed evidence, such as a shared agent-produced demonstration that could accompany a PR, rather than “shoulder surfing” an entire coding trajectory; one teammate would not demand another’s complete screen recording.
9. The system learns from its own trajectories, but not every task is solved
Peter Steinberger said the codebase contains about six shared skills. New behavior is first added to an existing skill because common patterns let agents transfer context cheaply, while changing one shared instruction is easier than retraining every human operator’s habits.
Peter said Codex session logs are collected into blob storage and analyzed daily to identify team-wide improvements. PR comments and failed builds receive the same treatment: each is evidence that “at some point the agent was missing context,” which should be distilled back into the repository.
Peter described policy, configuration, coordination, execution, integration, and observability layers; Shawn proposed a “zero layer” that asks whether the workflow itself should change. Agents can update workflows and file follow-up tickets, although the hosts reframed “don’t put the agent in a box” as giving it a box containing everything required within its domain.
The unresolved quadrant is work that is both hard and new. Ryan still struggles to turn a net-new mock into a playable product in one shot, and spends the most synchronous time on white-space design and “the gnarliest refactorings”; both reveal requirements only as the trajectory unfolds.
10. Frontier packages the same control plane for enterprise labor
Peter Steinberger described Frontier as a platform for deploying identifiable, observable, controlled agents across enterprises, integrated with native IAM, security systems, and workplace tools. The Agent SDK is intended to combine models, shell access, the Codex harness, attachments, and containers into a reliable default that builders can customize.
Safety must also reflect enterprise-specific reality. Peter cited the GPT-OSS-Safeguard model and bespoke safety specifications covering exfiltration risks, internal codenames, and company policies; Frontier’s building blocks include steering and the ability to revoke authorization when an agent becomes misaligned.
Peter described two product layers: employees using agents, and IT, GRC, governance, security, or AI-innovation teams overseeing deployment. A control dashboard can descend to individual trajectories, while an internal data agent exposes company ontology—including disputed concepts such as revenue or active users—so agents understand how the business actually operates.
Shawn cited roughly 1 billion tokens of intelligence deployed daily. David Luan reported that Codex had passed 2 million weekly active users and was growing 25% week over week. Shawn characterized the approach as an “on-policy” harness; Peter’s underlying point was that native guardrails—tests, code, and output-aligned checks—can improve model behavior without surrounding it with a restrictive scaffold likely to become obsolete.