Why AI Can Write Code But Can’t Keep It Running (Yet)

Summary

• AI coding has crossed from autocomplete into industrial-scale leverage, but the gains accrue disproportionately to engineers who already understand complex systems. Martin Casado describes shipping production code for a data-lake “auto-promotion” capability alone in 10 days versus roughly six months historically. Jad Naous calls that a “10- to 20-fold improvement.” His caveat is sharp: his team is small, not a 100-person sample, and “it’s not like some bad engineers suddenly became good engineers.”
• The bottleneck has moved from writing syntax to specifying architecture and verifying behavior. Jad runs three or four Claude Code sessions, has agents review design and PR documents, parallelizes implementation, then personally recomposes the work “point by point.” He says he uses Windows and WSL, names Opus 4.5, and later describes a technique with Opus 5. Tests, benchmarks, algorithms, responsibility boundaries, and human review matter more as large PRs become cheap to generate.
• Code and operations may occupy fundamentally different learnable distributions. Martin argues that language and code reduce a high-dimensional world into manageable representations, while production systems face rare states, changing workloads, hidden context, and “heavy tails, exceptions, power laws.” Systems history reflects that difficulty: instead of perfect prediction, operators repeatedly choose to “overprovision” resources.
• AI for operations needs signals and investigation, not just raw logs or lagging alerts. Martin describes looking for changes and early warning signals rather than starting with a metric spike and trying to infer root cause. Jad then describes scanning as much as a petabyte of logs per day for new patterns, feeding anomaly signals to an AI agent that inspects code, searches history, forms causal hypotheses, and runs experiments.
• Early operational evidence is promising but remains far short of autonomous cluster management. Martin says the system works internally and in a few customer environments; there were cases where signals the team ignored later became production incidents. Jad believes AI is “very close” to replacing human query-optimization work, yet answers “not yet” when asked whether it can independently run the distributed cluster whose failure takes down the company.
• The clearest commercial wedge is measurable cost reduction, not an abstract promise of AI reliability. Companies were spending 5–20% of infrastructure cost on observability while receiving diminishing marginal reliability; Jad says Grepr cuts telemetry costs by 90% before using extracted signals for troubleshooting. Customers hear the reliability pitch, then return to the immediate budget line: “You said you could save 90% of our Datadog bill? … That’s what I want to buy.”
• AI’s expansion beyond digital domains will depend on constraining environments and finding an economic equilibrium. Jad argues that models can run experiments and work through real-world complexity; Martin remains cautious because self-driving took 20 years and $170 billion of investment to reach “worse unit economics than Uber.” Martin sees decades of value creation, but operational AI must prove that its workflows can tame reality cheaply enough to deploy.

Deep dive

1. Observability’s cost curve broke before AI offered an alternative

• Jad’s company began with a 2023 cost problem: engineering leaders wanted to cut spending, and observability was repeatedly raised after infrastructure as a cost target. Companies spent 5–20% of infrastructure cost on telemetry because what they were really purchasing was reliability and cost predictability.

• That bargain deteriorated as architectures fragmented into smaller, vertically integrated microservices. Telemetry penetrated more layers and grew faster than the functional units being monitored, while “the marginal increase in reliability for every additional dollar spent on observability just leveled off.” Observability increasingly became expensive noise and insurance.

• Jad’s response was to extract signal at ingestion rather than retain everything downstream: place the system in the data stream, reduce telemetry costs by 90%, then feed the surviving signals into an AI layer for diagnosis and remediation. The strategic asset is not merely cheaper storage; it is a real-time stream of operationally meaningful events.

2. AI multiplies systems judgment more than typing speed

• Jad watched coding AI move rapidly from Copilot autocomplete—“look, it autocompleted these tests”—into a workflow that taught capable engineers how to communicate with it. Different models and tools retain different context, select agents differently, and require different prompting habits; without constraints, Claude Code might “launch 20 agents” for an ordinary investigation.

• The performance split widened rather than narrowed. In Jad’s limited experience, good engineers became much better, while weak engineers did not suddenly become strong ones: “The more complexity your brain can grasp, the better you can divide tasks and distribute them.” He explicitly limits the claim to a small team building distributed systems and algorithms, not a universal 100-person test.

• Jad uses Windows, WSL, and Claude Code. He says he is using Opus 4.5, then later refers to a technique with Opus 5. He usually works across three or four concurrent session windows; more sessions become counterproductive because outputs arrive faster than he can preserve context. Persistent command-line agents can serve as implementers and technical leads, question one another, review PRs, and iterate against shared design documents.

• Human work moves upstream and downstream. Jad reviews the PRD, design, algorithms, responsibility boundaries, and dependency graph before telling agents to begin; afterward, he reconstructs the result “like a human, point by point.” The validation cycle now emphasizes tests, execution, and benchmark changes because enormous PRs are easy to produce.

3. A 10-day build does not erase the gap between code and behavior

• Martin’s strongest productivity example is production code for data-lake “auto-promotion”: turning arbitrary fields in semi-structured logs into independently filterable columns, with formats such as Parquet or Iceberg. The work involved metadata handling, commit semantics, and coordinating different authors. Martin says he completed it alone in about 10 days; previously, he estimates, it would have taken about six months. Jad calls that a 10- to 20-fold improvement.

• Martin’s challenge is that generated syntax does not determine runtime behavior. Availability, correctness, consistency, caching, locking, and performance depend on loads, access patterns, schemas, configuration, and live data. Jad’s example is his Gaussian-splat distributed renderer, which he calls “complete crap” under AI generation because millions of splats must be loaded, unloaded, and streamed in a system-dependent way.

• Martin finds models effective when behavior can be turned into measurable feedback. During query optimization, the model initially saw nothing wrong with duplicated conjunctions and produced poor code; once given concrete query data, user workloads, and performance metrics, it generated hypotheses and improved the implementation. Ask instead, “How much memory should I allocate?” and “it has no idea” without real experiments.

4. Effective AI-SRE separates anomaly detection from causal reasoning

• Martin describes an approach different from AI-SRE products that begin with lagging indicators—a metric spike or an alert—and then try to infer root cause through a complex system. He says that is why those systems do not work well: tracing root cause after the fact is problematic. Instead, he describes looking for changes in the environment and early signals, then working backward to see whether they indicate a real problem.

• Jad later describes scanning a petabyte of logs per day for new patterns. A predictive model detects anomalies and sends those signals to an AI agent, which can inspect the code producing a message, look for prior occurrences, test hypotheses, and decide what to do. The goal is not for the agent to predict what will happen, but to give it a signal that something changed.

• Martin reports that the system works internally and in a few customer environments; in some cases, signals the team ignored later became production incidents.

5. The state space may be enormous while the debugging workflow stays tractable

• Martin doubts that operations share code’s manageable distribution. Adding one bit doubles a program’s possible operational states; the count rapidly exceeds “atoms in the universe.” Rare failures, power-law tails, changing workloads, and hidden state—perhaps known only because “a colleague at the cooler” mentioned a deployment—make long-running clusters dependent on human intervention.

• Systems history reinforces his caution. Designers repeatedly sought smart virtual-memory management, congestion control, and adaptive routing, yet production practice defaulted to surplus capacity: “We overprovision all the time.” Swap-to-disk meant the computer was effectively dead; operators bought more RAM rather than perfectly predicting demand.

• Jad agrees that operations is “a much harder problem,” involving external loads, unannounced deployments, sequential failure states, multiple people, and distributed knowledge. Yet he argues that most incidents follow a change, and sufficiently complete telemetry can make that change observable even if the resulting failure state was never present in training.

• Their partial synthesis is that the failure universe may be effectively infinite, but the investigative procedure is lower-dimensional. AI need not memorize every bad state; it can formulate hypotheses, request evidence, write experiments, and simulate irreducible steps. Martin allows that models may reach an “event horizon” where they solve everything short of what must actually be executed.

6. Real-world AI is ultimately an economics problem

• Security raises the difficulty again because the rare state is not merely accidental: an adversary actively searches for behavior the defender cannot anticipate. Jad still applies the same loop—detect a statistical or behavioral perturbation, ask AI for plausible causal structures, then test which structure fits the observed data—rather than expecting one model to intuit the attack from raw logs.

• Martin expects the fastest progress where solution and training distributions overlap: code, images, and other bounded digital domains. Real-world systems will take longer and cost more. His cautionary benchmark is autonomous driving: after Stanford’s DARPA Grand Challenge optimism, deployment took 20 years and $170 billion while producing “worse unit economics than Uber.”

• Jad’s counter is that people can change the environment to fit automation, as with warehouses, rather than forcing AI to master an unconstrained world. Martin agrees that constraining the system changes the economic equilibrium: almost any problem may be solvable with enough computation and money, but only some solutions will support attractive unit economics.

• The buyer already reveals that equilibrium. Jad pitches proactive reliability, yet customers interrupt at the concrete saving: “You said you could save 90% of our Datadog bill?” Coding sells easily because developer productivity maps directly to a major cost center; operational AI remains early, harder to value, and commercially strongest when attached to an immediate budget reduction.