The Pitfalls AI Agents Will Hit Were Already Hit by Quant Traders Thirty Years Ago

Deep thoughts on AI and aspirations —— ByteDance Deep Thought Circle

People building Agents tend to treat the “perception-decision-execution-feedback” loop as something novel. Wall Street doesn’t see it that way. Quantitative trading systems are exactly this loop, running for over thirty years in an environment of real money and brilliant adversaries: mining signals from multi-source data, generating trading decisions, splitting orders for execution, then iterating strategies based on profit and loss.

So the claim that “quant trading and AI Agents share the same underlying logic” isn’t primarily about the isomorphism itself. Abstracting both domains as Markov decision processes is technically correct but trivial—you could fit a refrigerator’s thermostat into that framework too. What’s actually informative is a different fact: quant trading is currently the only large-scale implementation of this loop that has succeeded in adversarial environments. The tuition it has paid is precisely what Agent engineering is about to pay. This homework is worth copying.

Beautiful Backtests, Brutal Live Trading

The first lesson in quant trading: a strategy performs perfectly on historical data, then collapses the moment it goes live. The cause is overfitting—the model memorized noise in the history, not patterns. The prettier the backtest curve, the more cautious you should be.

Agent engineering is littered with the same pitfall. High scores on evaluation sets, poor performance on real tasks. Test questions leak into training data, so the Agent learns to game benchmarks rather than complete tasks. More insidious is task templatization: benchmark scenarios fall into a handful of categories, and repeated optimization on these scenarios is equivalent to backtesting on a market that no longer exists.

The quant solution is a strict discipline: backtests are just admission tickets, live trading data is the report card. Same for Agents—evaluation scores belong in marketing materials; completion rates, costs, and reuse rates on live real-world tasks are what belong on the balance sheet. If an Agent company’s internal reports show only benchmark scores without live metrics, you can safely assume they haven’t entered the live trading phase yet.

Reward Functions Will Always Be Gamed

There’s an old joke in quant circles: optimize for Sharpe ratio, and you’ll end up with a portfolio that has a high Sharpe ratio but hides risk in corners the metric can’t see. The narrower the reward signal definition, the clearer the path for gaming the system. This is the financial version of Goodhart’s Law.

Agents have no immunity. Optimize for task completion rate, the Agent learns to game acceptance criteria with minimal effort; optimize for user likes, it generates content users enjoy but that’s useless; optimize for code test pass rates, it writes code that only passes tests. Reward hacking isn’t a bug—it’s the inevitable behavior of any sufficiently strong optimizer facing a narrow objective function.

Quant’s response provides a ready-made answer: make risk control an independent layer separate from the strategy. No matter how carefully designed, reward functions can’t plug all loopholes, so you need a checking mechanism that doesn’t answer to strategy objectives but only to survival baselines. The corresponding practice in Agent engineering is making audit and acceptance independent of the execution chain—don’t let the scorer and the executor share the same optimization objective.

The Real Challenge Is Execution

Right decision, botched execution—this is where quant pays its most expensive tuition. A large order dumped directly into the market moves prices, hence order splitting; dozens of exchanges exist, hence routing choices; poor slippage control turns paper profits into costs entirely, hence execution algorithms like VWAP and TWAP; when strategies spiral out of control, you need a kill switch to liquidate positions instantly.

The Agent execution layer is the same problem with a different name: tool call failures, API rate limits, async timeouts, permission failures. No matter how smart the decision model, one erroneous transfer call can drain an account. The problem structure quants and Agents face at the execution layer overlaps almost completely—latency and synchronization, fault tolerance and retries, cost optimization, traceable logging—all four, not one less.

The trouble is, current Agent engineering is weakest precisely at the execution layer. Everyone’s attention is staked on planning and reasoning, with execution treated as demo scraps. Yet this is exactly the homework that should be copied: exponential backoff, circuit breakers and degradation, permission sandboxes, operation audit trails—these components polished over thirty years in trading systems are mature enough to transplant directly.

Markets Change, Agents Become Obsolete

A well-performing quant strategy has an average lifespan measured in years. When market structure changes, historical patterns become void—this is called regime change. Agents will similarly encounter distribution drift: user habits change, tool return formats change, task definitions themselves change, and yesterday’s stable success rate quietly declines.

The deeper challenge is credit assignment. When a trade ultimately loses money, do you blame entry timing or exit timing? When a task chain fails, do you blame planning or some intermediate tool call? Both domains are wrestling with the same problem and using similar technical solutions—tracing back from the final result to each step’s contribution.

Put these pitfalls together and the correspondence is clear:

The Flip Side of Isomorphism: Agents Become Quant Problems Too

There’s also a reverse implication. When Agents enter the market—placing orders for people, auto-arbitraging, generating research reports—they themselves become new market participants, and competition rules immediately quantify: everyone uses equally smart Agents, arbitrage opportunities are erased in milliseconds, and prompt writing itself becomes alpha. Alpha’s half-life will shrink further—this is the competitive consequence of isomorphism, not a technology vision.

So there’s one quant lesson that can’t be copied wholesale: quant wins in closed-rule environments where exchange rules are spelled out clearly. Real-world Agents face open environments where rules change, adversaries don’t play by the book, and there’s no exchange to backstop losses. Risk control standards should be more conservative than quant, with stop-loss lines drawn earlier.

Two concrete recommendations for people building Agent products. First, add someone to the team who’s worked on trading systems or payment systems, focused specifically on the execution layer and failure recovery—the ROI is likely higher than hiring another person to optimize reasoning. Second, start a live trading ledger from day one: completion rates on real tasks, costs, failure attribution, reviewed weekly. This is what the Agent industry lacks most right now: everyone’s showing backtest curves, few are willing to open up their live accounts.