Datadog CPO 李严冰: AI's Biggest Deployment Barrier Is Operations

Summary

• 李严冰’s central view is that the biggest obstacle to deploying and scaling AI is how to operate and govern it. Enterprise concerns have shifted from whether AI works to whether it can be trusted and controlled, and then to token costs blowing past budget. The real questions are whether AI delivers correct results, remains controllable and trustworthy, and generates business ROI. Observability is the key to answering them.
• Datadog’s market cap plunged with SaaS stocks early in the year, but the company was among the first to rebound, doubling year to date. Her case: the world’s 10 largest AI companies all need Datadog for observability, and the market consensus is now that AI needs more observability.
• Asked about Datadog’s moat, she points to three layers: operational complexity, data, and proprietary intelligence. Drawing on her experience at Aurora, she says driving involves just 3 controls—accelerate, brake, and steer—while operating large-scale production systems is far more complex than autonomous driving. Datadog has the world’s largest operational dataset, and its proprietary models trained on observability data come close to frontier-model quality at least an order of magnitude lower cost.
• A top AI lab once asked whether it could buy Datadog’s labeled operational data. The request shows that even frontier labs lack depth in specialized domains. She says post-training is now common and lets companies turn their own data into a moat; last year, Datadog acquired RL company Adaptive ML to accelerate its investment in this area.
• While preparing next year’s R&D or AI budget, Datadog found spending growth far exceeded expectations; engineers’ coding-agent costs could be one of its largest budget items. The company can tie token use to outcomes including PR velocity, product stability, feature shipping, and product security—and assess whether spending translates into engineering velocity.
• The enterprise AI risk she worries about isn’t a “world-ending” headline; it’s amateur agents getting the basics wrong. Tools built through vibe coding by non-engineers in sales or HR may use tokens inefficiently or overlook security, creating risks such as leaking business data or exposing API keys. That is a major barrier to wider enterprise AI adoption.

Deep dive

1. Chinese talent is a major force in AI: 57% of NeurIPS authors did their undergraduate studies in China

• 李严冰 cites a new Carnegie Mellon study of NeurIPS authors: 57% received their undergraduate education in China, including authors now based in China, the US, and elsewhere. She also notes that Chinese founders or co-founders have built 41 US companies valued at $1B or more.
• She frames this year’s HYSTA AI conference around what it takes to scale AI in practice: the agenda spans frontier research, enterprise applications, and founders who have achieved commercial success.

2. AI adoption’s three stages of anxiety: from “does it work?” to “where’s the ROI?”

• 李严冰 describes how enterprise concerns have shifted: first, whether AI works at all; soon after, whether it can be trusted and kept secure; and this year, whether token costs—the AI budget—will run far over plan. Companies want correct results, control and trust, and business value—not just “token maxing.”
• The common thread is observability: understanding what AI is doing, how agents relate to one another, and what value they produce. She sees AI observability as essential to production-scale deployment and broad adoption, with Datadog well positioned to benefit.

3. Behind the doubling market cap: products shifting from “for people” to “for people and agents”

• SaaS stocks sold off sharply early in the year, and Datadog was caught in the decline—but it was “among the first to emerge from the trough.” The company’s market cap has doubled year to date. She says the world’s 10 largest AI companies all need Datadog for observability. The market consensus has shifted from “AI will replace observability” to “AI needs more observability” (“AI 需要更多的 observability”).
• Datadog’s product experience is changing, too. Its rich UX, UI, and dashboards were built primarily for people; going forward, both people and agents will use them. The interaction model is moving from web and mobile toward “wherever we work, that’s where the tools should be”: accessible from the CLI for coding, or in Slack for team conversations. In her view, AI is changing how users think about the boundaries of a product.

4. Three layers of defensibility: operational complexity, operational data, and proprietary intelligence

• Asked what makes Datadog indispensable now that models can write code, 李严冰 points to her experience at Aurora. She says Aurora was the first company in the US—and, in her view, remains the only one in the world—to operate commercial autonomous trucks on public highways. Driving requires awareness of the local scene within sight and just 3 controls: accelerate, brake, and steer. By contrast, operating software, hardware, and infrastructure in large-scale production environments—along with complex AI agents—is far more complicated than autonomous driving. Fully automating production operations at scale remains a long way off.
• Coding has its own technical challenges, she says, but it is largely about producing code; operations are much more varied. Datadog is not worried about being replaced by AI, and is itself developing a range of AI products.
• The other layers of its moat, she says, are years of domain expertise, the world’s largest operational dataset, and investment in research and foundation models. General-purpose LLMs can solve many problems, but may be slow and expensive, and are not always best in highly specialized fields. Datadog has trained its own observability frontier model on observability data; its quality comes close to the most advanced models at least an order of magnitude lower cost.
• A top AI lab she visited a few days ago is also a Datadog customer, and the relationship runs both ways. The lab asked, “严冰, we really want to do better in operations. Can you sell us your labeled data?” She says the request shows that even frontier labs can lack depth in specialized domains. She does not expect one leading model to dominate, or a handful of frontier models to provide every model. Post-training is already common and lets companies turn their own data into a moat. Last year, Datadog acquired leading RL company Adaptive ML to accelerate its investment in the area.

5. Datadog’s internal AI ledger: budgets growing faster than expected, real security risks from amateur agents

• When 李严冰 recently put together next year’s R&D or AI budget, she was “stunned”: spending growth was far beyond intuition and expectations. On the product side, customers are using AI-enabled products at increasingly high rates, with usage multiplying. Research investment takes time to pay off, given the high costs of GPUs, AI infrastructure, and training runs.
• On internal efficiency, engineers’ spending on coding agents could now be one of the company’s largest budget items. Datadog can see token usage and where it is happening, then track outcomes such as PR velocity, product stability, feature shipping, and improved product security to assess whether the investment is translating into engineering velocity.
• She is more concerned about AI use in knowledge work. Many people are still vibe coding—for example, sales staff building their own sales tools or HR teams creating tools for their own needs. These agents may be rudimentary, with token use poorly optimized and security insufficiently considered.
• She says the security risks companies are actually weighing are more basic than headlines about “AI killing humans” or “thousands of agents escaping a sandbox to attack other companies.” An agent might accidentally leak business data, or a tool built without adequate AI review and controls might expose an API key. She sees these as important barriers to further enterprise AI adoption.