DevDay 2025: Apps SDK, Agent Kit, MCP, Codex and why Prompting is More Important than Ever
Summary
Apps SDK turns ChatGPT from a chatbot attached to other products into the distribution layer in which those products run. OpenAI reported 4 million developers, while Sherwin Wu recalled roughly 800 million ChatGPT weekly active users and described it as around the world’s fifth- or sixth-largest website. The lesson from plugins was that distribution alone is insufficient: developers must “really own and steer the experience,” including UI and brand.
OpenAI adopted MCP as an already-emerging, general protocol. Sherwin credited Anthropic with treating MCP as genuinely open, including OpenAI representation on its steering committee, while the discussion placed adoption around March alongside the Agents SDK and Responses API. The unresolved layer is connector quality: first-party sync connectors enable indexing and reranking, while third-party MCP servers cover the long tail but leave performance dependent on each creator’s API shape.
AgentKit is becoming an end-to-end production platform rather than another agent-demo framework. Agent Builder, the Agents SDK, Connector Registry, ChatKit, and eval tooling combine visual workflow design, deployment, testing, and optimization; Christina Huang’s stage demo built a live DevDay assistant within an eight-minute slot. The deeper opportunity is complex orchestration—human decisions lasting “over the course of weeks,” reusable enterprise templates, and eventual movement both from canvas to code and code back into the canvas.
Agent evaluation remains early, but the optimization unit is shifting from a single answer to an entire trajectory. Christina estimated OpenAI has completed “maybe 10% of the progress that we need”: today’s system can grade long traces, but evaluating a 20-minute task requires decomposing and improving individual steps, often with human review. OpenRouter integration also lets the eval product compare non-OpenAI and open-source models in one place.
Prompting is becoming a durable production asset—effectively “zero-gradient” fine-tuning—rather than disappearing as models improve. OpenAI made a major automated prompt-optimization launch around GPT-5 because each model brings new quirks, and intends to connect prompts, agents, and eval feedback into continuous suggested improvements. Sherwin’s reversal is explicit: people once thought “prompting is going to be dead. No”; it has instead become “more and more entrenched.”
ChatKit reduces front-end work, but identity, inference funding, and the widget ecosystem remain unsettled. Its evergreen iframe and widgets borrow the philosophy of Stripe Elements or Checkout: OpenAI maintains new reasoning and modality UX while developers customize the application. Bring-your-own-key is not supported today, copy-pasted keys are “huge security hazards,” and cheaper inference can still increase total spend because demand rises faster.
Codex is already compressing development cycles, while API reliability is becoming a visible product commitment. Christina described interns using “full YOLO mode,” with broad tasks one-shot roughly 30–40% of the time; the visual Agent Builder began less than two months before launch and “wouldn’t be possible without Codex,” though humans still review its work. At serving scale, OpenAI has surpassed the cited 6 billion tokens per minute and now exposes organization-specific throughput and error SLOs—the path from three to five nines requires “exponentially more work.”
Deep dive
1. ChatGPT becomes the application layer
Sherwin’s strategic frame starts with distribution: OpenAI cannot bring AGI’s benefits to “every corner of the world” alone, making APIs and third-party developers mission-critical. Apps SDK continues the iterative path from plugins to GPTs, now attached to what he recalled as roughly 800 million ChatGPT weekly active users; DevDay also reported 4 million developers.
The host’s sharpest framing was an inversion: websites once added a chatbot in the corner, whereas ChatGPT can now sit above the experience with the application embedded inside it. Canva demonstrated the new shape—recognizably Canva, but conversational and operating within the context of ChatGPT. Christina called it “an experience I’ve never seen.”
Christina framed plugins as a corrective lesson: one piece of feedback was that companies had little control over how their integrations were used and were “really bound by ChatGPT.” Apps SDK therefore gives developers custom UI and room to “really own and steer the experience,” preserving their brands rather than becoming invisible backend functions.
2. MCP is the natural shared protocol candidate, but connectors still trade breadth for quality
OpenAI’s MCP commitment crystallized around March as the Agents SDK and Responses API created a more agentic, tool-calling stack. Sherwin credited Anthropic for treating MCP as a genuinely open protocol: changes feel open, OpenAI’s Nick Cooper sits on its steering committee, and its general, streamlined design made integration straightforward. A host joked, “It’s already solved. Why would you make it?”
First-party “sync connectors” demand more infrastructure because OpenAI stores substantial state, but that state enables indexing, vector search, reranking, and other quality improvements. MCP covers the enormous long tail without requiring OpenAI to build every integration, but then developers are “at the mercy” of whether an individual server’s API shape works well with models.
Sherwin said the Connector Registry should make it easier for companies to manage what their developers can access, configure integrations, and establish private internal links. The host highlighted fragmented authentication—Gmail, Calendar, and Drive may each require authorization—but the intended ecosystem includes OpenAI-built connectors, preferred third parties, and open-ended MCP servers.
Christina’s honest non-answer on workflow portability was: “Unfortunately, I don’t have a great answer today of yes or no.” MCP and OpenAI’s commerce protocol, ACP, showed the value of shared standards, and she and Sherwin would like portability for both agent workflows and stateful Responses-style APIs so developers need not maintain three integrations merely to switch models.
3. AgentKit makes production orchestration visual
Christina described AgentKit as a full set of tools to “build, deploy and optimize agents,” born from customers discovering how much expertise production requires: prompt iteration, eval construction, confidence-building, and deployment. Its components—Agent Builder, Agents SDK, Connector Registry, ChatKit, and evals—work independently but become “stronger together” as one end-to-end system.
The eight-minute DevDay build intentionally showed a simple customer-facing assistant, completed with time remaining before download issues arose, but real customer canvases require zooming far out. Translating those same flows into Agents SDK files produces long code, making visual orchestration especially useful for customer support, document discovery and comparison, enrichment, planning, structured-data Q&A, and internal knowledge assistants.
Human approval is more than a binary gate before an MCP tool runs. Christina said enterprise workflows can involve genuine decision-making “over the course of weeks,” so human-in-the-loop support must eventually represent prolonged state, review, and judgment rather than a single approve-or-reject node.
Agent Builder has two entry points: a playground for modeling and exporting workflows into developers’ own systems, or a managed route that uses natural language, subject-matter experts, datasets, and ChatKit to deploy with easier defaults. Future ambitions include importing code, running code inside the builder, standalone workflow runs, voice, and richer logical nodes.
4. Agent evaluation starts with traces, not a single score
Christina’s deliberately sober status report was that agent evals have made “maybe 10% of the progress that we need.” The current advance is ingesting long Agents SDK traces and grading the overall run; multimodal evaluation and more granular diagnostics remain unfinished.
A 20-minute agent task breaks the old prompt-completion evaluation model. OpenAI wants to score the overall trajectory, isolate individual trace segments, and optimize each segment separately, often involving a human in the loop. The hosts expect LLM-as-judge rubrics to become a central mechanism, but the product is still at “step one.”
Evals became multi-model on launch day: developers can bring an OpenRouter setup, write evals with OpenAI’s Datasets tool, and compare multiple providers, including open-source models served through Together. That is not a workflow protocol, but it acknowledges the operating reality that agent builders increasingly evaluate several models in one system.
5. Prompting becomes “zero-gradient” model customization
Automated prompt optimization received a major push around GPT-5 because every new model brings quirks that otherwise demand another extensive prompting guide. Christina’s desired loop connects an agent’s evals to continuous prompt and workflow improvements, turning the platform into a source of “suggested improvements to the things that you’re building.”
Sherwin called prompt optimization an unusually active product and research area, pointing to the Databricks team’s work—GEPA—while stressing that techniques will keep changing. The larger reversal matters more: two years earlier, many expected prompting to disappear; instead, “if anything, it has become more and more entrenched.”
Sherwin relayed the characterization of this approach as “zero-gradient fine-tuning”: enough prompt adaptation can yield behavior resembling a different model without updating weights. He favored that route because serving many fine-tuned snapshots is operationally difficult even with LoRA; prompt hill-climbing offers gains without forcing developers or providers to manage that infrastructure.
Tinker represents a complementary low-level path. Sherwin said John Schulman had pursued its shape for years—a research-oriented fine-tuning abstraction where users need only “these 4 functions” rather than the underlying infrastructure—and welcomed its release because it empowers builders and researchers even though it comes from Thinking Machines.
6. ChatKit lowers deployment friction while economics stay unresolved
AgentKit spans both internal automation and public applications. Companies repeatedly built internal systems for prompt and template management, while customer support and products such as Ramp demand external-facing quality; Christina said customer support at help.com is already powered by AgentKit alongside various internal use cases.
ChatKit’s iframe is intentionally evergreen and opinionated: when OpenAI adds a model with different reasoning behavior or new modalities, embedded applications can inherit it without rebuilding their front ends. Christina compared the philosophy to Stripe Elements and Checkout—common infrastructure stays maintained centrally, while developers focus on the parts unique to their agents.
The team said ChatKit itself is an embeddable iframe and discussed trade-offs around open-sourcing it; its evergreen, opinionated design is the point. Christina and Sherwin pointed developers to ChatKit Studio, Widget Studio, demo apps, and ChatKit.world, while the host asked whether widgets could become a broader ecosystem. The team emphasized “consumer grade” polish, responsive design, smooth animation, and AI-generated widgets.
Bring-your-own-key does not work out of the box. Sherwin acknowledged longstanding demand from indie developers who cannot fund unrestricted public inference, while Christina warned that copy-pasted-key workarounds are “huge security hazards.” Lower prices help only partly: “as we make it cheaper, the demand for that goes up even more,” leaving responsibility and spending controls unresolved.
Broader ChatGPT identity is exploratory, not a declared identity-provider strategy. Sherwin said Apple already carries some identity into Siri, routing according to free versus Plus or Pro subscription status, including GPT-5 access; Kakao offers another sign-in integration. Asked about ChatGPT as an OpenID provider, his direct answer was “no plans right now.”
7. Codex accelerates building; reliability determines whether it compounds
Christina’s Codex lesson came from younger, more AI-native developers: push past the habit of treating the model like an untrusted intern. Interns gave it whole features in “full YOLO mode”; although it failed sometimes, roughly “30% or 40% of the time it just one-shots it,” forcing her to readjust her priors about useful task size.
The hosts’ pushback—worth keeping—was that vibe-coded pull requests can transfer the workload to reviewers. Christina answered that Codex PR reviews are themselves highly valued by her team, but did not claim full autonomy: output remains “checked and edited,” with the model serving as a powerful starting point rather than approving its own work.
The concrete productivity evidence was the visual Agent Builder, started less than two months before DevDay and, in Christina’s view, impossible on that timeline without Codex. Team members also launch perhaps five Codex tasks during a 30-minute bus ride, using the results to map files and regain context even when they discard the generated PR and write the implementation themselves.
Christina’s closing launch was the organization-scoped service-health dashboard, which tracks responses, token velocity, TPM, throughput, and status codes as a “real-time personal SLO.” After a December outage lasting roughly 3–4 hours, OpenAI discussed progressing through three, three-and-a-half, and four nines toward five; each extra nine takes “exponentially more work.” The cited 6 billion tokens per minute had already been surpassed.