Inside OpenAI Enterprise: Forward Deployed Engineering, GPT-5, and More | BG2 Guest Interview

Summary

• OpenAI’s enterprise business predates ChatGPT — “the original product for OpenAI actually was not ChatGPT. It was a B2B product. It was the API” — and the company now runs a Palantir-style forward deployed engineer model, embedding at customers like T-Mobile (voice support handled live by OpenAI models), Amgen (a top GPT-5 customer for drug-development paperwork), and Los Alamos (o3 physically installed on the air-gapped Venado supercomputer, shared with Lawrence Livermore and Sandia).
• Against the MIT “95% of AI deployments don’t work” headline, Olivier’s pattern from ~200 enterprise deployments: winners have top-down buy-in plus a bottom-up “tiger team,” and evals first — “whenever the customer fails to come up with good evals, it’s a moving target.” Most enterprise knowledge lives “in people’s heads,” not in the SOPs, and hill-climbing from 46% to 99% is “art, sometimes more than science.”
• The episode’s sharpest frame: “physical autonomy is ahead of digital autonomy in 2025” despite a higher safety bar — because self-driving had 10-15 years plus roads and stoplights as scaffolding, while “AI agents are just kind of dropped in the middle of nowhere.” Agents date only to the o1-preview reasoning paradigm; “the slope I think is incredibly steep,” and by revenue they may have already crossed Waymo.
• GPT-5’s differentiator is “the craft — the style, the tone, the behavior,” not saturated benchmarks: a host said he thought one eval showed hallucinations “basically went to zero,” and the core unresolved trade-off is reasoning tokens vs latency — GPT-5 Pro “one-shots” unsolved problems but takes 10 minutes. The monkey’s-paw lesson: instruction following got so literal that legacy “be concise” prompts broke outputs.
• Reinforcement fine-tuning (RFT) — a term Olivier said OpenAI made up — is “an order of magnitude more powerful” than SFT and shifts the pitch from customization to building “a best-in-the-world model” on proprietary data (Rogo in financial services; Accordance hitting SOTA on TaxBench). Olivier’s call: for frontier capability, “RFT will pretty much become the norm.”
• The long/short game: Sherwin is short “the entire category of tooling around AI products” — evals products, frameworks, vector stores, and now RL-environment startups — because the stack churns too fast for abstractions to survive a model generation. Olivier is short memorization-based education (“knowledge tokens”) and long healthcare — “probably the industry that will benefit the most from AI.”
• Softer signals worth logging: vastly more software engineering ahead even if engineer headcount is ambiguous (“there is a massive software shortage in the world” — OpenAI PMs now ship coded prototypes instead of PRDs); Codex CLI + GPT-5 usage is ripping; and “the blip” (the board coup) left OpenAI antifragile — “a thicker skin and an ability to recover way quicker.”

Deep dive

1. The API came first — enterprise is how OpenAI “distributes AGI”

• Sherwin’s reminder: when he joined three years ago, the API “was actually the only product that we had” — ChatGPT came later. ChatGPT is now roughly “the fifth largest website in the world,” but the platform — API, a government/public-sector product, and an emerging direct enterprise line — is framed as the fuller expression of the mission, reaching use cases and end users inside businesses that a consumer app can’t.
• Olivier’s version: B2B is core because “there are a large category of use cases that only go through B2B” — a world with 10x more medicines a year, better education and civil service — “those are the businesses who are actually making stuff happen in the real world.” He names it explicitly as the Palantir thesis.

2. T-Mobile: voice support on OpenAI models, plus Palantir-style FDEs

• After roughly a year embedded, T-Mobile support calls in-app are “actually handled by OpenAI models behind the scenes” — text and voice — sounding “supernatural, human-sounding latency, quality-wise,” powered by the real-time API that Olivier said had gone GA around the prior week.
• What sits above the models is the real work: forward deployed engineers (“we borrow the term from Palantir”) orchestrating models against CRMs and enterprise tools that “do not even have APIs” — standing up gateways, then defining golden eval sets. Audio evals are “extra hard to grade”: on a five-minute call transfer, “how do you actually know that the right thing happened?”
• The flywheel Sherwin flags: learnings from T-Mobile fed directly into the new real-time GA model snapshot — deep enterprise embeds double as model R&D.

3. Amgen and Los Alamos: drug paperwork and an air-gapped o3

• Amgen — “a top customer of GPT-5” — splits healthcare needs into two buckets: pure R&D over massive data, and the underrated one, admin and document authoring: once R&D “has essentially locked the recipe of a medication,” regulatory submission and review is “a ton of work.” Apoorv’s gloss: hundreds of millions of lives if drugs ship faster.
• Los Alamos got a custom on-prem deployment of o3 onto the air-gapped Venado supercomputer — “we literally had to bring the weights of the model physically into their supercomputer,” in a facility where cell phones are banned. OpenAI has limited visibility, but feedback shows it speeding experiments, crunching data notebooks, and — new with reasoning models — serving as a “thought partner” on experiment design, “something that we couldn’t really say about our older models.” It’s shared with Lawrence Livermore and Sandia.

4. Why 95% fail: tiger teams, evals first, then climb the hill

• Against the MIT report that “shook the markets for a couple of days,” Olivier’s pattern-match from “a couple of hundreds” of deployments: top-down buy-in plus a bottom-up tiger team mixing technical skill and institutional knowledge — because even in customer support, “the vast majority of the knowledge is in people’s heads,” not the documented SOPs.
• Second: evals first — “whenever the customer fails to come up with good evals, it’s a moving target.” Olivier’s addition: evals must come bottom-up from actual operators; a top-down eval mandate doesn’t work.
• Third, the climb: “you have your evals, the goal is to get to 99%. You start at, like, 46” — and getting there is “art, sometimes more than science,” occasionally requiring OpenAI to fine-tune the model itself when it hits clear limitations.

5. The autonomy paradox: Waymo works, your ticket-booking agent doesn’t

• Apoorv’s puzzle: the bar for physical safety exceeds human capability — lives at stake — yet “physical autonomy is ahead of digital autonomy in 2025.” Why would the easier-sounding problem be harder?
• Sherwin’s two-part answer: timelines — self-driving took 10-15 years including a trough of disillusionment, while agents really date to the o1-preview reasoning paradigm “less than a year” old, “but the slope I think is incredibly steep”; and scaffolding — cars got roads, stoplights, and traffic laws, whereas “AI agents are just kind of dropped in the middle of nowhere.”
• The tradeable inference: failed enterprise deployments “likely don’t have the scaffolding” — much of FDE work is building connectors and organizing data so models have something standardized to interact with. And the crossover may already have happened: “I would not be surprised if AI agents products are making more revenue than Waymo at this point.”

6. GPT-5: craft over benchmarks, and the thinking-time trade-off

• Olivier’s framing: benchmarks like what sounds like “SWE-bench” run high, but “equally important and impactful was the craft — the style, the tone, the behavior of the model.” It’s the first large release built through months of customer feedback loops, and often the ask isn’t more intelligence but “a model that is more likely to say no when it doesn’t know.”
• The hardest trade-off, still being iterated per Sherwin: reasoning tokens versus latency. GPT-5 Pro “one-shots” unsolved problems no other model could handle (the Andrej tweet Sam retweeted) — “but the trade-off here is you’re waiting for 10 minutes,” and a business user “might be more okay with the substandard answer in no wait at all.” Thinking should get more dynamic; the next snapshot also targets code-quality idioms.
• On robustness: a host said he thought an eval showed hallucinations “basically went to zero” — “it’s not perfect, there’s still a lot of work to be done,” but reasoning makes the model likelier to decline than confabulate.

7. The monkey’s paw of instruction following

• Developers asked for better instruction following and got a model that follows “almost to a T”: the “too concise” complaint in testing traced to legacy prompts with ten lines of “be concise” begging — GPT-5 obeyed literally and returned one terse sentence. Strip the scar tissue and it behaves. “Prompt engineering is still very, very important.”
• Investor-relevant aside from Apoorv: portfolio company Expo (cybersecurity) saw such a jump on GPT-5 that “they’re going to need a new eval soon” — met with “it’s all about evals.”

8. Speech-to-speech beats the stitch — but the logic layer isn’t unified

• Why real-time API over the “stitch model” (speech-to-text → thinking → text-to-speech): extra hops mean slowness, and you “lose the emotion” — pauses and tone that matter on real phone calls. Apoorv’s pushback: function-calling logic is written in text, so voice means “a slightly different architecture” — unifying orchestration across modalities is active work, and many customers still stitch.
• Voice models excel at “easy casual conversation — talk to your coach, your therapist” but must be taught economically valuable behavior: know what an SSN is, and “if one digit is fuzzy, it actually has to repeat versus guess.”
• Sherwin’s wonder, worth keeping: “you’re literally taking the audio bits from someone speaking… and it’s generating audio bits back. To me, it’s actually crazy that this works at all” — let alone handling accents, tone, and a live support call.

9. RFT: turn the RL crank on your own data

• Reinforcement fine-tuning — a term Olivier said OpenAI made up (“it wasn’t a real thing until we announced it”), announced late last year, likely during the “12 Days of Christmas,” and since GA’d — brings RL into fine-tuning: “way more complex, way more finicky, but an order of magnitude more powerful” than SFT. It needs gradable tasks and an objective grader, not prompt-completion pairs.
• The reframe: not customizing a model, but using proprietary data to “create a best-in-the-world model for something you care about for your business.” Proof points: Rogo (financial services, ex-DeepMind hires) on financial-document tasks; Accordance, which Olivier said was getting what he thought were SOTA results on TaxBench for CPA-style work.
• Apoorv’s call: base models are now good enough that fine-tuning for behavior steering is unnecessary — but for pushing frontier capability, “my hunch is that RFT will pretty much become the norm,” with custom environments. And the data required is bottom-up: intricate task knowledge from domain experts.

10. Longs, shorts, coding futures, and the blip

• Sherwin’s long is esports — stadium-scale in Asia, youth attention migrating there, “undervalued” after the COVID spike. His “spicy” short: “the entire category of tooling around AI products” — evals products, frameworks, vector stores two years ago; RL-environment startups now — because “today’s really hot framework might just not get used in the next generation of models.” Olivier’s pair: short any education emphasizing memorization (“knowledge tokens” LLMs already have), long healthcare — “probably the industry that will benefit the most from AI,” a perfect storm of structured data, admin-heavy culture, and R&D-friendly companies.
• On whether 40-50M software engineers grows in 10 years: unambiguously more software engineering — the Reddit story of a man who used ChatGPT to build custom tools so his non-verbal brother could browse the internet — even if titles are ambiguous. Apoorv says “there is a massive software shortage in the world”; Olivier says OpenAI PMs now ship coded prototypes in hours instead of PRDs. Personal tool picks: two votes for Granola, and Codex CLI + GPT-5 — “I feel so mind-melded with the model.”
• Thorn and rose: “the blip” (the board coup) was brutal on the day but left OpenAI antifragile — “a thicker skin and an ability to recover way quicker” — alongside a 3-4-hour total outage in November or December of the prior year that proved the API is “almost like a utility.” Roses: shipping GPT-5 at massive token scale with no outages, and dev day November 2023 — Sherwin’s Waymo ride home after the live demos landed.
• Both say they think they are AGI-pilled (“I think I’m AGI-pilled.” — “You’re definitely AGI-pilled.”): Olivier’s moments were realizing in 2023 he’d “never need to code manually ever again” and voice understanding his French accent; Sherwin’s were seeing GPT-4 internally in September 2022 pre-ChatGPT, and deep research — “I’ll throw something at the model that I was like, there’s no way this thing will be able to get. And then it just knocks it out of the park.”