Cohere's Chief AI Officer, Joelle Pineau: Why Scaling Laws Will Continue & Future of Synthetic Data

Summary

• Scaling laws hold — don’t bet against them. Pineau, who was at Meta from 2017 to 2025 working on fundamental AI research and has now joined Cohere, says the scaling laws “have been remarkably robust… lots of people have bet against scaling laws in the past” and she wouldn’t. But compute and data drive only linear-ish progress; the nonlinear jumps come from algorithms (transformer, Adam, reasoning) — which is precisely the hardest place for an investor “to know where to put your chips.”
• AI coding today is image generation in 2015. Pineau’s analogy is that image generation in 2015 had bad resolution and composition, while coding now produces a lot of bad code, much of it thrown away; she thinks the quality will be excellent in another ten years. The consequence: when generation is abundant, value migrates to selection — “curation, verification — this is work that doesn’t go away,” a “chief curation artist” sitting atop machine-generated volume.
• The 10x reframe that shook Stebbings’ thesis: Pineau counters David Khan’s “can it replace your bottom 5%” barometer with “can most of your employees do 10x the work” — achievable “in the next couple of years,” some work seeing 100x. Stebbings questioned whether this undercuts the VC assumption that returns require human labor budgets converting to AI spend; efficiency gains, not headcount replacement, may be the actual mechanism.
• Data is the rising cost line. Easy labeling (“this is a cat, this is a dog”) is over; what’s expensive now is specialized-domain talent and creative people building synthetic environments for agent training. On the labeling firms (Mercor, Surge, likely Turing): “some of these firms may not be around in five years, but this notion of having humans guide and train the behavior of AI systems — that is here to stay.”
• You can’t just buy the Galacticos. Teams need vision, execution muscle, and social glue — “I don’t think it becomes that productive to put a bunch of AI superstars all together in a room.” Pressed on Andrew Tull, Daniel Gross, and Alex Wang, you do need “a few of these uber-talents… if you can afford it,” but on multi-billion price tags, only “time will tell.”
• Agents open a new security front with much still unknown: the agent parallel to LLM hallucination is impersonation — agents acting for entities “they don’t legitimately represent,” including infiltrating banking systems. “There’s a lot we don’t know yet in terms of the vulnerability of these systems.”
• It’s “a bubble with bigger variance” — bigger upswings and bigger downswings, a great investment only if you tolerate risk. Her contrarian streak runs both ways: she admits she was “quite wrong” doubting neural networks, and dismisses existential-risk discourse as lacking scientific rigor — the one buzzword she’d ban.

Deep dive

1. RL is fundamental — and terribly inefficient

• Pineau has worked on reinforcement learning for over 20 years and watched it become widely discussed since reasoning models and agents — her lesson from Meta being “just how long it takes to prove out a hypothesis.” Responding to Andre’s “RL is terrible” line from a prior episode: “less terrible than 20 years ago.” She remains “super bullish on RL” as a concept — training through rewards “is so fundamental, it’s not going away” — but “thinking that just RL out of the box is going to give us AGI, that part a lot less so.”
• Why RL is inefficient, spelled out: sequential decision-making means errors compound down every branch of the action tree — “like finding a needle in a haystack” — and the model must act to learn, requiring simulators and synthetic data that are expensive to build in variety.
• Costs have fallen where reward functions are writable — AlphaGo, mathematics, well-defined reasoning, games. But “RL to shape the behavior of models to get them to be social creatures — we have no idea how to do” that; like parenting, “the number of times you can repeat the same thing and still they do something else… you don’t know how to write that out mathematically.”

2. Scaling laws robust; algorithms are the nonlinear lever

• Her decomposition of progress: compute and data are linear-ish — more compute, bigger models, better performance — while algorithms are nonlinear, citing the transformer, the Adam optimizer, and reasoning-in-the-loop as paradigm changes. The catch: algorithmic ideas “may take a long time to prove themselves out” — thousands of papers sit untried at the right scale with the right data and hyperparameters.
• On whether scaling continues: “The scaling laws have been remarkably robust… they don’t work alone, we also need these algorithmic innovations, but most of the time I wouldn’t bet against it.”
• Algorithms are also the hardest axis to buy: compute and data are purchasable, but algorithmic direction-picking “is a little bit like reinforcement learning — you don’t know until you get there whether it was the right one.” Her verdict: “the most interesting one, the most frustrating one, and the most difficult one certainly from an investor’s point of view.”
• The economics’ biggest structural problem is unpredictability: “Everyone wants to know when are we going to hit the breakthrough… how many GPUs do I actually need… what’s the return I can expect” — and this technology simply doesn’t offer that, forcing risk into every data-center, workforce, and data decision.

3. 10x the workforce, don’t replace it — Stebbings’ thesis wobbles

• Against David Khan’s barometer (can AI replace your bottom 5% in any function), Pineau counters: “can most of your employees do 10x the amount of work with AI” — flatly replacing workers is “actually pretty unrealistic” because human and AI abilities are complementary. When Stebbings called 10x the more intimidating ask: “I don’t think that’s unrealistic at all… in the next couple of years.”
• Her concrete case: machine translation of multi-page documents went from hours to seconds — humans still ask the question, verify, and shape the task, but once specified, “you press the button and you’ve got an answer in seconds for something that used to take weeks and months.”
• Stebbings’ live re-evaluation, worth keeping: his VC thesis assumed money is made when human labor budgets convert to AI spend — Pineau’s answer implies the gain shows up as efficiency instead, “some work will be harder… whereas other work you’ll see 100x.” The dividing line: ambiguity in task specification is what machines can’t handle; precisely specifiable tasks automate first.
• On Orman’s claim that the young use AI as an OS and the old as next-gen Google, a mild dissent: “I see a lot of people using it as a tool more than as a companion” — a Swiss knife in their work life.

4. Cohere’s on-prem wedge and the enterprise reality check

• Cohere (she joined less than a month ago) develops models enterprises run on-premises. Against Stebbings’ framing that inference could be 95% of the market, Pineau says “the company” has to worry about training while the enterprise doesn’t have to worry about inference costs — which she argues creates structural pressure for efficient models. Stebbings’ pushback: if the client pays for inference, where’s your incentive for efficiency? Her answer: “We’re still in the early days of AI adoption in enterprise, so what’s good for the client is good for us.”
• Why she left pure research: “when you need to sell AI to a business, you get a real signal of what works, what doesn’t work” — academic benchmarks give some signal, “but it’s not the same as getting this to do productive work.” That real-world feedback then steers the search through the idea space.
• The biggest enterprise blocker is integration with decades of accumulated information systems, which is why on-prem data confidentiality is Cohere’s focus — plus a cultural one: “many people feel they have to get it right the first time,” when the technology’s maturity calls for “a spirit of exploration and curiosity.”

5. Data is the rising cost line — from labeling to crafting environments

• Given a hypothetical $10bn, she’d balance talent and compute (“too much talent and not enough compute, you’re wasting your time”) and spend “a good chunk” on data, which “we often underestimate” — Cohere itself is “reasonably well resourced” on compute.
• Why data is getting expensive: the cat/dog labeling era is over; enterprise AI needs people with deep tool and business-logic understanding, and agent training needs “pretty creative folks who are going to build you synthetic simulators” — the robot-simulator playbook applied to work processes.
• On the labeling firms (likely Mercor, likely Surge, and likely Turing) and whether theirs is an enduring market: “some of these firms may not be around in five years, but this notion of having humans guide and train the behavior of AI systems — that is here to stay”; what changes is what information humans must supply. The bigger trend she sees is “moving from just labeling data to crafting environments.”

6. Synthetic data collapses only where diversity dies — and code is savable

• Whether synthetic data degrades models “really depends how you’re generating” it. Where diversity can’t be injected — LLMs talking to each other — you get distribution collapse, her analogy told in full: “take a bunch of people, put them on an island and let them reproduce… at some point the genetic diversity keeps shrinking.” Closed worlds like chess and Go generate tons of synthetic data, though not endlessly, because the worlds are closed.
• Coding sits in between — and on the right side: “I can take a couple of repositories, mix and match, apply an LLM to transform it” — structure plus injectable diversity means heavy synthetic training without collapse.
• Her signature frame for code quality fears: code generation now is image generation in 2015 — bad resolution, bad composition, then huge progress through ~2022. “Yes, there’s a lot of bad code getting generated… but wait another 10 years and the quality of the code produced is going to be excellent.”
• Carrying the analogy forward, the endgame is volume: what matters becomes “picking the quality out of the volume” — a selection mechanism, an editorial design choice. Stebbings: “so it’s a chief curation artist.” Pineau: “Yes… curation, verification — this is work that doesn’t go away,” and when Stebbings noted that removes the human partnership: “Well, that’s your 10x productivity improvement there” — though intent and critique remain human, and team composition “changes significantly” once designers go straight from idea to digital world.

7. Agent security: impersonation is the new hallucination

• A new front is opening with agents where “there’s a lot we don’t know yet in terms of the vulnerability of these systems.” LLM risk vectors (prompt injection, jailbreaking) are becoming understood through red-teaming; agents haven’t had that scrutiny. The parallel she draws: as LLMs hallucinate, agents impersonate — “taking actions on behalf of entities which they don’t legitimately represent, whether it’s infiltrating banking systems.” Security stays “a cat-and-mouse game”; the blunt mitigation is running agents cut off from the web, trading information access for risk.
• On who arbitrates agent verification: “governments can be good for defining standards on which we all agree; companies are much better at building the solutions at scale.” Her model is aviation — government-defined norms produced an incredible 50-year safety record — but regulation “shouldn’t be ahead of the technology… that would be the wrong order of things.”
• On sovereign AI: models built outside the US and China are “healthy in terms of diversity of thought,” but Cohere’s “vision isn’t to be a Canadian company” — it’s global, with Canadian headquarters conferring sensitivity that one-size-fits-all fails: “you go to Japan, you go to Korea and they do want models that work well in their language.”

8. Galacticos: you need a few, not a roster

• Her team recipe: one to three people with vision, people with “amazing execution muscle” who “don’t care that it’s their idea,” and social glue — “where I’ve seen it fail is to have just one type of person inside the team… a bunch of AI superstars all together in a room without the execution machine, without the social glue.” Plus focus: clarity on the north star, “even if over time that needs to change.”
• Stebbings’ blunt pushback — if you don’t need to buy the Galacticos, why is the group assembling Andrew Tull, Daniel Gross, and Alex Wang? Her concession: “You do need a few of these uber-talents… there’s a relatively short number of people who just understand this technology very deeply… if you can afford it, you should get some” — but not the whole roster. On whether $3bn-person price tags are justified: “Time will tell. I don’t think it’s necessarily needed to go at that scale.”
• On the premium for having “seen success” (likely Mira Murati raising $2bn out of $10bn): track record carries both the core recipe and “the achievement of having put together amazing teams building world-class models — there’s a lot of subtlety to that.”
• Her lesson from Zach: “he does not coast… just the depth of the questions that he’d ask” when learning AI — “you can have the most amazing team, but as a leader you need to go deep and understand the work.”

9. A bubble with bigger variance — and the convictions she’s dropped

• On good bubble vs bad bubble: “I think about it as a bubble with bigger variance — the upswing is going to be bigger and there’s going to be big downswings as well. As long as people have a tolerance to risk, AI is a great investment.” Evals are “unit tests for the performance of your system” — good indicators, but not targets to optimize for: “none of our clients ask, are you able to win the math olympiad with this model?”
• Her most instructive change of mind: “I used to be quite skeptical that neural networks were necessarily the ultimate solution to machine learning” — every prior scale jump saw something better emerge, like SVMs in the early 2000s — “and I seem to be quite wrong on this one.” Her epistemics: “I have weak conviction but very strong respect for the scientific method.”
• Where she thinks others are wrong: “I don’t have a lot of patience as a scientist for people predicting the extremist scenarios… the AI-becomes-our-overlord kind of scenario… I think you lack scientific rigor to analyze these kinds of scenario.” The buzzword she’d ban: “existential risk” — “it’s not out of fear that we make our best work.”
• Two closing calls: demand for efficient small models is real — RoBERTa, a 2019 small model, was getting 20 million downloads a month during the large-language-model frenzy, and she wants to see what runs on one or two GPUs — and the industry’s turn toward closed systems/access is “a deep mistake… ideas will circulate, and people are circulating.” If investing today: healthcare and scientific discovery, with “real tangible progress within 5 years.”