🔬 Google's AI Scientist Started as an Attempt to Automate Kaggle — John Platt, Google Fellow

Summary

• Platt says ERA became practical only after recent Gemini improvements. He says it “would have been impossible under Gemini 2.0”; after experiments began working, progress across major and half versions was striking. He recalls Gemini 2.5 writing boosted-decision-tree code immediately and tells scientists who disliked 2.0 that it was “an eternity ago.”
• ERA’s core insight is that many scientific problems can become “scorable tasks”: code that maximizes a defined score. Its Monte Carlo research loop mutates notebooks using Gemini, which can read papers, propose code, and exploit useful model-building “gradients.” Random code mutation had largely failed because most mutations are harmful; an LLM mutation operator can make more informed changes.
• Humans still have to define and police the objective. Scoring functions can contain loopholes, and both agents and people will reward-hack them. Platt cites Kaggle competitors finding a half-pixel labeling error in a contrails competition. He says researchers need both creativity and rigor, with humans checking whether an optimized result is genuinely descriptive rather than merely predictive.
• Persistent contrails contribute about 1% of anthropogenic warming and may be avoidable at relatively low cost. Ice-supersaturated regions can amplify emitted water, ice, or soot by roughly 10,000 to 1; avoiding them may require only dropping two flight levels. ERA also found a simple counterfactual model that solved a reflected-sunlight problem Platt’s team had struggled with for two years.
• Platt thinks commercially relevant fusion could be “three years away, not 30,” but presents that as a probability, not a certainty. The Lawson criterion requires density, temperature, and confinement time to clear a threshold, and every approach has an Achilles heel. He also stresses that electrification and batteries do not provide one universal climate solution.
• Weather has benefited from a major machine-learning step change, while climate remains much harder. Weather predicts the atmosphere’s trajectory; climate concerns statistics of an attractor that is itself changing. With limited future data and large uncertainty in carbon-cycle behavior, Platt hopes for systems that can combine data with the scientific knowledge spread across many papers—but says ERA cannot yet do that fully.
• The biggest bottleneck is physical experimentation. Platt’s ideal system is an “everything lab” that accepts a JSON description and performs any experiment. ERA is computational, while experiments remain the ground truth and a major bottleneck; he also says scientists should develop deep domain expertise, rigor, creativity, and enough unoptimized practice to build those skills.

Deep dive

1. ERA’s founding move: science problems become “scorable tasks”

• Platt’s framing after more than 10 years of AI-for-science work at Google Research: around two years ago, the team “stumbled into this mapping” — many scientific problems can be phrased as “I really would like to have a piece of code that maximizes some score.” Statistical model fitting is only one instance of the broader framework.
• The non-obvious example: Michael Brenner, lead author on the ERA paper, uses the tool to work through scientific papers quickly. For asymptotic expansions of ordinary or partial differential equations, ERA can propose solutions, test whether they are asymptotically correct at epsilon = 1e-4, and ask Gemini to improve the mathematical reasoning while maximizing the fit.
• A remote-sensing example involves informed super-resolution across satellites: combining OCO-2 or OCO-3’s narrow, accurate CO₂ measurements with GOES’s roughly five-minute, coarse infrared imagery, plus weather and long-term albedo data, to estimate one satellite’s signal from another.

2. Under the hood: tree search over notebooks, Gemini as the mutation operator

• The harness first helps the scientist define a scorable task, produces a Python notebook with a score function, and then runs Monte Carlo research over hundreds or thousands of candidate notebooks. The candidates can form a tree as mutations produce children.
• Selection uses an upper-confidence-bound strategy: an optimistic estimate of a mutation’s potential, described by Platt as roughly predicting a 95th-percentile outcome. It therefore does not always choose the best current notebook; sometimes it chooses the fifth-best candidate because exploration may be more valuable. The team has also tried recombining ideas from two candidates into a third.
• Seeding does not require starter code. Given a textual problem description and, optionally, five relevant papers, Gemini can read the papers and take a first coding attempt. Bugs may produce an effectively minus-infinity score, after which the system can mutate the candidate.
• The default is about 10 parallel searches, or roughly 10 leaves grown at once. Too much parallelism prevents searches from learning from one another; after a batch, pruned history and prior results flow back into essentially one shared context.
• This works better than earlier evolutionary coding because “random mutation in code space is pretty much worthless.” Most random mutations are harmful, but Gemini can identify more informed directions to try because the mutation operator itself has substantial world knowledge.

3. The scoring function stays a human job — and Goodhart is always waiting

• The host says the hardest and most human part may be identifying the right score. Platt agrees: an agent can exploit a loophole in the objective, producing the result that was literally requested rather than the result the scientist intended. Researchers therefore have to revise the score and instructions iteratively.
• The payoff of the relentless inner loop is that scientists can move upward from implementation details toward the scientific problem itself. Instead of spending most of their time importing files or making databases work, they can think about what the cost function should mean. ERA can also suggest datasets to join.
• Platt invokes Goodhart’s law: once a metric becomes a target, it stops being a reliable metric. The principle applies to each leaderboard or benchmark, so optimization requires multiple layers of rigor and careful checking.
• The workflow is explicitly human-in-the-loop. ERA may run for a few hours, return examples and progress, and then accept further guidance — “like having a hyper-eager grad student who doesn’t sleep.”

4. The Gemini capability curve gated everything

• Asked about a phase transition, Platt says ERA “would have been impossible under Gemini 2.0.” The team experimented for some time, saw things begin to work, and then saw rapid improvement across Gemini versions and half-versions.
• His remembered inflection point was Gemini 2.5: he asked it to write boosted-decision-tree code, and “it just did,” leading him to say, “This is a new world.”
• His advice to scientists who tried Gemini 2.0 and gave up is that the model was already a year out of date — “that was an eternity ago.”
• A preprint combines ERA with Antigravity, another public harness that can pull in papers and write code, but the ERA-plus-Antigravity combination was not yet publicly available in the conversation.

5. Predictive versus descriptive models: Newton wasn’t modeling apples

• Platt distinguishes predictive models, which minimize error on a dataset, from descriptive models, which science wants to use for extrapolation because they capture physics or an actual description of reality. His illustration is that gravity is not really about apples: a model trained only on falling apples would not know what to say about planets.
• Pressed on where priors come from, he says the distinction is blurry. Scientists use intuition and accumulated facts to constrain models; in current ERA work, the analogous source is LLM intuition and world knowledge learned during pretraining.
• When pointed to existing papers, ERA can adapt published methods and often reverse-engineer a paper’s code. Platt gives a rooftop-solar-design example in which he believes the system reproduced the method from a paper without installing the original simulator.
• The limit is important: ERA does not, in Platt’s experience, discover a completely new physical model from scratch. If it is not told about a concept such as Clebsch–Gordan coefficients, it cannot reliably invent that entire formalism. Given known constraints and relevant literature, however, it can evolve models incorporating them.

6. Multiple-hypothesis testing, and why humans stay in the loop

• The host worries that mutating hypotheses and hyperparameters expands the search space and creates severe overfitting risks. Platt’s structural answer is that the scientist must still determine whether the result is a descriptive model of the world rather than merely a successful prediction on the tested data.
• He says the system has not yet become a mechanism for discovering wholly new physics, but it is a power tool for helping scientists discover new science.
• On ordinary overfitting, his answer is that researchers must use the same techniques as before and be especially careful not to fool themselves. The host summarizes the point as: “You need to use the same techniques, but be very careful with them,” and Platt agrees.
• On taste versus rigor, Platt sees both as necessary. Some people will focus on creative hypotheses and new science; others will ensure that systems do not crash, scale badly, or produce wrong results. Those roles need not belong to the same people.
• The frontier is jagged: coding, knowledge gathering, and finding related material have advanced sharply, while rigor, philosophy, and creativity remain weaker. Platt’s AI co-scientist once taught him about an unfamiliar ion occurring in magma, but he does not think it substitutes for human creativity.

7. ERA began as “AutoKaggle” — and humans reward-hack too

• ERA began as the AutoKaggle project because Kaggle is part of Google: the initial goal was to build a system that could compete in Kaggle-style problems. It performed very well in some playground competitions and did especially well in a CDC competition predicting the following week’s COVID and flu cases across U.S. states and territories.
• In other competitions, it came close without closing the final gap. Platt says performance often reflects how much attention someone devotes to squeezing out the last 0.001 or 0.01.
• In a Google contrails competition, human competitors found a half-pixel error in the labels: the distinction between a pixel’s center and lower-left origin when artificial data were rotated. They exploited it and beat the organizers.
• Platt’s conclusion is that people can behave like LLMs and reward-hack objectives too. Goodhart’s law therefore applies to every leaderboard, not just to automated systems.

8. Contrails: about 1% of anthropogenic warming, with a possible two-flight-level fix

• Persistent contrails can form a visible “waffle” of cirrus. They reflect sunlight during the day, but at roughly 10-micron wavelengths associated with Earth’s approximately 300-Kelvin blackbody emission, they absorb some outgoing infrared radiation and re-emit it in both directions. The net effect is generally warming because the infrared effect operates day and night.
• Platt cites estimates that contrails cause about 1% of anthropogenic global warming. In high-traffic regions such as Europe, contrail cirrus may create roughly 1 watt per square meter of local forcing, compared with about 3 watts per square meter for all anthropogenic warming averaged globally.
• The leverage comes from invisible, persistent ice-supersaturated regions in the atmosphere. They can be pancake-shaped and only a few hundred meters tall. For each gram of water, ice, or soot emitted in one of these regions, roughly 10 kilograms of water can be drawn out — an approximate 10,000-to-1 ratio.
• Planes can sometimes avoid the regions by dropping about two flight levels, at some fuel cost. Google combines satellite monitoring with a custom convolutional or U-Net-style model because weather models are not accurate enough to locate ice supersaturation directly, then supplies maps to flight-planning software.
• The cooling effect is not a simple geoengineering opportunity. Platt says warming is broadly persistent, while confident cooling occurs mainly over the poles during polar summer, where there are essentially no flights over Antarctica and relatively few flights elsewhere near the poles.

9. ERA’s flagship win: a counterfactual problem stuck for two years

• Estimating a contrail’s warming is counterfactual: researchers can measure what happened with the contrail, but not the alternate universe in which it was absent.
• The team’s model for outgoing longwave radiation worked reasonably well, but the reflected-sunlight component remained difficult for two years. They created synthetic tests by injecting artificial contrails into data, and their own approaches failed those tests.
• ERA searched over possible confounders and found a model that passed. It was simple and used a combination of confounders the team had not previously tried. In retrospect it looked obvious, but it had solved a real blockage. The work had been discussed at EGU, with a further paper still in preparation.
• The related CO₂ work has similarly large stakes. The uncertainty in future biosphere carbon uptake could amount to roughly 300 parts per million by 2100, compared with current concentrations of about 440–450 ppm. Satellite CO₂ estimation is one step toward reducing that uncertainty.

10. Weather had its step change; climate is still AI-resistant

• Weather prediction concerns the atmosphere’s trajectory over roughly 15 days, a horizon limited by chaos and the butterfly effect. Platt says weather has seen a major step change from machine learning, large datasets, and substantial compute — largely work dating from around 2018 rather than the newer LLM wave.
• A highlighted improvement is tropical-cyclone prediction, including more accurate tracks and earlier handling of intensification. Platt notes that hurricanes are heat engines powered by ocean heat.
• Climate is harder because it concerns the statistics of the system rather than one near-term trajectory, while the underlying system is changing. In Platt’s formulation, weather asks where you are on the attractor; climate asks about the statistics of the attractor, whose shape may itself move or change at a tipping point.
• When a climate simulation becomes unstable, researchers may not know whether the instability is a real physical effect or a model failure. There is also no direct data from 30 years in the future to reveal whether an extrapolation was overfit.
• Platt describes process models as reductionist decompositions of a complex climate system into many pieces, sometimes relying on a specialized fit for each piece. Ice microphysics illustrates the problem: crystal shape, fall speed, and mixing behavior are all uncertain, yet they can strongly affect climate models.
• His hope is for a future system that can read and integrate far more carefully evaluated scientific knowledge than any one researcher can, while fitting data in a constrained and scientifically sane way. He explicitly says that current ERA does not yet do this fully.

11. The AlphaFold test: when is a black box enough?

• The host frames AlphaFold as forcing a reconsideration of what scientific understanding requires. Data-driven models can be extraordinarily useful without providing the kind of mechanistic intuition that physicists traditionally sought. Platt invokes the maxim that all models are wrong but some are useful.
• Platt’s decision rule is primarily about data richness and whether the problem is closed. In a regime with enough data to cover the relevant cases — he cites proteins and the PDB — a statistical model such as AlphaFold can be highly effective. In open, nonstationary, extrapolative settings such as climate, caution is much more important.
• AlphaFold’s exhaustive application to the PDB is, to Platt, a particularly impressive example of a model being useful within a well-covered regime.
• The host also argues for reductionist assembly: individually understandable pieces can be composed into a coherent picture, whereas a giant black-box model claiming to explain a cell may be difficult to trust or inspect.
• Platt endorses the enduring baseline discipline of starting simply. The host recommends fitting linear regression first; Platt adds SVMs as another simple baseline, and they agree that many biological problems resist improvements beyond simple baselines even with substantial data.

12. Fusion: “three years away, not 30”

• Platt says there is “a definite probability” that commercially relevant fusion could be achieved by the end of the decade, and frames his personal estimate as “three years away, not 30.” He presents this as a possibility, not a certainty.
• His skepticism toolkit is the Lawson criterion: density, temperature, and confinement time must combine to exceed a threshold. Each fusion approach has an Achilles heel in at least one of those quantities, so headlines emphasizing a stable value for one quantity do not establish that the full criterion has been met.
• In work with TAE, Platt encountered field-reversed configurations, which theory can treat as unstable under certain magnetohydrodynamic assumptions but which can be stable in practice. A Z-instability can produce a wobble controlled by a PID system, while occasional discharges can still damage equipment and halt operations for weeks.
• Google DeepMind has worked on control systems for tokamaks to prevent disruptions. The hosts describe the failure mode as the plasma’s energy striking the vacuum chamber; they joke that a major disruption could make ITER a very expensive brick.
• The macroeconomic frame is the “pie chart of sadness”: there is no single climate silver bullet. Electrifying more of the economy could increase electricity demand by roughly five times, while batteries become increasingly expensive when covering the final 10–20% of demand. Some form of firm or baseload power may therefore be valuable, and Platt sees fusion as one possible contributor rather than a universal solution.
• The conversation also touches on a speculative proposal to transmute mercury into gold inside a fusion reactor. Platt calls it clever but says it might not work.

13. FireSat: catch fires while they’re the size of a room

• Platt says a wildfire can be easy to extinguish when it is about the size of a room but becomes much harder once it reaches an acre. Fires may remain small for a while before growing rapidly.
• Google designed a low-Earth-orbit constellation using midwave infrared sensing, where fires stand out through their temperature. Depending on the orbit configuration, roughly 50–80 satellites could detect fires about five meters across anywhere on Earth within approximately 15–20 minutes.
• The native sensor resolution is about 50 × 50 meters, with multispectral super-resolution and other AI methods used to reach an effective fire-location resolution near 5 × 5 meters. The sensors are not small enough simply to hitch onto arbitrary microsatellite constellations because midwave infrared requires refrigeration.
• Google worked with the Earth Fire Alliance, a nonprofit of which it is part, and with Muon Space; one prototype satellite had been launched. Google also propagates fire-boundary information through Android and Search.
• The team worked with the U.S. Forest Service on a fast neural-network proxy for the Rothermel fire-propagation model. Platt cites the World Health Organization’s estimate of roughly 300,000 excess deaths per year from wildfire smoke.
• His climate-resilience philosophy is that society may need both adaptation and mitigation: climate change is like a serious disease, so one should treat symptoms while also addressing the underlying cause.

14. Advice after the phase change: go deep, and hike the mountain anyway

• Platt sees a phase change over the prior 12–18 months. The field is moving from individually built specialized models toward general AI systems, but the AI-for-science community is still working out the best tools and workflows rather than treating the transition as settled.
• His example is his 21-year-old son, who is developing domain expertise in RNA and chemistry while also using vibe coding and other AI tools. Platt’s advice is to become deep in a domain while actively experimenting with the new systems.
• When the host asks how people should build expertise if an AI can instantly solve the basic problems that once provided training, Platt uses a hiking metaphor: it may still be worthwhile, and sometimes enjoyable, to hike up a mountain even when one could drive.
• He recalls rewriting boosting several times in several languages and says the process taught him boosting. Like athletic training, scientific and technical work needs both productive execution and time spent building the underlying muscles.
• Platt strongly protects 20% time in his group for learning and experimentation. He warns that hyper-optimizing productivity can itself become a form of overfitting and can remove the time needed for creative work.

15. Don’t be an empty suit — and the “everything lab” wish

• In response to the host’s joke that people may become middle managers supervising agents, Platt says they should not become “empty suits.” LLMs make unusual mistakes and cannot be fully trusted, but people should apply the same skepticism to their own work.
• He recalls the warning associated with Feynman that people cannot fool themselves and are the easiest person to fool. Enduring skills include knowledge of fundamental domains such as biology and the physical sciences, mathematics, rigorous checking, creativity, and thinking outside the box.
• Platt’s magic wish is “the everything lab”: a system that accepts a JSON blob and performs any experiment at all. That would require solving something like an AI-complete robotics problem.
• ERA is currently computational; someone still has to gather the physical data. Experiments remain the ground truth and a bottleneck, while general lab-in-the-loop systems remain very open.

16. 1982 with Feynman, near-zero compute, and Hopfield come full circle

• Platt took Feynman’s physics-of-computation class in 1982, in a setting involving John Hopfield and Carver Mead. Guest lectures took place on Tuesday, and Feynman would explain on Thursday why the material was wrong. Platt describes the “Feynman effect”: leaving convinced one understood, then realizing outside the room that one did not.
• The class debated reversible billiard-ball computers, thermodynamic limits, and what kinds of computation to pursue. It was DARPA-funded, and students were expected to attend a weekly prime-rib dinner.
• Platt describes the available computing power as “to rounding error, zero”: as Carver Mead’s system administrator, he used a VAX 11/750 capable of roughly 1 MIPS, while the research group shared an 80-megabyte disk about the size of a dishwasher.
• NeurIPS grew out of a nominally private Snowbird workshop that many people tried to attend, following earlier Santa Barbara and Caltech “Hopfest” activity. Platt notes that transformers can be understood as a form of associative memory, echoing the title “Hopfield Networks Is All You Need.”
• He says neural networks benefited from the hardware and software ecosystem around BLAS and GPUs. He does not claim to know whether brains work by matrix multiplication. Neural networks began substantially outperforming CPUs around the era of the original ImageNet and speech-recognition breakthroughs, when GPUs became much more useful for the workloads.
• The host adds a story about Platt’s graduate-school friend Brian Catanzaro persuading Jensen Huang in a short conversation that CUDA and GPUs could serve deep learning, contributing to NVIDIA’s shift toward that market.
• Platt also says he coined the generic term “convolutional net” because he did not work for Yann LeCun and did not want to call his work LeNet.

17. Two asteroids, an Oscar, and quantum’s steady march

• Platt discovered his asteroids through a 1980s Caltech planetary-science class with Gene and Carolyn Shoemaker. Students photographed the same sky position on film, used a stereoscope to find moving “floaters,” measured them against reference stars, and sent the observations to Brian Marsden at the Minor Planet Center to connect apparitions into orbits.
• He found two asteroids. One was named after his father; after Carolyn named another after a professor and Platt complained that he had wanted to name it, she gave him one to name. That asteroid was later named after his mother. An amateur astronomer subsequently found a moon around the mother-named asteroid through occultation observations; Platt says he believes this was last year. The main body is estimated at about 4 km and the moon at about 1 km.
• Platt says the Vera C. Rubin Observatory’s Simonyi Survey Telescope discovered 11,000 asteroids in six weeks, though he is unsure how many will receive names.
• His Oscar came from work begun during a 1986 Schlumberger internship, where he used elasticity simulation to model fabric and other floppy objects. The work appeared in SIGGRAPH, and descendants became part of physics simulation used in Pixar films. The award arrived about 20 years later.
• On quantum computing, Platt says he averages progress over decades. The field is in the NISQ era, a term coined by John Preskill that Platt does not particularly like. He describes Google’s Quantum Echoes work as a Feynman-style approach in which a parameterized Hamiltonian is fit to observed data such as NMR measurements; whether it is large enough to produce major breakthroughs remains undecided.
• He says Google’s quantum team is progressing along Hartmut Neven’s roadmap on a timescale of a few to several years, while the exact milestone timing should be checked with Neven. Superconducting qubits remain promising to him because of their scalability, although alternative architectures may ultimately win.
• Platt does not expect an instantaneous phase change: quantum systems are analog and highly finicky, and scaling physical qubits into useful logical systems remains difficult. The host notes that physical-to-logical qubit overhead and connectivity are important constraints.
• Dave Bacon’s joke is that Platt is always 20 years ahead of his time; after 10 years, he is “halfway there.” The host offers a counter-theory that Platt may be causal: he starts working on an area and reality eventually catches up.