Summary

The episode’s central bet is that scaling and large-scale imitation learning are powerful but insufficient for agents that must adapt from a handful of interactions. Kevin Ellis and Zenna Tavares want systems grounded in uncertainty, causality, abstraction, and reasoning, while still exploiting GPUs and pre-trained models. The practical test is whether an agent can confront an unfamiliar webpage, appliance, or robot environment and learn by “poke[ing] things and push[ing] things and try[ing] things out.”
The challenge is not raw expressivity alone but steering through the combinatorial space opened by compositional representations. Reusable primitives let systems extrapolate when familiar causal mechanisms appear in new combinations, yet create Elizabeth Spelke’s “curse of a compositional mind”: the system becomes “immensely powerful and immensely overwhelmed by possibilities.” The practical challenge is learned search guidance and good representations, not merely larger hypothesis spaces.
Their ARC result argues for portfolios of reasoning modes rather than one universal inference path. Two Llama 8-billion models split the work: induction generates explicit Python functions that can be tested against examples, while transduction directly predicts an output grid after test-time adaptation. Some problems reward systematic search; others deteriorate when a solver tries to verbalize a rule and are better handled by something closer to intuition.
Explicit programs have a practical advantage because they can be checked, revised, and are harder to overfit. Python naturally expresses loops, counting, and repeated deterministic operations, while neural transformations empirically solve some messy patterns that are difficult to encode programmatically. Their preferred direction is therefore neither symbolic purity nor an opaque monolith, but deeper representations that combine neural flexibility with programmatic validation.
Wake/sleep offers a scalable route from a small human seed to a task-adapted program synthesizer. Starting from roughly 10 to 100 human-written programs, an LLM dreams variations, runs them, and trains an inverse model to infer programs from behavior; waking on real problems then shifts the next dream distribution. Kevin calls this substantially more effective than much of his earlier lambda-calculus machinery, while preserving the ability to adapt when the initial prior is wrong.
Useful world models should be plural and purpose-dependent, not pixel-perfect replicas. A camera can be modeled as a button that takes a picture, a circuit, or a sensor; resource limits and anticipated tasks determine which details matter. Zenna calls this “polystructural,” while Kevin defines abstraction as “some hiding of details but some retention of the essence.”
Project Mara is the concrete three-year R&D vehicle behind the thesis. The Basis nonprofit’s program spans modeling, abstraction, reasoning, and agency, developing both algorithms and interactive benchmarks around “everyday science.” It is funded and recruiting researchers and collaborators; its practical wager is whether agents can form and revise models of unfamiliar devices by interacting with them.

Deep dive

1. Scaling supplies the substrate, but not the full learning algorithm

Kevin’s target is machine learning that acquires abstract, communicable knowledge from few examples, generalizes far outside the training distribution, and lets symbolic structures cooperate with neural weight matrices. He focuses on capabilities where humans remain unusually strong and current AI remains comparatively weak.
His position is explicitly not anti-scale: algorithms should exploit new hardware, GPUs, and pre-trained models. But scaling “is not the full story,” and other labs already carry that torch, so their research hedges by combining those assets with cognitive science and classic AI.
Zenna treats Bayesian inference as a normative account of how an ideal agent should revise beliefs, not a literal claim that humans perform exact inference. The implementation is probably approximate; uncertainty, causality, and reasoning provide pieces, while many first-principles components remain unknown.
The contrast is with mainstream machine learning as highly effective “large scale imitation learning.” Their scientific and engineering question is whether intelligence can also be constructed from the ground up, with mechanisms that form hypotheses rather than relying only on abundant demonstrations.

2. Compositionality creates both extrapolation and combinatorial overload

Kevin defines compositionality as learning atomic knowledge in one situation, then recombining those atoms into larger structures for a new one. It helps when a changing world reuses the same building blocks and causal mechanisms; it does not make an agent instantly competent in a wholly alien world.
Elizabeth Spelke’s “curse of a compositional mind” captures the downside: a compositional language admits explosively many concepts, including wildly out-of-distribution ones, without automatically indicating which are plausible now. “You can kind of think infinitely many things,” becoming “immensely powerful and immensely overwhelmed by possibilities.”
Learned guidance over program search and neural implementations of basic atoms may tame that curse. Yet no primitive basis is universally best: Python can be broken “in a million ways,” programming systems admit a partial ordering, and the diversity of languages suggests a Pareto frontier conditioned on problems and environments.

3. ARC exposes a real split between induction and transduction

Their ARC system used two Llama 8-billion models. The induction model generated many explicit Python transformations and retained candidates consistent with training examples; the transduction model directly produced the output grid, supported by augmentation and transductive active fine-tuning. The ensemble tried validated induction first, then fell back to transduction.
Zenna separates two often-confounded axes: representation and type signature. A solver may represent a transformation as Python or a neural network; independently, it may output a reusable function from grids to grids or directly map an entire problem to its final answer.
Kevin connects their empirical split to cognitive science: making people carefully verbalize rules can worsen performance on statistical associations or rules with exceptions. ARC showed an analogous divide—sometimes sweeping through thousands of explicit hypotheses underperformed simply “blurt[ing] out an answer.”
The ensemble needs no learned router because symbolic hypotheses can be executed against the examples. It can search systematically, reject explanations that fail, and then use neural intuition. Their Venn diagram showed that each mode solved tasks the other missed rather than one cleanly dominating.

4. “Thinking” is a procedure, not a synonym for symbolic output

Zenna resists a hard boundary around thinking, noting that “now people say thinking is what o1 does.” A plausible core is step-by-step internal computation that revises beliefs, but symbolic representation and deliberative procedure remain distinct dimensions and can appear in different combinations.
Slow hypothesis formation may require variable-time computation that determines how much work a question warrants. Neither ARC method fully captures that process: induction still emits programs, while transduction produces an answer more directly. Reasoning is the narrower case of forming conclusions from knowledge amid real-world messiness.
Kevin prefers induction first because high-level programs are strongly regularized. A clear executable account that fits the examples is difficult—though not impossible—to overfit and is likely to generalize better than vector interpolation. Neural systems can eventually learn good representations at scale, but do not reliably do so in the small-data regime.
Zenna’s proposed middle ground is a transductive-inductive model whose transformation might itself be neural yet remains applicable point-by-point and testable on the training set. More broadly, neurosymbolic programming could let neural components encode irregular transformations while classical components supply structure and verification.

5. Representation determines what is learnable in practice

Kevin extends the old AI maxim, “You can’t learn what you can’t represent”: nominal representability is insufficient if the language makes search explode. A narrow domain-specific language can gain a Turing-complete escape hatch and formally match Python, yet still be practically unlearnable because the relevant program becomes too hard to find.
After working with DSLs and lambda calculus, Kevin finds Python “just a lot more practical” for ARC, agents, and visual question answering—“miles better than lambda calculus.” Software-engineering languages have undergone a convergent evolution toward useful structures, even though transduction shows that Python still does not cover every task gracefully.
Zenna points to class structures, type structures, and modular structures as evidence that languages evolved to support construction, not merely express computation. AI now creates a new feedback loop—ChatGPT calls Python, and Python calls ChatGPT—making the design space fluid rather than settled.
Both favor iterative metaprogramming over expecting the correct program in one shot. Humans and science build hypotheses, encounter evidence, and revise them; the unresolved mechanization problem is how to reward good refinement paths when no cheap terminal objective exists. Kevin’s emphasis: “A lot of this really hinges on being able to check that you’re heading in the right direction.”

6. Wake/sleep turns synthetic programs into an adaptive prior

Kevin explains wake/sleep through inverse problems: rendering an image from three-dimensional structure is easier than recovering structure from an image. Likewise, a system can imagine a program, run it forward, observe its behavior, and learn the reverse mapping: “When I see this kind of behavior, I should infer this kind of program.”
The important addition is interleaving. Rather than generating one fixed synthetic batch, the system dreams, wakes to solve real problems, remembers what it encountered, and adjusts later dreams. If it expected short programs but finds long ones, subsequent sleep cycles move toward that distribution.
Dreams should not exactly reproduce waking experience. A compositional language can join two previously learned atoms into a plausible unseen structure, preparing the neural network for nearby possibilities: “You actually want them to go a little bit beyond” the observed distribution.
In the NeurIPS program-induction work, roughly 10 to 100 human-written programs prompted an LLM to generate similar code; execution supplied synthetic behavior for training a synthesizer. Kevin says this neuralized version was “a lot more effective” than much of his PhD-era lambda-calculus machinery, with a full wake/sleep cycle adapting it to solved problems.

7. Libraries compress the past, while agency tests causal truth

Zenna frames library construction as cached computation with both present and expected future value. A reusable function shortens today’s program, but engineers also build libraries because they expect themselves or others to need that structure later—a trade-off that might be formalized through rational decision theory.
DreamCoder learned explicit symbolic libraries; newer systems keep successful code and use neural in-context learning to generate related code. Kevin calls this an implementable “halfway measure,” not a rejection of libraries: explicit reuse remains valuable, but automatically debugging an entire AI-written library is substantially harder than debugging one program.
Causal abstractions require contact with the world. Equivalent programs may describe the same static function, but an agent with actions, goals, and interventions can falsify claims about causal mechanisms. In ordinary program synthesis, a library is merely useful or not useful; in a world model, an intervention can show it is wrong.
Zenna’s Autumn project synthesized source code for interactive game-like environments from human traces, including unobservable latent state evolving over time. Its limitation is instructive: the inferred programs aimed at ground-truth dynamics rather than selective abstraction, whereas practical intelligence must omit the right parts of reality.

8. No single world model is correct for every purpose

Kevin’s common denominator for abstraction is “hiding of details but some retention of the essence.” A lambda expression ignores particular variable values; a causal abstraction relates a detailed model to one that discards mechanisms while preserving what matters.
Abstractions are often constructed on the fly rather than read from one ground-truth hierarchy. Full-pixel systems such as Sora- or Genie-style models preserve everything; reward can instead constrain a useful abstraction to what supports planning, as in MuZero’s predictions of reward, policy, and value.
Visual Predicator, work led by Yichao Liang, applied that principle to simulated robots that saw pixels while learning abstractions for tools and mechanisms. Kevin’s harder open question is how people form useful models while freely exploring an object or application without being told a reward; intrinsic motivation or robustness across possible rewards might help.
Zenna’s resource-rational answer balances expected tasks against computational costs. “There isn’t a single world model”: a camera can be a picture-taking button, a circuit, or a sensor. Her tentative term “polystructural” describes representations containing several models and their relationships—including why a COVID model may legitimately omit hair color.

9. Learned priors should guide epistemic search without becoming Frankenstein systems

Kevin wants to avoid “Frankenstein systems” assembled from hand-coded representations and reasoning heuristics. First-principles evaluation should determine whether abstractions serve anticipated reasoning tasks, while pre-trained neural networks act as proposal mechanisms—using common-sense intuition to suggest promising code inside an otherwise intractable search.
Zenna retains the Bayesian need for priors but accepts the “bitter lesson” against manually encoding every inductive bias when sufficient data and compute can learn them. Those priors might come from richer evidence than internet text, including observations of human actions from which beliefs can be inferred.
For ARC, Zenna proposes abstract programs with holes: first infer that an object moves, while leaving its color transformation unresolved, then test candidate completions. Kevin’s own process mixes perceptual “denoising” with half-formed hypotheses; human mistakes differ from model mistakes, suggesting current systems miss important solution-building dynamics.

10. Project Mara turns ARC into an interactive science problem

Zenna notes that François Chollet has called ARC imperfect; their solution mixes fundamental ideas with “ARC hacks.” She says Chollet believed conditioning on the ARC collection could be sufficient without internet pre-training, yet the strongest empirical approaches are at least partly pre-trained. A cleaner solution would be “simple and elegant.”
One defense against benchmark overfitting is to require a method to solve ARC plus related but distinct domains. Python excels at counting, loops, and applying one operation to every object; neural networks empirically excel at some transformations humans cannot easily program. Repeated random seeds confirmed that split, though Kevin offers no complete theory for it.
Project Mara is a three-year Basis program spanning modeling, abstraction, reasoning, and agency or acting. It develops both algorithms and benchmarks around an “active ARC”: a model still learns from very little data, but must obtain that data by interacting—“poke things and push things and try things out.”
Its narrower theme is “everyday science”: discovering how a new microwave, hotel AC system, toy, interface, webpage, or dishwasher works by forming hypotheses and choosing experiments. Basis is described as a nonprofit; the program is funded, accepts donations, and is recruiting research scientists, research engineers, and adjacent collaborators for work outside the larger labs’ mainstream.

Dave Smith: Israel, Hamas, Ukraine, Russia, Conspiracies & Antisemitism | Lex Fridman Podcast #464 Windward's Marc Chalfin Turtle Beach Thesis $TBCH