Tabular Foundation Models and Synthetic Data
Examines deep learning paradigms for tabular datasets trained with prior-informed synthetic data.
0 sourcesVoices in this topic
All paragraphs
Sep 23, 2026
- Frank Hutter Deep learning has finally cracked tabular data — Tim Scarfe's framing is that TabPFN now "works dramatically better than CatBoost and XGBoost," after many failed attempts such as Google's much-hyped 2019 TabNet. Frank Hutter's explanation is that in-context learning transfers patterns and feature interactions across datasets: individual rows from a …
- TabPFN is "the first foundation model that's actually state-of-the-art, yet entirely trained on synthetic data" — hundreds of millions of datasets sampled from priors over structural causal models. There was no useful internet corpus to scrape ("people do not put their tabular datasets online"), which turns the usual data-moat logic …
- Context scale is compounding at "two orders of magnitude a year": TabPFN1 handled 1K rows, v2 10K, v2.5 100K, v3 1M, with 10M the stated next target. LLMs are poorly suited to this regime — a million rows by a thousand columns is 3–4 billion tokens, and sequence models "don't understand …
- Google's TabFM (described by Scarfe as roughly 30x larger) validates the field but exposes the trade Prior Labs is exploiting: it's 15x slower per forward pass, clearly strongest on tiny and small data, and "just breaks, out of memory" beyond roughly 100K rows. Hutter's post-recording update: TabPFN 3.5 took first place on seven benchmarks, is 20x faster than TabFM at equal quality or over 100 Elo …
- The causal roadmap is the most thesis-relevant moonshot: Do-PFN can use observational data at test time to make interventional predictions, after meta-training on synthetic causal interventions, and papers already show it can reduce the number of RCT data points needed for comparable performance. Hutter says that could "completely revolutionize medicine and A/B tests" if drugs can reach market on shorter studies — hedged with "we need …
- The commercial wedge is the agentic token economy, not a standalone app: "LLMs call calculators because they're better at doing math... just like that, they should call tabular foundation models." Revenue stack: free non-commercial open-source use, production license fees, an …