Why World Models Could Change Robotics, 3D, and Creativity

Summary

• World Labs’ newly launched Atlas introduces a genuinely new foundation-model primitive — novel-view prediction — unifying 3D reconstruction and generation in one architecture for the first time. Justin Johnson’s formulation: “LLMs are built on predicting the next token… video models are built on predicting the next frame. Atlas is truly a prediction of new perspectives.” Ben Mildenhall adds that each input image carries a 3D camera pose, versus the “slot machine” effect of text-prompted video models.
• The capture-economics claim is a 50–100× cost reduction: dense reconstruction that needed 100–300 photos per room now works from three iPhone shots. The Matrix bullet-time effect that took a ring of hundreds of cameras on a green screen now needs three tripods and no expensive calibration — and old footage with 95% of its photos deleted still reconstructs, which Ben says “completely turns the idea of what kind of data can be reconstructed upside down.”
• The team insists scaling has barely started — the released model was bounded by a release deadline, not architecture, scale, or data limits. “We’re at the very beginning,” with compute the main bottleneck; every scale-up “got significantly better,” and Fei-Fei Li concedes the surprise that “the first cycle” of pretraining a new paradigm worked at all — she’d expected several iterations.
• The robotics thesis rests on data, not silicon: “the biggest problem in robotics right now is actually data. Someday it will be chips, but now it’s data.” The acquired robotics team, formerly Synapse, can use Atlas to improve its painful real-to-sim pipeline, and Johnson proposes data-driven neural simulators as a future foundation; Casado then asks whether the simulator itself could become a player — consistent with the universality thesis of world models.
• The known gap — dynamics — is already latent in the model, they claim, not an architectural limitation as it was in Marble. Counterintuitive training insight: even for static output, “the best way to achieve that is to actually show the model dynamics” and let it filter; this checkpoint was merely post-trained toward statics, with waves and moving cars already visible in outputs.
• The commercial wedge beyond entertainment is design iteration, where translating feedback into a 3D model “is 95% of the job.” Architecture, construction, even conference booths — with editability as the bar to clear: control must come “without degrading the quality of the model, otherwise it will just become entertainment.”
• The closing claim is AI-completeness: “predicting the next viewpoint is equivalent to predicting the next token.” Ben Mildenhall’s evolutionary kicker — “Nature gave animals eyes. But nature did not give trees eyes… when you move, you see a new vantage point” — and Martin Casado’s verdict: “probably the most important model launch this year.”

Deep dive

1. Atlas’s primitive: predicting the next viewpoint, not the next frame

• Justin Johnson’s launch summary: Atlas generates, reconstructs, and simulates the world — camera-conditioned generation from an image plus trajectory, sparse 3D reconstruction from one to ~100 frames, and simulation shown via bullet-time videos and robotic sim. The primitive underneath: “LLMs are built on predicting the next token… video models on predicting the next frame. Atlas is truly a prediction of new perspectives” — point a virtual camera anywhere in space and time and Atlas renders it.
• The Matrix demo as told: Neo’s falling shot required a ring of hundreds of cameras around a green-screen studio; Atlas recomposes a frozen-time flyby — milk splashing mid-air as the camera sweeps past — from “three iPhones on tripods,” with no studio, green screen, or calibration.
• Mildenhall on what separates this from video models claiming world-model status: in Atlas “each image actually has a corresponding three-dimensional camera pose,” so, for a sufficiently covered scene, reconstruction can reproduce the observed space with extremely high accuracy rather than being negotiated via text prompts — escaping “the ‘slot machine’ effect of having to redo the generation over and over again.”

2. Fusing reconstruction and generation bridges a half-century split in computer vision

• Fei-Fei Li’s historical framing: reconstruction and generation have been separate conference tracks for over half a century — “I can’t tell you how many PhD theses have been written” on novel-view synthesis — and a single “elegant model that combines or unifies” them is “just incredibly powerful.”
• The enabler is native multimodality from pretraining: text, images, video, plus camera positions and depth maps as first-class inputs, “which I don’t think anyone has done before in the pre-training phase,” per Johnson.
• Why both halves are load-bearing, in Fei-Fei’s explanation: classical reconstruction triangulates visible points, but anything unseen is a hole. Even a world expert with a DSLR taking hundreds of shots misses under microphones and between chair legs, so generation must fill the gaps “because you can never get everything.”

3. From Marble’s Gaussian-splat bottleneck to 50–100× sparser capture

• World Labs is two and a half years old; last year’s Marble output Gaussian splats — renderable on phones and VR headsets, but the fixed output representation was a “bottleneck.” Atlas separates modalities and makes view prediction the primitive, producing RGB, 3D, or splat worlds on demand — a redesign that took “sweat, blood, and tears” over representation trade-offs.
• The team’s numbers on dense versus sparse: traditional capture needs “somewhere between 100 and 300 photos of this room”; Fei-Fei saw a first-timer spend about 2 hours scanning a multi-room space. Atlas cuts the target to three, “a 50–100× reduction”: Mildenhall deleted 95% of the photos from old recordings and still got reconstructions, and reduced 2,000 house photos to 30–40 inputs with a near-identical flyby.
• His LLM parallel: the “context wars” took language models from 128 tokens to a million, but nobody pushed context into image/video models in the same principled way — “reconstruction is just generation with a very long context.”
• Fei-Fei’s favorite overlooked demo: 3–25 ground-level photos of the Stanford courtyard yielding an aerial flythrough — “everything you see is generated, but according to the laws of reconstruction.”

4. Conviction, the soccer-ball Slack message, and “we’re at the very beginning”

• Casado asks whether they knew it would work. Johnson: “I was pretty confident”; Fei-Fei says all three had “complete conviction about the law of scaling” — but she preserves the honest caveat: “the fact that the first cycle worked is simply incredible”; there was no scaling law for spatial intelligence when they founded the company.
• The origin story as insider detail: early summer, a smaller pre-Atlas model, Ben fed in the famous NeRF garden-table data — and a Slack message that night showed the camera flying under the table past a soccer ball. “The three of us looked each other in the eye and said, ‘This is it’… We made the decision in literally 5 seconds.”
• On headroom: “No, no, we’re at the very beginning” — no architecture change is currently needed, compute is the main binding constraint, and the released model’s size was set by “a release deadline,” not by scale or data limits.

5. Use cases: 3D-consistent creative pipelines, design, and robotics’ data problem

• Mildenhall noticed Marble users uploading an image, taking a few screenshots of the splat scene, and leaving — so Atlas can generate those views directly, grounding creative work in a 3D-consistent world versus fighting models where “give me different angles of the ruins” always drifts. “This consistency and resilience are key to how humans think in spatial categories.”
• His beyond-entertainment wedge: architecture, construction, conference-booth builders — because translating a creative director’s feedback into a 3D model “is 95% of the job, right? You hold a meeting, get feedback, and then spend a week making edits.”
• Fei-Fei on the robotics acquisition, formerly Synapse: its real-to-sim-to-real pipeline depended on “extremely painful” dense reconstruction, and Atlas is “the next generation technology for this.” Her categorical claim: “the biggest problem in robotics right now is actually data. Someday it will be chips, but now it’s data” — including randomizing cable bends, box sizes, and scene layouts.
• Johnson’s deeper argument: robot policies aren’t static artifacts like code or images with abundant web examples — they’re agents in a world that “doesn’t always react the way you expect,” so training must expose them to everything that can go wrong. Hence data-driven neural simulators; Casado then asks, “why couldn’t the simulator itself become a player?”

6. Dynamics are latent, editability is the moat, and view prediction is “AI-complete”

• The sharpest critique Casado relayed from an expert: “more dynamics are needed.” Fei-Fei’s answer — Marble was “fundamentally static” by architecture and training; Atlas’s architecture and data already contain dynamics (waves on water, small cars moving in aerial views). The counterintuitive lesson: static-only data doesn’t scale, so “even if I want to get a static result… the best way is to actually show the model dynamics” and let it filter; this checkpoint was simply post-trained toward statics.
• Mildenhall’s next frontier is industrial-grade control — scene interaction, layout, object identities, time — “without degrading the quality of the model, otherwise it will just become entertainment.”
• The closing thesis: just as next-token prediction is AI-complete — the detective-novel example Johnson tentatively attributes to Ilya, ending “And the murderer was…” — generative new-view prediction is too: watch a whole film, then predict who appears in the killer’s reveal. Mildenhall’s evolutionary version: “Nature gave animals eyes. But nature did not give trees eyes… when you move, you see a new vantage point.” “Predicting the next viewpoint is equivalent to predicting the next token.”