World Labs and Fei-Fei Li: Atlas, New View Prediction, and the Horizon of Spatial Intelligence

Original Video: Why World Models Could Change Robotics, 3D, and Creativity

a16z · 2026-09-04 · 43 min 43 sec

Host: Martin Casado, General Partner at a16z · Guests: Fei-Fei Li, Justin Johnson, and Ben Mildenhall, Co-founders of World Labs

Subtitle Note: Official English transcript provided by a16z; technical terms and project names have been verified against original publications.

Primary Sources: World Labs Official Site and Atlas Announcement · a16z Podcast Homepage · NeRF Research Paper and Project

On September 4, 2026, World Labs—co-founded by Fei-Fei Li, Justin Johnson, Ben Mildenhall, and Christoph Lassner—officially unveiled Atlas, its next-generation foundational world model.

The demonstration videos released alongside the launch quickly sent ripples through the technical community. Using nothing more than footage captured on three everyday iPhones resting on standard tripods, the model reproduced the iconic "bullet time" camera swoop from The Matrix without green screens, specialized studio capture rigs, or precise physical camera calibration. In another benchmark, casual ground-level snapshots of the historic Stanford Quad were translated into a sweeping, photorealistic aerial drone fly-through high above the campus.

In an in-depth conversation hosted by a16z shortly after the launch, the founding trio sat down with General Partner Martin Casado to conduct an architectural post-mortem. The discussion deliberately looked past superficial visual novelty, addressing an underlying paradigm shift: generative artificial intelligence is evolving from producing visually pleasing but ephemeral two-dimensional pixels toward generating pixels that are fundamentally anchored in spatial geometry and grounded in physical reality.

Where large language models rely on next token prediction and video generation models build on next frame prediction, Atlas introduces a fundamentally new primitive: new view prediction.

Unifying Generation and Reconstruction: The Convergence of Fifty Years of Vision Research

Throughout more than five decades of computer vision history, three-dimensional reconstruction and image generation have inhabited almost completely isolated academic tracks.

Conference agendas illustrated this division vividly. On one side stood the geometric reconstruction community, championing mathematical and optical rigor across Structure from Motion, Multi-View Stereo, Neural Radiance Fields, and 3D Gaussian Splatting. On the other side operated the generative modeling track, advancing from early Generative Adversarial Networks to the modern diffusion models that dominate creative media.

Both traditional lineages carried inherent structural limitations.

Classic dense 3D reconstruction operates much like scientific or medical imaging. To reconstruct any given cubic centimeter of space, algorithms require that every surface point be observed across multiple intersecting camera rays through triangulation. This demands exhaustive physical scanning. Scanning a multi-room interior often takes an untrained user hours of walking with a camera to capture hundreds or thousands of overlapping images. Even when captured by world-class specialists, minor occlusions beneath chair frames, under table edges, or between foliage inevitably leave conspicuous holes.

Pure video diffusion models ventured toward the opposite extreme. By training on vast corpuses of unannotated internet video, they acquired striking associative intuition, yet functioned essentially as visual slot machines. A user pulls the lever with a text prompt, but the moment the virtual camera orbits or loops back, perspective warps, object geometries drift, and structural continuity collapses.

Atlas resolves this impasse by unifying pixel generation and geometric reconstruction within a single underlying architecture.

When presented with a physical scene, Atlas does not treat unobserved occlusions as unfillable dead zones, nor does it treat novel viewpoints as unconstrained hallucinations. By anchoring directly to calibrated camera poses, it preserves absolute geometric fidelity across observed regions while leveraging generative priors to synthesize occluded gaps in a physically and architecturally coherent manner.

New View Prediction: Physical Intelligence's Counterpart to the Next Token

Underpinning this architectural consolidation is a newly formalized mathematical primitive: new view prediction.

Comparing this concept to the historical trajectory of large language models, Justin Johnson emphasized that natural language processing achieved general-purpose reasoning precisely because the discipline consolidated around a single autoregressive primitive: next token prediction. Whether translating prose, writing code, or evaluating formal logic, every task was mapped to the probability distribution of the next token given an antecedent context.

Video generation architectures attempted to apply that same autoregressive template across the temporal axis, establishing next frame prediction. However, chronological frame extrapolation lacks intrinsic three-dimensional geometric understanding. When the camera shifts, frame-by-frame prediction frequently misplaces spatial coordinates because the model possesses no underlying mental representation of what occupies the three-dimensional world beyond the immediate viewing plane.

Atlas instead constructs what the team terms a spatial context by encoding input observations alongside their explicit 3D camera poses. Once this implicit spatial representation is established, the user can supply an arbitrary virtual camera coordinate in space and time. Atlas's task is not to guess what happens next in time, but to deduce what the physical universe looks like when observed from that specific vantage point.

Fei-Fei Li contextualized this primitive through evolutionary biology. Nature endowed mobile animals with eyes while denying vision to stationary trees, precisely because motion generates shifting viewpoints. In physical reality, spatial intelligence evolved out of an organism's imperative to predict, navigate, and verify environmental feedback as it moves through space.

Seen through this lens, new view prediction is AI-complete within the physical universe. If a foundational model can flawlessly predict incoming visual and volumetric depth from any arbitrary trajectory in space and time, it has effectively internalized a comprehensive world model encompassing matter, lighting, occlusion, and physical dynamics.

From Three iPhones on Tripods to the Stanford Quad: Compressing Capture Costs by Two Orders of Magnitude

The practical ramification of this technical primitive is a dramatic collapse in the physical cost of capturing and rendering complex environments.

In cinema history, Keanu Reeves's bullet-dodge sequence in the original Matrix film established a visual effects benchmark. To freeze a dynamic actor while orbiting a camera across three hundred and sixty degrees, cinematographers erected an elaborate circular array comprising hundreds of synchronized cameras inside a dedicated green-screen studio, requiring days of optical alignment and calibration.

In Atlas, that identical bullet-time effect was achieved with three consumer iPhones affixed to budget tripods. Recording dynamic scenes such as a basketball descending through a hoop or a strawberry splashing into a bowl of milk, the team required no green screens, no motion-capture rigs, and no studio facilities. Atlas took the three sparse views and reconstructed smooth, sweeping trajectories through the frozen temporal slice.

The Stanford Memorial Quad demonstration highlighted an even starker contrast. Ben Mildenhall walked across the courtyard capturing between three and twenty-five casual ground-level snapshots with a handheld camera. When fed into Atlas, the model generated an aerial fly-through sweeping high above the red-tiled rooftops and colonnades.

No high-angle photography or drone footage had been supplied in the prompt. Every courtyard roof tile, archway shadow, and geometric alignment in the aerial trajectory was generated purely from the model's spatial reasoning, adhering faithfully to perspective laws extrapolated from the ground-level perspective.

This represents a fifty- to one-hundred-fold reduction in physical data capture requirements. Real-world physical spaces that formerly demanded thousands of photogrammetry passes and terrestrial laser scans can now be reconstructed and extrapolated from sparse, informal imagery.

Architectural Evolution: Decoupling Gaussian Splats and the Spatial Context Window

Prior to Atlas, World Labs launched Marble as its exploratory first-generation product. Tracing the engineering progression from Marble to Atlas illuminates how spatial foundational architectures have matured.

Marble formulated its output directly as 3D Gaussian Splats. While Gaussian splatting enabled rapid rasterization on mobile chips, virtual reality headsets, and game engines, outputting a splat array end-to-end created a severe representational bottleneck. Splat representations resisted clean scaling to massive multimodal pre-training, restricting the number of input views the model could ingest simultaneously.

With Atlas, the engineering team decoupled the generation engine from the terminal asset representation, enacting three structural redesigns.

First, Atlas was architected as natively multimodal from the ground up. In addition to text, RGB images, and video, the pre-training objective natively ingests three-dimensional camera poses and metric depth maps. Depth maps establish explicit geometric surfaces, RGB channels inform radiance and material reflectance, and camera coordinates serve as universal navigational anchors.

Second, the architecture incorporates a spatial context window analogous to long-context language models. Just as extended context windows enabled language models to perform needle-in-a-haystack retrieval across lengthy documents, Atlas approaches 3D reconstruction as generative extrapolation across a dense spatial context. A user can provide sixty-four casual views of an entire multi-room residence, allowing the model to generate continuous interior walkthroughs whose geometric consistency matches previous scans requiring thousands of images.

Third, output modalities were decoupled from internal representations. Atlas operates fundamentally as an autoregressive view prediction engine. It directly generates photorealistic RGB frames and synchronized depth fields, which can subsequently be baked downstream into high-density Gaussian splats, dense point clouds, or textured polygonal meshes depending on production pipeline requirements.

Scaling Laws for Spatial Intelligence: The Beginning of the Compute Curve

In foundation model research, the presence or absence of predictable scaling laws determines whether an architectural framework represents a temporary optimization or enduring infrastructure.

When asked whether world models obey empirical scaling laws, Justin Johnson answered unequivocally in the affirmative. Throughout development, World Labs trained a sequential ladder of models across increasing compute budgets. As parameter counts expanded, training durations stretched, and accelerator clusters grew, the model demonstrated steady, monotonic gains in novel-view reconstruction fidelity and geometric coherence.

A pivotal milestone occurred early in the summer on a preliminary experimental checkpoint. Feeding in the classic garden table benchmark from early NeRF literature, the model autonomously navigated a virtual camera underneath the tabletop—a region never observed directly from above. The model generated the shaded underside of the table and rendered a soccer ball resting on the grass beneath with precise geometric parallax. The founding team recognized within seconds that the scaling hypothesis held true.

Furthermore, World Labs confirmed that the prevailing bottleneck constraining further capabilities is neither data volume nor architecture, but raw training compute.

The public Atlas checkpoint reflects the maximum compute budget that could be allocated prior to the scheduled launch date rather than a ceiling in the model's structural capacity. The scaling frontier for spatial intelligence is only beginning to be charted.

Embodied Intelligence and Real-to-Sim: Turning World Models into Neural Simulators for Robots

Beyond entertainment, digital visual effects, and gaming, World Labs' strategic roadmap focuses heavily on embodied robotics. The startup's earlier acquisition of robotics simulation venture Synnex directly underpins this objective.

The foremost roadblock in modern robotics is a severe scarcity of high-quality physical interaction data. While language and coding agents leverage decades of accumulated internet text, robots attempting to manipulate industrial cables, sort parcels, or wash dishes operate in real-world environments governed by friction, compliance, and unexpected physical contact. Gathering thousands of physical failure modes with hardware in actual factories is prohibitively slow and expensive.

The historical solution has been Real-to-Sim: digitally recreating real-world workcells via exhaustive laser scanning, importing them into physics engines, and running reinforcement learning policies. Yet manual reconstruction of a single cabling station routinely consumed multiple days of specialized labor.

Atlas upends this pipeline by enabling rapid Real-to-Sim conversions. An engineer can photograph an industrial workspace with a standard mobile phone, and Atlas immediately reconstructs a geometrically metric, photorealistic 3D simulation environment.

Crucially, this foundation unlocks rapid domain randomization. Within the generated simulation, developers can parametrically vary cable bend stiffness, surface reflectivity, bin dimensions, and ambient lighting across tens of thousands of procedural permutations, preparing robotic control policies to bridge the reality gap when deployed on real hardware.

Over the long term, World Labs envisions world models functioning as interactive neural simulators. Once a world model thoroughly masters how physical environments react to robotic actuation, the simulator itself can serve as an action planner. By projecting the physical downstream consequences of candidate motions, an embodied agent can directly infer the optimal sequence of actions required to achieve a target state.

Baby Dynamics, 4D Interaction, and a Paradigm Shift for Creative Workflows

Addressing initial public commentary regarding Atlas's current emphasis on static scene exploration, the founders clarified the sequencing of their engineering roadmap.

Atlas ingested substantial dynamic video datasets throughout pre-training, and the current weights already demonstrate nascent temporal motion. Careful inspection of released fly-throughs reveals subtle dynamic phenomena, including wind-blown surface ripples across bodies of water and traffic moving along distant roadways.

A counter-intuitive technical realization guided this staged release: mastering static geometry is a prerequisite to mastering dynamic physics. Because uncurated internet video is saturated with camera shake and transient object movements, the model had to learn to disentangle invariant spatial geometry from momentary temporal changes. The team deliberately emphasized spatial camera control in post-training to anchor geometric stability before releasing full physical dynamics.

The subsequent milestone on the roadmap is true four-dimensional world modeling, where both the camera and the scene's constituent elements undergo continuous temporal transformation, deformation, and physical interaction.

In closing their exchange with a16z, the founders outlined how spatial intelligence will transform traditional creative and industrial design workflows. Today, across architecture, industrial fabrication, film pre-visualization, and spatial computing, ninety-five percent of project labor is spent translating conceptual feedback, sketches, and aesthetic commentary into cumbersome, decades-old 3D software suites such as Maya, Blender, and CAD.

When world models combine grounded physical priors with intuitive multimodal controls and temporal persistence, manipulating virtual environments will become as natural and accessible as sculpting clay or assembling building blocks.

Atlas demonstrates that alongside large language model token sequences, the three-dimensional physical world is finding its universal grammar in spatial geometry and new view prediction.