World Models Explained: Why AI Labs Are Racing to Simulate Reality

A world model is an AI system that predicts how an environment will change, including what happens when something acts inside it. A chatbot predicts the next word. A video generator predicts the next frames of a clip. A world model predicts the next state of a place, so a person or a robot can act in it and see what follows.

That idea got very expensive this year. AMD has agreed to buy World Labs, the startup Fei-Fei Li founded in 2024, in an all-stock deal valued at about $8.2 billion (AMD). Yann LeCun's AMI Labs raised $1.03 billion before shipping anything (TechCrunch). Google DeepMind, NVIDIA, Meta and Runway all have world models out in some form.

The products are much smaller than the money. Today you can buy a 3D scene generator, rent a short research prototype through a Google subscription, or download open models if you build software for robots or self-driving cars. Our read: the 3D-scene and robotics-software uses are real now, and the promise to simulate reality is still mostly a demo reel. Below is how to tell the two apart, with prices for everything you can try.

A world model predicts the next state, not the next word

The lab definitions line up. Google DeepMind says world models use their understanding of physical environments to simulate them, so an agent can predict how a world evolves and how its own actions affect it (Google DeepMind). Runway's version is shorter: a system that builds an internal representation of an environment and uses it to simulate future events inside it.

The word that matters is action. Ask a language model about a ball rolling off a table and it writes a sentence. Ask a video generator and it renders a clip you watch. A world model takes the current state plus an input (a step forward, a camera turn, a robot gripper closing) and returns the next state, then repeats on your next input.

Read the bottom row as the whole idea. The output feeds back in as the next input, so a world model is judged on staying consistent over time. A single gorgeous frame proves very little.

The concept is older than the hype. In 2018 David Ha and Jürgen Schmidhuber published World Models, in which an agent learned a compressed model of a game and was trained entirely inside its own "hallucinated dream" before acting in the real environment. What changed is scale. Models trained on enormous amounts of video now produce worlds that look photographic, and that pulled the idea out of reinforcement-learning papers and into earnings calls.

The milestones below show how fast it moved once video models got good.

Six years separate the dream paper from playable worlds learned from internet footage. Almost everything else, from open robot models to a chipmaker writing an $8.2 billion check, landed within about two years.

Four bets on how to build one

Labs disagree about what a world model should output, and that disagreement explains most of the products. World Labs' own taxonomy is the clearest way to sort them. It splits world models into three kinds. Renderers output "pixels meant for human eyes." Simulators output a "geometrically, physically or dynamically faithful representation of the world." Planners output actions (World Labs).

Its warning about renderers is worth keeping in mind for every launch video: the buildings in a drone shot "may look flawless from above, but try to drive through the city below and they fall apart." With that lens, the field sorts into four camps.

Interactive worlds you steer in real time

Genie 3 from Google DeepMind turns a text prompt into a world you move through at 720p and 20 to 24 frames per second. Google says its worlds stay largely consistent for several minutes and that it recalls changes from specific interactions for up to a minute (Google DeepMind). Runway's GWM Worlds 2, a research preview, does something similar with video and audio and keeps generating for as long as you keep steering. World Labs' taxonomy names Genie 3 as a renderer, and GWM Worlds 2 sounds like the same kind of system: you get a stream of pixels, and nothing you can open in another tool.

3D world builders that hand you a file

Marble, World Labs' first product, turns text, photos, video or rough 3D blocks into a persistent world you can edit and export as Gaussian splats or meshes (World Labs docs). The world doesn't react to you the way a Genie world does. In exchange, it exists as a file that game engines and design tools can open. World Labs says its newer Atlas model, in early access with select partners, will power future versions of Marble.

Physical AI models for robots and cars

NVIDIA calls these world foundation models. NVIDIA Cosmos 3 reasons over video, generates synthetic footage from text, images, video, sound and action inputs, and can act as the backbone of a robot's control policy. The weights are open under the OpenMDW 1.1 license (NVIDIA). Decart's Oasis 3 is narrower: a real-time, multi-camera driving world that self-driving teams reach through an API.

Latent predictors that skip the pixels

Meta's V-JEPA line doesn't try to draw the future at all. V-JEPA 2 encodes video into compact representations and predicts how those representations change, which is cheaper than rendering every pixel and still enough to plan with. Meta reports 65% to 80% success at picking and placing unfamiliar objects in new environments, after training on the open DROID robot dataset and deploying straight to lab robots with no data collected from them first (Meta AI).

This is the camp LeCun has backed for years. He has since left Meta, and AMI Labs builds on the same Joint Embedding Predictive Architecture he proposed in 2022.

If the vocabulary is getting thick, these are the terms the pitches lean on.

Action-conditioned is the one to hold vendors to. If a product can't take your input mid-stream and doesn't give you a 3D state to work with, it's a video generator with a new label.

Video generators look like world models until you push them

OpenAI pitched its 2024 Sora research as a path toward general-purpose simulators of the physical world, while admitting the model couldn't get basic interactions like glass shattering right. With Sora 2, it pointed to progress: when a basketball player misses a shot, the ball now rebounds off the backboard instead of teleporting into the hoop. OpenAI also called the model "far from perfect" (OpenAI).

Then the product went away. OpenAI says the Sora web and app experiences were discontinued earlier this year, and its help center puts the Sora API's shutdown later in 2026. That's a business decision, but it's a useful reminder that "world simulator" was a research framing attached to a consumer video app.

Independent research is blunter. The Physics-IQ benchmark tested Sora, Runway, Pika, Lumiere, Stable Video Diffusion and VideoPoet on fluid dynamics, optics, solid mechanics, magnetism and thermodynamics. Its finding: physical understanding "is severely limited, and unrelated to visual realism" (arXiv). A separate study trained video models on simple simulated physics and found they copy the closest training example instead of learning the rule, concluding that scaling alone won't uncover physical laws.

So treat video models such as Veo 3.1 and Runway Gen-4.5 as excellent renderers and weak simulators. That's fine for an ad or a storyboard. It's a poor bet if you plan to trust the footage to teach a robot what a wet floor does. If video is what you need, start with our AI video stack comparison and the AI video generators board.

Why chipmakers and investors are paying up

AMD's deal is the clearest sign of where this is heading. The company values the all-stock transaction at about $8.2 billion, expects it to close by the end of 2026 subject to regulatory approval, and will make Fei-Fei Li an executive vice president and chief scientist reporting to CEO Lisa Su (AMD). TechCrunch reports that AMD expects World Labs' "frontier workloads" to shape its chip roadmap, and frames the purchase as a way to compete with NVIDIA, which already offers open world models through Cosmos (TechCrunch).

Venture money is moving the same way. Each round below was announced this year.

AMI Labs raised $1.03 billion at a $3.5 billion pre-money valuation, with NVIDIA, Samsung, Temasek and Toyota Ventures among the backers (TechCrunch). TechCrunch also reported that World Labs raised $1 billion, $200 million of it from Autodesk, months before agreeing to sell. Runway raised $315 million at a $5.3 billion valuation to pre-train its next world models, and Odyssey raised $310 million at a $1.45 billion valuation with Amazon and AMD Ventures involved.

Four motives keep showing up. Chipmakers want the workload: a DeepMind research director told TechCrunch that during a Project Genie session "there's a chip somewhere that's only yours" (TechCrunch).

Robotics teams need training data that's slow and risky to collect in the real world. 3D content is costly to build by hand. And some labs argue that language alone won't get AI into the physical world. Luma put it plainly when it raised $900 million last year: "To simulate the universe, LLMs and text is not enough." Notice how often NVIDIA and AMD's venture arm turn up on these cap tables. The companies selling compute are funding the companies that will burn it.

What you can actually use today

Strip out research previews and waitlists and the list gets short. Here's what's open to the public, or to any developer with a credit card, this year.

Prices come from the vendors' own pages (Marble pricing, Google AI plans). Two things jump out. Marble is the only product here that a non-developer can pay for and walk away with a file. And Project Genie, the most striking demo, is hard to justify on price unless you already pay for Ultra for other reasons. Google first opened it only to US subscribers over 18 and capped generations at 60 seconds, and it now lists more than 140 countries.

The matrix below compares the four you're most likely to try, on attributes we could confirm on vendor pages.

Marble gets a partial mark for robotics because World Labs pitches its World API to robotics teams, with hooks into simulators such as NVIDIA Isaac, while the consumer app isn't built for that work.

A one-week plan for trying world models

1. 1Open Marble's free plan and build a world from a photo of a real room. World Labs puts a full world at about five minutes and a draft at about 20 seconds (World Labs docs).
2. 2Build the same room from a text prompt. Walk through both versions and look for smeared or melting regions, which show where the model guessed.
3. 3If you pay for Google AI Ultra, rebuild the room in Project Genie. Walk away from a doorway, come back, and note what changed.
4. 4If you write code, pull a Cosmos 3 checkpoint from Hugging Face or try a hosted model on NVIDIA's catalog before you commit GPUs.
5. 5Write down one task you'd hand to a world model and the output it needs: video, a splat, a mesh or actions. The format decides the tool more than any demo does.

Not sure where to start? This tree routes you by goal in two questions or fewer.

Most paths that end with a person, rather than a robot, land on Marble. It's the cheapest way to get a world you can keep.

How to judge a world-model demo

World-model launches are cut for social video, and short edited clips flatter every model. A few checks cut through the gloss.

Start with duration. Google's own Genie 3 page says the model supports "a few minutes of continuous interaction, rather than extended hours" (Google DeepMind). Its predecessor, Genie 2, showed examples that mostly lasted 10 to 20 seconds. A 15-second clip tells you almost nothing either way.

Then look for a return trip. Memory is the hard part. Genie 3 recalls changes for about a minute, and Runway says GWM Worlds 2's long-term memory is imperfect. If the camera never swings back to a place it has already shown, ask why.

Check who's driving. A smooth cinematic path can be planned in advance, while a person pressing keys exposes lag. TechCrunch's hands-on with Project Genie found unresponsive controls and a character that sometimes walked through walls, even though stylized worlds such as claymation and watercolor looked good (TechCrunch).

Finally, separate looks from physics. Physics-IQ found the two unrelated in video models, so a demo full of sweeping scenery and no collisions is a rendering demo. Use the scorer on the next launch video you see.

A typical edited launch clip scores low here. Read that as marketing, and run your own hands-on test before you plan any work around the model.

The limits nobody has solved yet

Consistency is the first wall. These models generate each moment from what they've already produced, so small errors compound. TechCrunch's look at Decart's Oasis 3 found its driving world losing coherence over long sessions, cars passing through obstacles and scene memory degrading when it revisited areas (TechCrunch).

Cost is the second. Oasis 3 is priced at $0.02 per second, which works out to $1.20 a minute and $72 for an hour of simulated driving. Google's 60-second cap on Project Genie exists because each session ties up dedicated hardware. A finished game level, by contrast, costs almost nothing extra each time someone plays it.

Evaluation is the third, and the least discussed. Benchmarks are young, and several come from the same labs that publish the models. Meta's own physical-reasoning tests show humans scoring 85% to 95%, with a notable gap to the best models (Meta AI). NVIDIA claims first place on several world-generation leaderboards for Cosmos 3. Treat vendor rankings as vendor rankings.

The companies aren't saying much, either. TechCrunch reported this year that world-model startups are keeping product plans to themselves, and AMI Labs' CEO has said his company is not the kind of startup that ships a product in three months.

Who should wait? Anyone who needs a dependable production pipeline for games, film or buildings this year. World models make good drafts and reference material. They don't yet replace a game engine or a physics simulator your team has validated.

What changes for creators, game developers, architects and robotics teams

Creators get the most immediate value. Marble worlds export to Unreal Engine, Unity, Blender and Houdini through Gaussian splatting plugins, and you can record camera flythroughs as video. For previs and virtual sets, that's a real shortcut. Watch the license: commercial rights start on the $35 Pro plan, and the $20 Standard plan doesn't include them (Marble pricing). For single objects rather than whole scenes, the 3D model generators board is a better fit.

Game developers should treat world models as a blocking and concept tool for now. Marble's Chisel mode turns rough 3D shapes into a detailed world, which suits early level layout. Generated worlds carry no gameplay logic, tuned collisions or performance budgets, and those are what ship a game. The AI tools for game developers board tracks the assistants that cover the rest of the pipeline.

Architects and interior designers are next in line. Autodesk's chief scientist, Daron Green, told TechCrunch he could see customers starting with "a world-model-based sketch in World Labs (say, of an office layout)" and refining it in Autodesk tools. For client-ready work today, the AI interior and architecture design board and our interior design tools roundup cover more mature options.

Robotics software teams have the strongest case. Cosmos 3 is meant to be post-trained on your own camera and robot data and run as a closed-loop simulator. World Labs says its real-to-sim-to-real system turns one real task into thousands of simulated variations and tests each policy checkpoint on 2,000 simulated trials, with simulated scores tracking hardware scores. The payoff is screening: fewer weak checkpoints ever touch a physical robot. World Labs also connects its World API to simulators including Isaac Sim and MuJoCo, so a pilot doesn't mean replacing the simulator your team already trusts.

Frequently asked questions

Is ChatGPT a world model?

Not in the sense the labs mean. ChatGPT predicts text. It can describe what happens when a glass falls off a table, but it doesn't output a state you can act on step by step. LeCun's whole case for AMI Labs is that language models alone won't get there.

Can I try a world model for free?

Yes. Marble's free plan includes up to four world generations, and the weights for NVIDIA Cosmos 3 and Meta's V-JEPA 2 are free to download if you have suitable GPUs. Project Genie needs a paid Google AI Ultra plan.

Will world models replace game engines?

Not soon. Interactive generated worlds drift within minutes, and none of these tools carry game rules, so the likelier path is a hybrid: generate a draft world, export it, and finish it in Unreal, Unity or Blender. World Labs' export guides already assume that workflow.

What happens to Marble now that AMD is buying World Labs?

Neither company has said. The deal still needs regulatory approval and is expected to close by the end of 2026, and World Labs says Justin Johnson and Ben Mildenhall will keep leading its team inside AMD. If client work depends on Marble, export your worlds and keep local copies.

Is Sora still available?

No. OpenAI says the Sora product is no longer available: the web and app experiences were discontinued earlier this year, and the API was scheduled to follow later in 2026. For video generation, our best AI video generators guide covers the tools still on the market.