World Models, from first principles

A linear path from pixels to prediction to decisions. Build the hidden state an agent needs, learn how that state changes under actions, represent futures that branch, and use the model to imagine, plan, and control—without confusing a beautiful video with a trustworthy world.

The first-principles frame

The environment owns a hidden state s_t. Sensors reveal only an observation o_t; the agent changes the world with a_t. Because the same image can hide different velocities, objects, intentions, or occluded geometry, one frame is rarely a sufficient state. The agent must infer a belief b_t from history, implement that belief with a compact learned state x_t, and learn how it evolves:

Read the equation as a machine

The compact notation above can make the subject look more mysterious than it is. Read it left to right as four concrete computations. First, an encoder or filter looks at the observations and actions seen so far. It produces a belief state, not a photograph: the state may contain an object’s likely position, a distribution over its hidden velocity, a memory that it still exists behind a wall, and the uncertainty attached to each claim. Second, a transition model accepts that belief and a proposed action. It predicts a distribution over next beliefs. Third, a collection of heads or decoders turns the predicted belief into the quantities a product can check or use: future pixels, depth, reward, collision risk, termination, object poses, or value. Finally, reality supplies the next observation, and the filter corrects the prediction.

1. Infer before you predict: E. The agent first asks, “given everything I have seen and done, what is probably true now?” A Bayesian filter, RSSM posterior, causal transformer, tracker, or multimodal fusion stack can implement this step. The name is secondary. The decisive test is whether hidden information survives occlusion, sensor delay, and noise. If the current belief is wrong, every later prediction begins from the wrong world.
2. Keep only the state the future needs: x_t. The full history is too large to carry forever, so inference must produce a compact implementation of belief. It may be a vector, stochastic latent, token memory, object set, BEV grid, 3D field, or hybrid. To test the compression, probe whether the variables needed by future decisions remain recoverable. Compression is useful only when it discards nuisance detail rather than causal state.
3. Change that state under a proposed action: F. Now the model can ask what follows if the agent turns, pushes, brakes, or waits. A state-space transition, transformer, diffusion trajectory model, graph network, or physics-plus-residual model can implement the update. Hold the inferred state fixed and change only the action: if the predicted consequence does not change correctly, the system learned correlation rather than control.
4. Turn the imagined state into a decision answer: G. A decoder or head reads future pixels, occupancy, collision risk, reward, termination, or value from state. This step exists because a model is always built for questions. The final check is not whether the head produces a number, but whether that number is calibrated and sufficient to change the product’s decision correctly.

The dependency is strict. A query head cannot recover state that the encoder discarded. A powerful transition cannot repair an incorrect belief. A planner cannot make an action causal by searching it more aggressively. Read the architecture from left to right and debug it from right to left: begin with the failed decision, inspect the queried output, trace the imagined transition, and finally ask whether the starting belief contained the missing fact.

This decomposition is deliberately architecture-neutral. “Transformer,” “diffusion,” “JEPA,” “RSSM,” and “Gaussian” are choices inside the boxes; none defines the contract. Starting from the contract prevents the common mistake of choosing a fashionable model before specifying what state must remember, which interventions it must predict, and how its output will be judged.

A diagnostic motif: the object behind the wall

A minimal occlusion world exposes nearly every hard problem without hiding behind scale: an object moves, disappears behind a wall, and may be acted on while hidden. The lessons return to this motif, then switch to domain-specific worked examples when a forklift, vehicle, kitchen robot, or embodied assistant makes the mechanism more concrete. Lessons 14–16 share the Countertop Assistant as a final systems capstone.

1. Before occlusion, infer motion. Two or more observations are needed to distinguish position from velocity. Lesson 02 turns those frames into a belief over hidden state rather than pretending the estimate is exact.
2. Compress the history. The model should remember position, velocity, identity, and uncertainty, but it need not preserve every texture pixel on the table. Lesson 03 derives this predictive bottleneck.
3. Roll state forward. While the ball is hidden, the transition model integrates velocity and possible forces. Lesson 04 builds the prior/posterior machinery; lesson 05 represents several plausible emergence points instead of averaging them.
4. Survive self-generated inputs. A one-pixel velocity error becomes a large position error after many steps. Lesson 06 separates one-step validation from genuine free rollout.
5. Predict an intervention. Watching the ball move does not identify what a push will cause. Lesson 07 asks for action coverage and causal effects.
6. Choose behavior. The agent can train a policy in imagined trajectories (lesson 08) or search candidate pushes online (lesson 09).
7. Preserve identity and geometry. A persistent object representation and a stable 3D frame prevent the ball from being forgotten or confused with camera motion (lessons 10–11).
8. Scale and verify. Video and multimodal models make the toy world realistic, but lessons 12–16 insist that visual quality, compute, uncertainty, causal fidelity, and closed-loop success be measured separately.

Three neighboring fields—and the boundary

Do not separate these fields by how their demos look. Separate them by the direction in which information moves, then ask what operation is still missing.

1. Graphics moves from a known scene to an observation. Geometry, materials, lights, and a camera enter; pixels leave. A renderer can therefore serve as the observation decoder p(o|s) inside a world model. By itself it does not infer which hidden scene caused real pixels, and it does not learn what an action changes.
2. Vision and 3D move from observations toward a scene estimate. They recover geometry, tracks, pose, motion, and object permanence. That gives the agent a more stable present. But a beautiful reconstruction may still be static: it can say where the mug is without saying what a push will do.
3. Video generation moves from context toward plausible future observations. Its generative objective is valuable because the future can branch. Yet a plausible movie may ignore the action, violate object identity, or assign the wrong probability to a rare collision. Visual realism is evidence about observation synthesis, not causal control.
4. Model-based reinforcement learning moves from a model toward value and action. Imagination and planning turn predicted consequences into behavior. This is the consumer of the model, not a repair mechanism: search will actively exploit any dynamics or reward error it can find.

A complete agent closes the directions into one loop: vision infers a belief, dynamics advances it under action, graphics or task heads expose predicted consequences, planning chooses, and the next real observation corrects the belief. The world model is the persistent, action-conditioned middle of that loop.

The tensions that organize the field

World-model methods often look like a zoo because they choose different points on a small number of axes. Keep these tensions visible and the method names become consequences rather than facts to memorize.

Resolve the apparent method zoo by asking six product questions in order:

1. Must the product render, or only decide? If people need editable or photorealistic observations, prediction in pixel or video space preserves appearance and supplies dense supervision. If the consumer only needs reward, risk, or control, a latent can discard nuisance detail and make thousands of rollouts affordable.
2. Which queries and invariances must be cheap? A distributed vector is flexible when the model can learn its own organization. Objects, maps, and 3D fields are worth their structure when identity, locality, collision, or viewpoint change would otherwise need to be rediscovered by every head.
3. Can unresolved causes change the decision? If all relevant futures are effectively determined, a deterministic state is simple and fast. If hidden intent, contact, or random events produce distinct valid outcomes, a stochastic belief must keep those branches separate rather than average them.
4. Will goals and queries change later? An observation-complete model spends capacity explaining the sensor stream and can support new heads. A value-equivalent model is smaller because it preserves only what the present task and search need. Task stability determines whether that bargain is safe.
5. When can the system afford to think? Amortized policy learning pays imagination cost during training and acts quickly later. Online planning pays search cost for each decision but adapts to a new goal or constraint. Latency, goal churn, and compute availability decide the split.
6. Which laws are known well enough to encode? Explicit geometry and physics give invariants, data efficiency, and inspectable failure. Learned dynamics absorbs friction, human behavior, and other residual complexity. A hybrid starts from the laws we trust and learns where reality refuses to match the simulator.

No row has a universally correct side. An interactive video generator may need continuous appearance-rich trajectories; a control policy may need a tiny value-equivalent latent; a driving system may combine a metric BEV map, tracked objects, learned intent distributions, and a verified emergency controller. Each lesson returns to the same discipline: start from the decision and missing state, then choose the least expensive representation that keeps the necessary counterfactuals correct.

Why this order

The line is forced. Observations are partial, so we need belief state. Full history is expensive, so state must be compressed. A useful state must predict change, futures branch, and small one-step errors compound. Actions then force a causal distinction between watching and intervening. Only after that contract is credible can imagination or planning safely consume the model. Objects, geometry, video, and multimodal embodiment are structured implementations of the same contract; scaling and evaluation decide whether any of them can ship.

1. Lessons 01–03 build state. We first separate the world from its observation. Partial observability then forces a belief over hidden possibilities, and the cost of retaining full history forces a compact predictive latent. By the end of this stage, the agent has a defensible answer to “what do I currently believe?”
2. Lessons 04–07 build trustworthy futures. A state becomes useful only when it advances. We derive learned dynamics, preserve branches of uncertainty, measure compounding rollout error, and finally distinguish passive correlation from the causal effect of an intervention.
3. Lessons 08–09 turn futures into decisions. Once action-conditioned rollouts are credible, the system can reuse them in two ways: train an actor and critic in imagination, or search candidate actions online and repeatedly correct with reality.
4. Lessons 10–14 give the world durable structure. Object identity, spatial memory, 3D/4D coordinates, video generation, playable real-time interaction, language, touch, and proprioception are not separate definitions of a world model. They are answers to specific persistence and grounding failures in the same belief–dynamics loop.
5. Lessons 15–16 decide whether the loop can ship. The final stage budgets trajectory data, tokens, training compute, rollout throughput, and latency, then evaluates state, uncertainty, causal fidelity, exploitation, safety, and closed-loop value without hiding weakness inside an average score.

The lessons

How to read the series

1. Keep state and observation separate. Pixels can change when the camera moves while the world stays fixed; the world can change behind an occluder while pixels stay identical.
2. Ask “under which action?” A plausible unconditional future is not yet a controllable model.
3. Test open loop and closed loop. One-step validation measures local imitation. Decisions require rollouts, correction, and recovery.
4. Choose the abstraction from the decision. A photorealistic simulator may waste compute; a crude latent can be sufficient if it preserves reward, risk, and controllability.