World Models
A world model learns to predict how an environment evolves, letting an agent train and plan inside its own simulation.
Learning to imagine
A world model is a learned generative model of environment dynamics: given the current state and an action it predicts the next state and reward. An agent equipped with one can generate synthetic experience, evaluate plans, and reduce how much real interaction it needs.
A common architecture
The classic design has three parts: a vision module (often a variational autoencoder) that compresses observations into a latent code z; a memory module (a recurrent network, sometimes with a mixture-density output) that predicts the next latent given the action; and a small controller that maps the latent state to actions. Because the controller sees a compact latent, it can be tiny and trained cheaply, even by evolution strategies.
Training in a dream
Once the model is accurate enough, the controller can be optimized entirely on rollouts generated by the model itself, sometimes called training inside a dream. Policies learned this way must then be validated on the real environment, because model errors compound over long horizons.
- Sample efficiency: real interactions are reused to fit the model, then reused again as imagined rollouts
- Planning: the model supports lookahead search without touching the real system
- Representation: the latent is a useful state abstraction for the policy
The compounding-error problem
World models are only as good as their predictions. Small one-step errors accumulate over an imagined trajectory, so a policy can exploit the model's fantasies and fail in reality. Modern systems limit imagined horizons, quantify model uncertainty, and interleave real data to keep the model honest. Latent-space recurrent models such as those behind MuZero and the Dreamer family are direct descendants of this idea.