AI has evolved yet again. Just a few months ago, AI video generation was plagued by impossible physics and a lack of scene consistency that made AI videos…less than immersive. But world models are making these problems a thing of the past.
A world model is any generative AI model that can maintain the consistency of physical and spatial properties. With real-world physics, the model “knows” that water splashes and ripples when you drop a rock into it. It also knows that people don’t randomly grow an extra arm to catch a ball thrown at them.
In other words, world models account for spatial relationships, force, and motion. They offer increased setting and character consistency. To see a world model in action, watch the beautiful video below, made by Henry Daubrez with the help of Google Veo2.
How World Models Work
Researchers have designed world models to match our understanding of how human brains work. Humans develop a working model of the world based on what they experience. They expect current and future events to match the model based on past experience.
In other words, you know that a dropped rock will fall but a released balloon will fly because you’ve seen it happen. When you’re in a new situation, you can often extrapolate what might happen based on what you’ve seen before.
World models can do something similar. They develop a stable environment with consistent physics and clear cause and effect. This allows the model to remain consistent over time and make informed extrapolations about what might happen next. As a result, motion is smoother and elements within the model react to each other in predictable ways.
In short, world models solve many of the issues that have kept video pros from using generative AI thus far.
The Evolution of World Models
Chat GPT was built as a large language model, a method that allowed it to essentially guess at the right words and sentences based on a huge volume of training content. Then, the big news in 2023-2024 was the development of useful generative AI that could recognize patterns in huge amounts of data, and use that information to respond to a prompt.
Image and video generation tools like Dall-E 2 and Runway Gen-2 were built on diffusion models. These models essentially started with static and then refined it into a picture that matched the prompt. Because each render essentially starts from scratch, diffusion-model-based tools lack consistency from scene to scene and sometimes stray into uncanny valley territory.
In December 2023, Runway announced a long-term research effort into “general world Models.” Long is apparently a relative term in the AI industry, because progress has been swift. By the end of 2024, Google DeepMind had announced Genie 2 a “large-scale foundation world model.”
Moving from diffusion models to world models means that the world can stay consistent from shot to shot and scene to scene. Instead of rebuilding a world from scratch every time, the system creates a consistent world and allows users to manipulate elements within it.
This technological breakthrough potentially unlocks a range of uses for AI that go far beyond creating commercials and feature films.
How AI World Models Could Be Useful Beyond Video Creation
As video creators, we’re most interested in how AI world models could change the video creation process. But this new method of AI image and graphics generation could potentially open up all kinds of opportunities.
NVIDIA launched Cosmos, a physical AI, in January 2025. This “world foundation model” is designed to allow modeling of real-world environments, like warehouses and city streets. This technology could help create smarter warehouse robots or even self-driving cars. The demo video below includes a few glimpses of modeled worlds.
Room for Improvement in World Modeling
With a technology this new, it’s no surprise that there’s still plenty of room for improvement. Overall, world models are a major advance over previous modeling methods, but they still have drawbacks.
Although movement looks far more natural when generated from a world model, it’s not always perfect. Take a close look at the fox in the animation we shared above, and you may notice that it seems to be skimming over the grass rather than walking through it. Issues like these are solvable as the technology evolves.
What may be more difficult for people to manage is the amounts of data and energy needed to build and use world models. Companies have not released exact figures, but one team estimated that the carbon footprint of a single task given to a large language model was roughly equivalent to that of a transatlantic flight. World models are much more complicated, and consume even more energy to maintain.
The New World of AI Modeling Is Here to Stay
Ultimately, there’s no putting the genie back in the bottle. AI world models offer opportunities to advance science, technology, safety, and video generation. We’ll be watching with interest to see how this new frontier evolves.
Even so, we believe that human vision and creativity will always be an essential component of video creation. If you’re ready to start making your next commercial or explainer video, contact the video production experts at IdeaRocket.
