August 6 News.Google DeepMind officials released a blog post yesterday (August 5) announcing the launch of a new program called Genie 3 of"world model", a model that enables AI systems to interact with realistic real-world simulations toward general-purpose artificial intelligence (AGI) to take an important step forward.
According to Google, Genie 3 can be used to train robots and self-driving vehicles to interact in realistic recreations of environments such as warehouses.
Experts at Google's DeepMind Artificial Intelligence Division believe that world modeling is a key step in realizing AGI, a hypothetical level of artificial intelligence in which a system does more than just a single task such as playing chess or translating a language.Rather, they are capable of performing most tasks on par with humans and may replace certain jobs.DeepMind says these models will play an important role in developing AI agents or systems that perform tasks autonomously.
Compared to its predecessor models (e.g. Genie 1/2) and video-generated models (e.g. Veo 2, Veo 3's deep understanding of intuitive physics), Genie 3 is the first model of the world to allow real-time interactions, while at the same time improving its consistency and realism compared to Genie 2.
Its core competencies include:
- Physical properties of the simulated world: Genie 3 has a deep understanding of the laws of physics and can realistically simulate water flow, changes in light and shadow, and complex environmental interactions, such as a helicopter maneuvering carefully around a cliff waterfall.
- Simulation of the natural worldFrom the vibrant ecosystems of a glacial lake to the adorable furry creatures jumping over a rainbow bridge in a fantasy world, Genie 3 transforms imagination into explorable reality!
- Animation and fiction modeling: Imagination can be utilized to create fantastical scenes and expressive animated characters.
- Explore different locales and historical scenesThe model transcends the limits of geography and time, taking the user on an exploration of different places and eras of history, whether it's flying over a snowy mountain in a wingsuit or immersing yourself in an ancient city steeped in history!
- Pushing the limits of real-time performance: To achieve a high degree of controllability and real-time interactivity, during the autoregressive generation of each frame, the model must take into account previously generated trajectories that grow over time. For example, if a user revisits a location after one minute, the model must refer to relevant information from one minute ago. To achieve real-time interactivity, this computation must be performed multiple times per second in response to the arrival of new user inputs
- Long-term environmental coherence: In order for AI-generated worlds to be immersive, they must remain physically consistent over long periods of time. However, automated regression generation of environments is often a more difficult technical problem than generating an entire video, as inaccuracies tend to accumulate over time. Genie 3 environments remain largely consistent over a period of minutes, with visual memory going back a minute, and Genie 3-generated worlds are more dynamic and richer because they are created frame-by-frame based on the user's description of the world and their actions.
- Promptable world events (Promptable World Events: In addition to navigational input, Genie 3 supports a more expressive form of text-based interaction called Promptable World Events. Promptable World Events can alter the generated world, such as changing weather conditions or introducing new objects and characters, to enhance the experience of navigational control, and the ability to add breadth to counterfactual or "what-if" scenarios, which agents can use to learn from experience and deal with unexpected situations.
However, Google said Genie 3 is not yet ready for full public availability and did not give a specific release date, while also noting that the model has some limitations. The statement comes against the backdrop of an increasingly competitive AI market.
Google says its world model can also help humans experience a variety of simulations for training or exploration, such as skiing or walking around a mountain lake.DeepMind says Genie 3 is able to create scenarios instantly based on textual cues, and the simulated environments can be changed quickly, for example, by textually cueing the introduction of a herd of deer on a ski slope.
Google on Monday showed reporters ski and warehouse scenes created by Genie 3, but has not yet made the models public. According to The Guardian, the quality of the simulations is comparable to Google's latest video creation model, the Veo 3, but they last eight seconds longer than the Veo 3.
While AGI is seen as potentially eliminating white-collar jobs because autonomous systems can perform roles ranging from sales agents to lawyers or accountants, Google sees world modeling as a key technology for developing robots and self-driving vehicles. For example, a recreation of a warehouse with real physics and people can help train a robot as it "learns" from the simulation during training, helping it achieve its goals.