May 13 News.Kunlun WanweiIt's just been announced. Matrix-Game Large Model(17B+) OfficialOpen SourceThis is a large model of interactive video generation in the Matrix-Zero world model.
Kunlun says Matrix-Game is the official landing of Matrix series in the direction of interactive world generation, and it is also the industry's first open source 10B+ spatial intelligence large model, which is an interactive world base model for game world modeling, designed for high-quality generation and precise control in open environments.
According to the introduction, Matrix-Game includes three main cores:
- Matrix-Game-MC datasetThe large-scale interactive world dataset, built in-house, consists of two types of data: large-scale unlabeled Minecraft gameplay videos, and Minecraft and Unreal controllable video data with keyboard and mouse control signals with fine-grained motion annotations. The dataset supports efficient modeling and learning of complex environmental dynamics and interaction patterns.
- Matrix-Game Master Model: An image-to-world generation framework based on advanced diffusion modeling techniques, capable of generating coherent and controllable interactive videos based on user inputs (keyboard commands, mouse movements, etc.), taking into account visual quality, temporal consistency, and physical reasonableness.
- GameWorld Score Rating SystemThe proposed unified game interaction world evaluation standard comprehensively quantifies the model performance from four dimensions: visual quality, timing quality, action controllability and physics rule understanding of the video, which fills the gap of the lack of systematic evaluation benchmarks in this field.
Matrix-Game is capable of controlled generation in different Minecraft scenes (e.g. forests, beaches, deserts, glaciers, rivers, plains, etc.), including basic movement, composite movement, perspective movement, and so on. For example, in a desert scene, Matrix-Game can generate the corresponding game world video according to any control commands input by the user (note: W / A / S / D arrow keys on the keyboard, Space key for jumping, Attack key for attacking, and the mouse for perspective movement), and support dynamic behaviors such as backward and forward, left and right, jumping, attacking, and perspective change of the character. behavior.
On this basis, Matrix-Game supports autoregressive long video generation, which not only realizes silky smooth articulation between actions and perspectives, but also excels in temporal consistency and environmental adaptability, laying a solid model foundation for the development of applications such as immersive long-time experiences, creative content generation and game design.
References
-
Project home page:https://matrix-game-homepage.github.io
-
Technical report:https://github.com/SkyworkAI/Matrix-Game/blob/main/assets/report.pdf
-
GitHub open source address:https://github.com/SkyworkAI/Matrix-Game
-
HuggingFace open source address:https://huggingface.co/Skywork/Matrix-Game