The news of September 25thNvidiaYesterday, 24 September, a Boeparture was published and announcedOpen SourceGENERATING AI ANIMATION MODEL Audio2FaceIT COVERS MODELS, SOFTWARE DEVELOPMENT TOOLKITS (SDK) AND A COMPLETE TRAINING FRAMEWORKHOPE TO ACCELERATE THE DEVELOPMENT OF AI SMART VIRTUAL CHARACTERS IN GAME AND 3D APPLICATIONS。
By analysing the acoustic characteristics of audio, speech, etc. in the audio, driving virtual role facial actions in real time, the technology produces accurate oral synchronization and natural emotional expressions, which can be widely applied in areas such as games, video production and customer services。
Audio2Face supports two operating modes: pre-recorded offline rendering of audio and real-time current processing of dynamic AI roles. 1AI quotes from Boven, and Young Wei Da has several core components at the opening:
- Audio2Face SDK
- A 2.0 local execution plugin for Autodesk Maya
- Unreal Engineering 5.5 and 2.5 plugins above
- return model (v2.2)
- proliferation model (v3.0)
- The Open Source Training Framework supports developers in using their own data fine-tuning models to adapt to specific application scenarios。
The technology has been widely adopted by industry. The game developer, Survios, has integrated Audio2Face in Alien: Rogue Incursion Evolution (Alien: Rogue Involved Evolution), which greatly simplifys the oral synchronization and facial capture process。
Farm 51 has also been used in Chernobyl People 2: Forbidden Zone (Chernobylite 2: Exclusion Zone), which directly produces fine facial animations through audio, saves considerable production time, enhances role realism and insinuation. The Director of Innovation, Wojciech Pazdur, called it a “revolutionary breakthrough”。