Sonic

Sonic is an audio-driven portrait animation framework from Tencent and Zhejiang University that generates realistic facial expressions and movements based on global audio perception.Sonic is based on context-enhanced audio learning and motion decoupling controllers, which extract long-term temporal audio knowledge within the audio clip and independently control the head and expression movements, respectively, to enhance local audio perception.Sonic uses a temporal-aware position offset fusion mechanism to extend local audio perception to global and solve the jitter and mutation problems in long video generation.Sonic outperforms the existing ones in terms of video quality, lip-synchronization accuracy, motion diversity, and temporal coherence ...