AniPortrait

AniPortrait is a project that generates talking and singing dynamic videos based on audio and image input. It can generate realistic facial animations based on audio and static face images, with consistent lip shape. It supports multiple languages and face redrawing and head posture control. Features include audio-driven animation synthesis, face reproduction, head posture control, support for self-driven and audio-driven video generation, high-quality animation generation, and flexible model and weight configuration. Demand group: "Used to generate dynamic videos or animations, which can achieve effects such as human face talking and singing." Use...

AniPortrait is an innovative project that generates high-quality animations based on audio and a photo of a reference face. The project works in two steps. 1. AniPortrait extracts 3D intermediate data from the audio and converts this data into a sequence of 2D facial contour markers. This process is the initial phase of the project and lays the foundation for subsequent face animation generation. 2. The project utilizes a powerful diffusion model and combines it with a motion module to transform these facial contour markers into both realistic and smooth face animations. This step is the project...