{"id":13766,"date":"2024-06-22T09:36:00","date_gmt":"2024-06-22T01:36:00","guid":{"rendered":"https:\/\/www.1ai.net\/?p=13766"},"modified":"2024-06-22T09:36:00","modified_gmt":"2024-06-22T01:36:00","slug":"%e5%a4%8d%e6%97%a6%e5%bc%80%e6%ba%90%e9%a1%b9%e7%9b%aehallo%e5%b7%b2%e9%80%82%e9%85%8dcomfyui%e6%8f%92%e4%bb%b6","status":"publish","type":"post","link":"https:\/\/www.1ai.net\/en\/13766.html","title":{"rendered":"Fudan open source project Hallo has been adapted to ComfyUI plug-in"},"content":{"rendered":"<p>Fudan open source project<a href=\"https:\/\/www.1ai.net\/en\/tag\/hallo\" title=\"[See article with [Hallo] label]\" target=\"_blank\" >Hallo<\/a>, a project that generates talking videos based on audio and images, has adapted the<a href=\"https:\/\/www.1ai.net\/en\/tag\/comfyui\" title=\"_Other Organiser\" target=\"_blank\" >ComfyUI<\/a>Plug-ins. Although the installation process requires more dependencies and has a relatively high threshold, the emergence of this open source ecosystem opens up more possibilities and fun for subsequent repainting and other processes.<\/p>\n<p><img loading=\"lazy\" decoding=\"async\" class=\"alignnone size-full wp-image-13767\" src=\"https:\/\/www.1ai.net\/wp-content\/uploads\/2024\/06\/6385457642088722919561146.png\" alt=\"\" width=\"593\" height=\"313\" \/><\/p>\n<p>The Hallo project allows facial photographs to start talking through audio input, accompanied by corresponding expressions, and the effect looks very natural. The project employs an end-to-end diffusion paradigm that introduces a layered audio-driven visual synthesis module to improve the accuracy of the alignment between the audio input and the visual output, including the movement of lips, expressions, and poses.<\/p>\n<p>This layered audio-driven visual synthesis module provides adaptive control over the diversity of expressions and poses, enabling more effective personalization for different identities. This means that no matter who's face is pictured, a talking video can be generated through the Hallo project with a natural effect, as if a real person is talking.<\/p>\n<p>Although the installation process of the Hallo project may be relatively complicated, its emergence has certainly brought new vitality to the open source ecosystem. As technology continues to evolve, we can expect more such projects to emerge in the future, bringing more convenience and fun to our lives.<\/p>\n<p>Plugin address: https:\/\/github.com\/AIFSH\/ComfyUI-Hallo<\/p>","protected":false},"excerpt":{"rendered":"<p>The Fudan open source project Hallo, a project that generates talking videos based on audio and images, has been adapted with the ComfyUI plugin. Although the installation process requires more dependencies and a relatively high threshold, the emergence of this open source ecosystem provides more possibilities and fun for subsequent transcription and other processes. The Hallo project allows facial photos to start talking through audio input, accompanied by corresponding expressions, and the effect looks very natural. This project employs an end-to-end diffusion paradigm that introduces a layered audio-driven visual synthesis module to improve the accuracy of the alignment between the audio input and the visual output, including the movement of lips, expressions, and poses. This layered audio-driven visual synthesis module provides adaptive control over expression and pose diversity, more efficiently realizing the<\/p>","protected":false},"author":1,"featured_media":0,"comment_status":"open","ping_status":"open","sticky":false,"template":"","format":"standard","meta":{"footnotes":""},"categories":[146],"tags":[1989,3171,2284],"collection":[],"class_list":["post-13766","post","type-post","status-publish","format-standard","hentry","category-news","tag-comfyui","tag-hallo","tag-2284"],"acf":[],"_links":{"self":[{"href":"https:\/\/www.1ai.net\/en\/wp-json\/wp\/v2\/posts\/13766","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/www.1ai.net\/en\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/www.1ai.net\/en\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/www.1ai.net\/en\/wp-json\/wp\/v2\/users\/1"}],"replies":[{"embeddable":true,"href":"https:\/\/www.1ai.net\/en\/wp-json\/wp\/v2\/comments?post=13766"}],"version-history":[{"count":0,"href":"https:\/\/www.1ai.net\/en\/wp-json\/wp\/v2\/posts\/13766\/revisions"}],"wp:attachment":[{"href":"https:\/\/www.1ai.net\/en\/wp-json\/wp\/v2\/media?parent=13766"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/www.1ai.net\/en\/wp-json\/wp\/v2\/categories?post=13766"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/www.1ai.net\/en\/wp-json\/wp\/v2\/tags?post=13766"},{"taxonomy":"collection","embeddable":true,"href":"https:\/\/www.1ai.net\/en\/wp-json\/wp\/v2\/collection?post=13766"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}