{"id":8342,"date":"2024-04-18T10:03:16","date_gmt":"2024-04-18T02:03:16","guid":{"rendered":"https:\/\/www.1ai.net\/?p=8342"},"modified":"2024-04-21T10:19:07","modified_gmt":"2024-04-21T02:19:07","slug":"%e4%bd%bf%e7%94%a8%e4%b8%80%e6%ac%be%e6%96%b0%e5%9e%8b%e7%9a%84%e6%8d%a2%e8%84%b8%e6%8f%92%e4%bb%b6instantid%e5%ae%9e%e7%8e%b0%e4%ba%ba%e7%89%a9%e8%a7%92%e8%89%b2%e4%b8%80%e8%87%b4%e6%80%a7","status":"publish","type":"post","link":"https:\/\/www.1ai.net\/en\/8342.html","title":{"rendered":"Use a new face-changing plug-in InstantID to achieve character consistency"},"content":{"rendered":"

I wrote an article before [AI Image Tool]InstantID<\/a>\u3011\uff1aCreate different styles of pictures from a single photo in seconds<\/a>\u3011InstantID has been introduced in detail. It can achieve personalized image synthesis using only a single facial image reference while maintaining high-fidelity identity preservation, and supports a variety of different styles.<\/p>\n

When I wrote that article, InstantID had just been out for a short time.Stable Diffusion<\/a>The plugin did not support it yet, but InstantID was soon introduced in the ControlNet plugin of Stable Diffusion.Model plugin<\/a>Today we will look at the use of the InstantID model in the ControlNet plug-in of Stable Diffusion.<\/p>\n

1. Introduction to InstantID<\/strong><\/p>\n

\"get-519\"<\/div>\n

InstantID uses InsightFace to detect, crop, and extract face embeddings from reference faces. The embeddings are then used with the IP-Adapter to control image generation. This part is very similar to IP-Adapter Face ID. However, it also uses ControlNet to detect and inpaint multiple facial landmarks (eyes, nose, and mouth).<\/p>\n

Using IP-Adapter Face ID in conjunction with ControlNet, we were able to replicate the reference images we provided with high fidelity, ultimately achieving consistency across the characters.<\/p>\n

Note: InstantID requires the use of the SDXL large model, and there is currently no corresponding version for Stable Diffusion 1.5.<\/p>\n

2. Installation of InstantID plugin<\/strong><\/p>\n

First, upgrade controlnet to the latest version, preferably V1.1.440 or above.<\/p>\n

\"get-520\"<\/div>\n

Download the IP-adapter model for InstantID.<\/p>\n

\n

Download address: https:\/\/huggingface.co\/InstantX\/InstantID\/blob\/main\/ip-adapter.bin<\/p>\n<\/blockquote>\n

Rename the downloaded file to ip-adapter_instant_id_sdxl.bin.<\/p>\n

Download the InstantID ControlNet model.<\/p>\n

\n

Download address: https:\/\/huggingface.co\/InstantX\/InstantID\/blob\/main<\/p>\n

\/ControlNetModel\/diffusion_pytorch_model.safetensors<\/p>\n<\/blockquote>\n

Rename the downloaded file to control_instant_id_sdxl.safetensors.<\/p>\n

Put the two files downloaded above in the stable-diffusion-webui > models > ControlNet folder directory. You can follow your own directory for specific situations.<\/p>\n

Then restart our SD tool, and you can see that there is an additional Instant_ID option in the control options, indicating that InstantID has been installed.<\/p>\n

\"get-521\"<\/div>\n

3. Use of InstantID plugin in SD 1.5<\/strong><\/p>\n

There are a few things to note when using the InstantID model:<\/p>\n

    \n
  • Large models can only use SDXL large models<\/li>\n
  • The prompt word guidance coefficient (CFG Scale) is lower than the usual parameter value (3-5)<\/li>\n
  • Need to use 2 ControlNet as InstantID<\/li>\n
  • Reduce the control weights and ending control steps of the two ControlNets<\/li>\n<\/ul>\n

    Let's take a closer look at how to use the InstantID plug-in.<\/p>\n

    [Step 1]: Selection of large model<\/strong><\/p>\n

    Here we use the Dreamshaper SDXL Turbo large model. Select None for the VAE large model.<\/p>\n

    \"get-522\"<\/div>\n

    \u3010Step 2\u3011\uff1aWriting prompt words<\/strong><\/p>\n

    Positive prompt words:<\/strong><\/p>\n

    \n

    Prompt: watercolors portrait of a woman (happy laughing:1.15),masterpiece,artistry,<\/p>\n

    Prompt words: Watercolor portrait of a woman (laughing happily: 1.15), masterpiece, art<\/p>\n<\/blockquote>\n

    Reverse prompt words:<\/strong><\/p>\n

    \n

    Prompt: low quality, blurry, malformed, distorted<\/p>\n

    Hint words: low quality, blurry, deformed, distorted<\/p>\n<\/blockquote>\n

    Related parameter settings of Wensheng diagram<\/p>\n

      \n
    • Sampler: DPM++SDE Karras<\/li>\n
    • Sampling iteration number: 7<\/li>\n
    • Image width and height: 1016*1016 (the default image size of 1024\u00d71024 does not work with Instant ID. Use a resolution close to but not exactly 1024\u00d71024).<\/li>\n
    • Cue word guidance coefficient (CFG): 2 (CFG ratio must be set quite low for InstantID to be effective)<\/li>\n<\/ul>\n
      \"get-523\"<\/div>\n

      [Step 3]: ControlNet plug-in InstantID model settings<\/strong><\/p>\n

      Two InstantID models need to be used. In other words, two ControlNets need to be used.<\/p>\n

      ControlNet Unit 0 Settings<\/strong><\/p>\n

      The first ControlNet mainly uses InsightFace to extract facial features of people.<\/p>\n

      Here we use the picture of Zixia Fairy as the reference image.<\/p>\n

      \"get-524\"<\/div>\n
        \n
      • Enable: Checked<\/li>\n
      • Pixel Perfect Mode: Checked<\/li>\n
      • Control Type: Instant_ID<\/li>\n
      • Preprocessor: instant_id_face_embedding<\/li>\n
      • Model: ip-adapter_instant_id_sdxl<\/li>\n
      • Control weight: 1<\/li>\n
      • Guided intervention time: 0<\/li>\n
      • Boot termination time: 1<\/li>\n<\/ul>\n

        ControlNet Unit 1 Settings<\/strong><\/p>\n

        The second ControlNet is used to extract facial key points, such as the location of the eyes, nose, and mouth. You can use different images, but it is recommended to use the same image. Here we still use the image uploaded by the first ControlNet.<\/p>\n

        \"get-525\"<\/div>\n
          \n
        • Enable: Checked<\/li>\n
        • Pixel Perfect Mode: Checked<\/li>\n
        • Control Type: Instant_ID<\/li>\n
        • Preprocessor: instant_id_face_keypoints<\/li>\n
        • Model: control_instant_id_sdxl<\/li>\n
        • Control weight: 0.45<\/li>\n
        • Guided intervention time: 0<\/li>\n
        • Boot termination time: 1<\/li>\n<\/ul>\n

          [Step 4]: Generate pictures<\/strong><\/p>\n

          Click the [Generate] button, and the final generated image will look like this.<\/p>\n

          \"get-526\"<\/div>\n
          \"get-527\"<\/div>\n

          4. Picture style<\/strong><\/p>\n

          The style of the image is controlled by the prompt words. You can use SDXL's prompt words to optimize the display content and background elements of the image.<\/p>\n

          1. Portraits<\/strong><\/p>\n

          \n

          Prompt<\/strong>\uff1ahigh quality,masterpiece,rich details,realistic photography,8k,high-definition image quality,<\/p>\n

          portrait of a woman,(happy laughing:1.15),<\/p>\n

          Prompt word<\/strong>: High quality, masterpiece, rich details, realistic photography, 8k, HD quality,<\/p>\n

          Portrait of a woman, (laughing happily: 1.15),<\/p>\n<\/blockquote>\n

          \"get-528\"<\/div>\n
          \"get-529\"<\/div>\n

          2. Cyberpunk style<\/strong><\/p>\n

          \n

          Prompt<\/strong>\uff1ahigh quality,masterpiece,rich details,realistic photography,8k,high-definition image quality,<\/p>\n

          Cyberpunk style,portrait of a woman,(happy laughing:1.15),<\/p>\n

          Prompt word<\/strong>: High quality, masterpiece, rich details, realistic photography, 8k, HD quality,<\/p>\n

          Cyberpunk style, portrait of a woman, (happy smile: 1.15),<\/p>\n<\/blockquote>\n

          \"get-530\"<\/div>\n
          \"get-531\"<\/div>\n

          3. Line art style<\/strong><\/p>\n

          \n

          Prompt<\/strong>\uff1ahigh quality,masterpiece,rich details,realistic photography,8k,high-definition image quality,<\/p>\n

          line art style,portrait of a woman,(happy laughing:1.15),<\/p>\n

          Prompt word<\/strong>: High quality, masterpiece, rich details, realistic photography, 8k, HD quality,<\/p>\n

          Line art style, portrait of a woman, (happy smile: 1.15),<\/p>\n<\/blockquote>\n

          \"get-532\"<\/div>\n
          \"get-533\"<\/div>\n

          In the test example here, I changed the expression of the character in the original picture (smile). Overall, the Instant_ID plug-in is good at achieving face replacement and maintaining character consistency.<\/p>\n

          Okay, that\u2019s all for today\u2019s sharing. I hope that what I shared today will be helpful to you.<\/p>","protected":false},"excerpt":{"rendered":"

          Previously wrote an article [AI image tool [InstantID]: a single photo second generation of different styles of images] introduced InstantID in detail. it can be achieved while maintaining a high degree of fidelity identity preservation, using only a single facial image reference can be achieved personalized image synthesis, and support a variety of different styles. At the time of writing, InstantID had not been out long enough for the Stable Diffusion plugin to support it, but the InstantID modeling plugin was soon to be introduced in Stable Diffusion's ControlNet plugin. Today we'll take a look at the InstantID modeling plugin in Stable Diffusion's ControlNet plugin.<\/p>","protected":false},"author":1,"featured_media":0,"comment_status":"open","ping_status":"open","sticky":false,"template":"","format":"standard","meta":{"footnotes":""},"categories":[149,144],"tags":[1221,197,198,2283],"collection":[262],"class_list":{"0":"post-8342","1":"post","2":"type-post","3":"status-publish","4":"format-standard","5":"hentry","6":"category-jiaocheng","7":"category-baike","8":"tag-instantid","9":"tag-stable-diffusion","11":"tag-2283","12":"collection-stablediffusion"},"acf":[],"_links":{"self":[{"href":"https:\/\/www.1ai.net\/en\/wp-json\/wp\/v2\/posts\/8342","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/www.1ai.net\/en\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/www.1ai.net\/en\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/www.1ai.net\/en\/wp-json\/wp\/v2\/users\/1"}],"replies":[{"embeddable":true,"href":"https:\/\/www.1ai.net\/en\/wp-json\/wp\/v2\/comments?post=8342"}],"version-history":[{"count":0,"href":"https:\/\/www.1ai.net\/en\/wp-json\/wp\/v2\/posts\/8342\/revisions"}],"wp:attachment":[{"href":"https:\/\/www.1ai.net\/en\/wp-json\/wp\/v2\/media?parent=8342"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/www.1ai.net\/en\/wp-json\/wp\/v2\/categories?post=8342"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/www.1ai.net\/en\/wp-json\/wp\/v2\/tags?post=8342"},{"taxonomy":"collection","embeddable":true,"href":"https:\/\/www.1ai.net\/en\/wp-json\/wp\/v2\/collection?post=8342"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}