{"id":13308,"date":"2024-06-16T10:51:31","date_gmt":"2024-06-16T02:51:31","guid":{"rendered":"https:\/\/www.1ai.net\/?p=13308"},"modified":"2024-06-16T10:51:31","modified_gmt":"2024-06-16T02:51:31","slug":"%e7%94%a8ai%e7%94%9f%e6%88%90%e5%9b%be%e7%89%87%ef%bc%8c%e6%95%99%e4%bd%a0%e7%94%a8comfyui%e6%9c%ac%e5%9c%b0%e9%83%a8%e7%bd%b2stable-diffusion-3","status":"publish","type":"post","link":"https:\/\/www.1ai.net\/en\/13308.html","title":{"rendered":"Generate pictures with AI and teach you how to deploy Stable Diffusion 3 locally with ComfyUI"},"content":{"rendered":"

aboutStable diffusion 3<\/strong>The advantages will not be elaborated here, here we will mainly talk about how ordinary users can deploy it locally.<\/p>\n

Currently, the SD3 model has been open sourced in HuggingFace. The address is:https:\/\/huggingface.co\/stabilityai\/stable-diffusion-3-medium<\/strong><\/p>\n

However, to download the model, you need to log in to your Hugging Face account and sign a license agreement.<\/p>\n

\"get-513\"<\/div>\n

After signing, you can see the file list of the entire project.<\/p>\n

\"get-514\"<\/div>\n

The above file can be roughly divided into three parts:<\/p>\n

1. comfy_example_workflows<\/p>\n

This part mainly contains three Comfyui workflow files, which will be used below.<\/p>\n

The reason why Comfyui is used is that the current (as of the time of article publication) Stable diffusion webui does not support SD3, so it is not possible to run SD3's large model. I also tried to load sd3_medium.safetensors with the current Stable diffusion webui, but it reported this error.<\/p>\n

\"get-515\"<\/div>\n

Comfyui currently supports it, but you need to pay attention to upgrading the kernel version. Lower versions are not supported. As you can see from the picture below, the June 11 version has just started to support SD3 (it can be seen that Comfyui received the news very early).<\/p>\n

\"get-516\"<\/div>\n

I personally don't recommend just upgrading to version 8c4a9be, because the subsequent release notes actually contain a lot of updates about SD3, which shows that 8c4a9be is just a relatively rough version. I personally upgraded directly to the latest version.<\/p>\n

2. text_encoders<\/p>\n

There are four model files in text_encoders, namely:<\/p>\n

\u251c\u2500\u2500 text_encoders\/ \u2502 \u251c\u2500\u2500 clip_g.safetensors \u2502 \u251c\u2500\u2500 clip_l.safetensors \u2502 \u251c\u2500\u2500 t5xxl_fp16.safetensors \u2502 \u2514\u2500\u2500 t5xxl_fp8_e4m3fn.safetensors<\/code><\/pre>\n
There are three different text encoders here, namely two CLIP models and a T5 model. The T5 model also provides two quantization versions, one 16-bit quantization (fp16) and the other 8-bit quantization (fp8).<\/p>\n
Let me add here,CLIP model[1]<\/strong>Originally developed by OpenAI, it is a multimodal pre-trained model that can understand the relationship between images and text. CLIP is trained on a large number of image and text pairs to learn a representation method that can align text descriptions with image content. This representation method enables CLIP to understand the content of the text description and match it with the image content.<\/p>\n
\"get-517\"<\/div>\n
In simple terms,CLIP[2]<\/strong>It is to convert our prompt words into a "language" (vector) that the SD model can understand, so that the SD model knows what kind of pictures to generate.<\/p>\n
3. Checkpoints<\/p>\n
This time, SD3 released a total of four large models, namely:<\/p>\n
\u251c\u2500\u2500 sd3_medium.safetensors (4.34G) \u251c\u2500\u2500 sd3_medium_incl_clips.safetensors (5.97 G) \u251c\u2500\u2500 sd3_medium_incl_clips_t5xxlfp8.safetensors (10.9G) \u2514\u2500\u2500 sd3_medium_incl_clips_t5xxlfp16.safe tensors (15.8G)<\/code><\/pre>\n
sd3_medium.safetensors is a relatively pure base model that only contains the relevant weights of MMDiT and VAE, but does not contain any of the text encoders mentioned in point 2 above.<\/p>\n
sd3_medium_incl_clips.safetensors is a combination of sd3_medium + clip_g + clip_l. This model will be smaller, but the performance of the model without the T5XXL encoder will be different.<\/p>\n
sd3_medium_incl_clips_t5xxlfp8.safetensors is a combination of sd3_medium + clip_g + clip_l + t5xxl_fp8_e4m3fn, a model that strikes a balance between quality and resource requirements.<\/p>\n
sd3_medium_incl_clips_t5xxlfp16.safetensors is a combination of sd3_medium + clip_g + clip_l + t5xxl_fp16. The quality should be higher, but the video memory occupied is also the highest.<\/p>\n
Next, enterPractice<\/strong>If you have already installed Comfyui, you can skip the first part.<\/p>\n
1. Install Comfyui<\/h1>\n
The first step is to download the installation package. Here I have prepared two forms: Quark Cloud Disk and Baidu Cloud Disk<\/p>\n
# Quark Cloud Disk is the installation package provided by Qiuye, so it contains more content https:\/\/pan.quark.cn\/s\/64b808baa960#\/list\/share\/377bd955c75a411c8d1d01f366255cdb-ComfyUI*101aki # Baidu Cloud Disk is downloaded and uploaded by myself, and only an integrated package is put in it for the convenience of friends who don\u2019t have Quark Cloud Disk Link: https:\/\/pan.baidu.com\/s\/103QhzN5R7m-19-JFW6Z9Yw Extraction code: 6666<\/code><\/pre>\n
The second step is to unzip and install. Double-click the launcher. The layout of the launcher is basically the same as the previous SD launcher.<\/p>\n
\"get-518\"<\/div>\n
After the startup is successful, click one-key start, and this screen appears, which means you are halfway there.<\/p>\n
\"get-519\"<\/div>\n
The third step is to upgrade the kernel version. If you have already upgraded, you can skip it.<\/p>\n
\"get-520\"<\/div>\n
I have upgraded to the latest version, including the extended version.<\/p>\n
\"get-521\"<\/div>\n
There are some errors when I start it (probably due to the version upgrade), but after actual testing, it does not affect the output of SD3.<\/p>\n
\"get-522\"<\/div>\n
\"get-523\"<\/div>\n
2. Download the model<\/h1>\n
We don't need to use all the above models this time, we only need to use these four models:<\/p>\n