{"id":19831,"date":"2024-09-12T10:14:17","date_gmt":"2024-09-12T02:14:17","guid":{"rendered":"https:\/\/www.1ai.net\/?p=19831"},"modified":"2024-09-12T10:14:17","modified_gmt":"2024-09-12T02:14:17","slug":"%e5%9b%be%e5%83%8f%e5%8f%8d%e6%8e%a8%e6%8f%90%e7%a4%ba%e8%af%ad%e6%a8%a1%e5%9e%8bjoycaption%e6%9c%ac%e5%9c%b0%e9%83%a8%e7%bd%b2%e6%8c%87%e5%8d%97%ef%bc%8c%e6%94%af%e6%8c%81sfw%e5%92%8cnsfw%e5%9b%be","status":"publish","type":"post","link":"https:\/\/www.1ai.net\/en\/19831.html","title":{"rendered":"Image Backpropagation Prompt Language Model JoyCaption Local Deployment Guide with SFW and NSFW Image Description Support"},"content":{"rendered":"<p><strong><a href=\"https:\/\/www.1ai.net\/en\/tag\/joy-caption\" title=\"_Other Organiser\" target=\"_blank\" >Joy Caption<\/a>Introduction<\/strong><\/p>\n<p>In previous articles have described the current excellent<strong><a href=\"https:\/\/www.1ai.net\/en\/tag\/%e5%9b%be%e5%83%8f%e5%8f%8d%e6%8e%a8\" title=\"[Sees articles with [image reverse] labels]\" target=\"_blank\" >image inversion<\/a>Prompt model: Joy Caption<\/strong>.<strong>Joy Caption<\/strong>The backpropagation cue word produces detailed and unambiguous descriptive information about the image and supports the<strong>SFW<\/strong>and<strong><a href=\"https:\/\/www.1ai.net\/en\/tag\/nsfw\" title=\"[View articles tagged with [NSFW]]\" target=\"_blank\" >NSFW<\/a><\/strong>image descriptions that are getting a lot of attention. This is a program that combines Google<strong>siglip-so400m-patch14-384<\/strong>models and image-cue oriented fine-tuning of the<strong>Meta-Llama-3.1-8B-bnb-4bit<\/strong>model composition. Running locally the<strong>Joy Caption<\/strong>The model takes up about<strong>7GB video memory<\/strong>.<\/p>\n<p><strong>Joy Caption Installation Guide<\/strong><\/p>\n<p>It is first necessary to utilize the<a href=\"https:\/\/www.1ai.net\/en\/tag\/comfyui\" title=\"_Other Organiser\" target=\"_blank\" >ComfyUI<\/a>Plugin Manager Search<strong>Comfyui_CXH_joy_caption<\/strong>plugin and click Install this plugin and restart. Then you also need to download the corresponding model, which is also a step in the local complex installation:<\/p>\n<p><img loading=\"lazy\" decoding=\"async\" class=\"alignnone size-full wp-image-19832\" title=\"19a02c54j00sjoghn000fd000u0005dm\" src=\"https:\/\/www.1ai.net\/wp-content\/uploads\/2024\/09\/19a02c54j00sjoghn000fd000u0005dm.jpg\" alt=\"19a02c54j00sjoghn000fd000u0005dm\" width=\"1080\" height=\"193\" \/><\/p>\n<ul>\n<li>Note that the local runtime environment needs to confirm that<strong>transformers&gt;=4.44.2<\/strong>\u00a0Version.<strong>Plug-in address<\/strong>:: https:\/\/github.com\/StartHua\/Comfyui_CXH_joy_caption<\/li>\n<li>Download the model<strong>google\/siglip-so400m-patch14-384<\/strong>and place the catalog\u00a0<strong>\/ComfyUI\/models\/clip\/siglip-so400m-patch14-384<\/strong>. Here you need to download<strong>Entire project document<\/strong>, modeled at https:\/\/huggingface.co\/google\/siglip-so400m-patch14-384<\/li>\n<\/ul>\n<p><img loading=\"lazy\" decoding=\"async\" class=\"alignnone size-full wp-image-19833\" title=\"2dfa799dj00sjoghn000xd000oy00e8m\" src=\"https:\/\/www.1ai.net\/wp-content\/uploads\/2024\/09\/2dfa799dj00sjoghn000xd000oy00e8m.jpg\" alt=\"2dfa799dj00sjoghn000xd000oy00e8m\" width=\"898\" height=\"512\" \/><\/p>\n<ul>\n<li>Download the model\u00a0<strong>unsloth\/Meta-Llama-3.1-8B-bnb-4bit<\/strong>\u00a0and place the catalog\u00a0<strong>\/ComfyUI\/models\/LLM<\/strong>. Download<strong>Entire project document<\/strong>, modeled at https:\/\/huggingface.co\/unsloth\/Meta-Llama-3.1-8B-bnb-4bit<\/li>\n<\/ul>\n<p><img loading=\"lazy\" decoding=\"async\" class=\"alignnone size-full wp-image-19834\" title=\"f27d9bb4j00sjoghm000ld000oa00a7m\" src=\"https:\/\/www.1ai.net\/wp-content\/uploads\/2024\/09\/f27d9bb4j00sjoghm000ld000oa00a7m.jpg\" alt=\"f27d9bb4j00sjoghm000ld000oa00a7m\" width=\"874\" height=\"367\" \/><\/p>\n<ul>\n<li>Download the model\u00a0<strong>fancyfeast\/joy-caption-pre-alpha<\/strong>\u00a0and place the catalog\u00a0<strong>\/ComfyUI\/models\/Joy_caption<\/strong>. Here you need to download it manually<strong>Entire project document<\/strong>, modeled at https:\/\/huggingface.co\/spaces\/fancyfeast\/joy-caption-pre-alpha\/tree\/main\/wpkklhc6<\/li>\n<\/ul>\n<p><img loading=\"lazy\" decoding=\"async\" class=\"alignnone size-full wp-image-19836\" title=\"662913c4j00sjoghm000id000ny009am\" src=\"https:\/\/www.1ai.net\/wp-content\/uploads\/2024\/09\/662913c4j00sjoghm000id000ny009am.jpg\" alt=\"662913c4j00sjoghm000id000ny009am\" width=\"862\" height=\"334\" \/><\/p>\n<p>If you don't want to download the file manually, you can try to use git to download it, navigate to the corresponding directory in CMD and then use git to download the directory, e.g.<strong>google\/siglip-so400m-patch14-384<\/strong>\u00a0The model is as follows:<\/p>\n<p>git lfs install<br \/>\n# Replacement of corresponding model git address<br \/>\ngit lfs clone https:\/\/huggingface.co\/google\/siglip-so400m-patch14-384<\/p>\n<p>Basic Flux ComfyUI Workflow<\/p>\n<p>About Flux model native ComfyUI workflow experience see the previous article: FLUX [Continued]: 12B parameters 23G largest open source Vincennes diagram model, Dev version straight out of the stunning beauty to appreciate .<\/p>\n<p>The ComfyUI workflow and models mentioned in this article can be downloaded on LIBLIBAI or run online for experience:<\/p>\n<ul>\n<li>FLUX.1 is available online - Dark Forest Studio: https:\/\/www.liblib.art\/modelinfo\/488cd9d58cd4421b9e8000373d7da123<\/li>\n<li>Workflow - Flux text | picture + LORA + CN + prompt reverse push one-click switch workflow: https:\/\/www.liblib.art\/modelinfo\/782aacd70f604da39e83368c696a02a8<\/li>\n<\/ul>\n<p><img loading=\"lazy\" decoding=\"async\" class=\"alignnone size-full wp-image-19835\" title=\"d24d6978j00sjoghn001od000u000jnm\" src=\"https:\/\/www.1ai.net\/wp-content\/uploads\/2024\/09\/d24d6978j00sjoghn001od000u000jnm.jpg\" alt=\"d24d6978j00sjoghn001od000u000jnm\" width=\"1080\" height=\"707\" \/><\/p>\n<p><strong>Joy Caption Backpropagation Workflow<\/strong><\/p>\n<p>To increase<strong>JoyCaption<\/strong>Backpropagation prompts workflows by simply adding the Flux base workflow to the<strong>JoyCaption model loading and backpropagation<\/strong>Two nodes.<strong>Workflow Download Address<\/strong>For: https:\/\/www.liblib.art\/modelinfo\/cc112e6f18bf46049b680ec4b42c511a .<\/p>\n<p><img loading=\"lazy\" decoding=\"async\" class=\"alignnone size-full wp-image-19837\" title=\"50a709e5j00sjoghn000nd000gn00bwm\" src=\"https:\/\/www.1ai.net\/wp-content\/uploads\/2024\/09\/50a709e5j00sjoghn000nd000gn00bwm.jpg\" alt=\"50a709e5j00sjoghn000nd000gn00bwm\" width=\"599\" height=\"428\" \/><\/p>\n<p><strong>Notice<\/strong>: This paper uses the<strong>Flux Detail Texture Enhancement LORA Model<\/strong>,<strong>Enhances overall image quality<\/strong>. About the introduction of LORA see the previous article: [ComfyUI] Flux: it's awesome! Detailed texture enhancement, character drop oil light realistic, rich movie light, rich picture elements.<\/p>\n<p>01. Drying of clothes<\/p>\n<p>The woman is facing away from the camera, gazing towards the horizon with a simple, white, short-sleeved T-shirt and light blue denim shorts. She has long, straight black hair tied into a high ponytail, and is wearing a simple, white, short-sleeved T-shirt and light blue denim shorts. The woman is facing away from the camera, gazing towards the horizon with a serene expression. The woman is facing away from the camera, gazing towards the horizon with a serene expression. She holds a wooden clothespin in her right hand, which is holding a white T-shirt on a clothesline strung horizontally across the rooftop. The clothesline is made of thin, yellow string, and the clothespin is positioned near the sleeve of the shirt. In the background, there is a clear blue sky with a few scattered clouds, and the clothespin is gazing towards the horizon with a serene expression. blue sky with a few scattered clouds, and a view of a cityscape featuring multiple high-rise apartment buildings with balconies. The rooftop surface is concrete, with a few small plants adding to the surface. The rooftop surface is concrete, with a few small plants adding some greenery. The overall scene conveys a sense of tranquility and simplicity, with the bright sunlight casting The overall scene conveys a sense of tranquility and simplicity, with the bright sunlight casting soft, natural shadows.<\/p>\n<p><img loading=\"lazy\" decoding=\"async\" class=\"alignnone size-full wp-image-19838\" title=\"8f27df0dj00sjoghn001dd000u000e7m\" src=\"https:\/\/www.1ai.net\/wp-content\/uploads\/2024\/09\/8f27df0dj00sjoghn001dd000u000e7m.jpg\" alt=\"8f27df0dj00sjoghn001dd000u000e7m\" width=\"1080\" height=\"511\" \/><\/p>\n<p><img loading=\"lazy\" decoding=\"async\" class=\"alignnone size-full wp-image-19839\" title=\"73f4229bj00sjoghn0022d000u0012km\" src=\"https:\/\/www.1ai.net\/wp-content\/uploads\/2024\/09\/73f4229bj00sjoghn0022d000u0012km.jpg\" alt=\"73f4229bj00sjoghn0022d000u0012km\" width=\"1080\" height=\"1388\" \/><\/p>\n<p>02. Together<\/p>\n<p>this is a different person, and the other person, and the other person is the same person, and the other person is the same person.<\/p>\n<p><img loading=\"lazy\" decoding=\"async\" class=\"alignnone size-full wp-image-19840\" title=\"8df792c1j00sjoghn001fd000u000e2m\" src=\"https:\/\/www.1ai.net\/wp-content\/uploads\/2024\/09\/8df792c1j00sjoghn001fd000u000e2m.jpg\" alt=\"8df792c1j00sjoghn001fd000u000e2m\" width=\"1080\" height=\"506\" \/><\/p>\n<p><img loading=\"lazy\" decoding=\"async\" class=\"alignnone size-full wp-image-19841\" title=\"a988a485j00sjoghn000td000u000kim\" src=\"https:\/\/www.1ai.net\/wp-content\/uploads\/2024\/09\/a988a485j00sjoghn000td000u000kim.jpg\" alt=\"a988a485j00sjoghn000td000u000kim\" width=\"1080\" height=\"738\" \/><\/p>\n<p>03. Selling piglets<\/p>\n<p>i'm not sure if I'm going to be able to do that.<\/p>\n<p><img loading=\"lazy\" decoding=\"async\" class=\"alignnone size-full wp-image-19842\" title=\"602e7a4aj00sjoghn001jd000u000efm\" src=\"https:\/\/www.1ai.net\/wp-content\/uploads\/2024\/09\/602e7a4aj00sjoghn001jd000u000efm.jpg\" alt=\"602e7a4aj00sjoghn001jd000u000efm\" width=\"1080\" height=\"519\" \/><\/p>\n<p><img loading=\"lazy\" decoding=\"async\" class=\"alignnone size-full wp-image-19843\" title=\"557eac5fj00sjoghn0033d000u0012km\" src=\"https:\/\/www.1ai.net\/wp-content\/uploads\/2024\/09\/557eac5fj00sjoghn0033d000u0012km.jpg\" alt=\"557eac5fj00sjoghn0033d000u0012km\" width=\"1080\" height=\"1388\" \/><\/p>\n<p>04. Cucumber costume show<\/p>\n<p>this is a high-level, high-level, high-level, high-level, high-level, high-level, high-level, high-level, high-level, high-level, high-level, high-level, high-level, high-level, high-level, high-level, high-level, high-level, high-level, high-level, high-level, high-level, high-level, high-level, high-level, high-level, high-level, high-level, high-level, high-level, high-level, high-level, high-level, high-level, high-level, high-level, high-level, high-level, high-level, high-level, high-level, high-level, high-level, high-level, high-level, high-level, high-level, high-level, high-level, high-level, high-level, high-level, high-level, high-level, high-level, high-level, high-level, high-level, high-level, high-level, high-level, high-level, and senior-level.<\/p>\n<p><img loading=\"lazy\" decoding=\"async\" class=\"alignnone size-full wp-image-19844\" title=\"a7d7f000j00sjoghn001hd000u000e3m\" src=\"https:\/\/www.1ai.net\/wp-content\/uploads\/2024\/09\/a7d7f000j00sjoghn001hd000u000e3m.jpg\" alt=\"a7d7f000j00sjoghn001hd000u000e3m\" width=\"1080\" height=\"507\" \/><\/p>\n<p><img loading=\"lazy\" decoding=\"async\" class=\"alignnone size-full wp-image-19845\" title=\"ae7ada96j00sjoghn001zd000ow00w0m\" src=\"https:\/\/www.1ai.net\/wp-content\/uploads\/2024\/09\/ae7ada96j00sjoghn001zd000ow00w0m.jpg\" alt=\"ae7ada96j00sjoghn001zd000ow00w0m\" width=\"896\" height=\"1152\" \/><\/p>","protected":false},"excerpt":{"rendered":"<p>Introduction to Joy Caption The current excellent image backpropagation cueing model, Joy Caption, has been introduced in a previous article.Joy Caption backpropagation cueing produces detailed and unambiguous information about the image description, and has attracted much attention by supporting both SFW and NSFW image descriptions. It is a combination of the Google siglip-so400m-patch14-384 model and the Meta-Llama-3.1-8B-bnb-4bit model composition geared towards fine-tuning image cues. Running the Joy Caption model locally takes up about 7GB of video memory. Joy Caption Installation Guide The first thing you need to do is to utilize the ComfyUI Plugin Manager to search for Comfy<\/p>","protected":false},"author":1,"featured_media":0,"comment_status":"closed","ping_status":"closed","sticky":false,"template":"","format":"standard","meta":{"footnotes":""},"categories":[149,144],"tags":[1989,4276,3895,4341,2852],"collection":[],"class_list":["post-19831","post","type-post","status-publish","format-standard","hentry","category-jiaocheng","category-baike","tag-comfyui","tag-joy-caption","tag-nsfw","tag-4341","tag-2852"],"acf":[],"_links":{"self":[{"href":"https:\/\/www.1ai.net\/en\/wp-json\/wp\/v2\/posts\/19831","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/www.1ai.net\/en\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/www.1ai.net\/en\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/www.1ai.net\/en\/wp-json\/wp\/v2\/users\/1"}],"replies":[{"embeddable":true,"href":"https:\/\/www.1ai.net\/en\/wp-json\/wp\/v2\/comments?post=19831"}],"version-history":[{"count":0,"href":"https:\/\/www.1ai.net\/en\/wp-json\/wp\/v2\/posts\/19831\/revisions"}],"wp:attachment":[{"href":"https:\/\/www.1ai.net\/en\/wp-json\/wp\/v2\/media?parent=19831"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/www.1ai.net\/en\/wp-json\/wp\/v2\/categories?post=19831"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/www.1ai.net\/en\/wp-json\/wp\/v2\/tags?post=19831"},{"taxonomy":"collection","embeddable":true,"href":"https:\/\/www.1ai.net\/en\/wp-json\/wp\/v2\/collection?post=19831"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}