Title 09 ComfyUI from installation entrance to mastery, LoRA uses combat (selection/down/discharge/string weight)You've got a LoRA skill pack on the bottom model. But there's a question you'll probably hit sooner or later:I want it to follow a reference mapOr have two people sitting side by side and running around without a hint。
I used to make a "two handshake" with three hundred words in the tip, either back-to-back or hand-in-hand, and I had to redraw a dozen. It was only later that ControlNet knew that “the image was in command” and that it should not have been by the mouth, but by a map. And this hand-handed hand tells you what it is, where the model is, how the nodes are connected, how the three most used scenes fit, how the stength is screwed, and what the red-word error and the 8G mass collapse pits are。
One, ControlNet. What is it
One sentence:ControlNet is a line for the bottom model, and you take a reference map and you use it as a rope, and the image is pressed on that image, and you don't run around。Unlike LoRA -- LoRA is what the foundation model looks like, ControlNet is what it says。
Zenium OpenPoseLocked person position (bones), assigned movement, multiple positions
Zenium Depth: Lock space deep, perceiving, front- and back-level, building
Zenium CannyLocks the contour line, colours on the draft, make-up, rewinding
Remember:The LoRA tube's face, the ControlNet tube's position and structure, is called a genuine control map。
II. Where to place the document (the wrong / model is not effective for the whole)
The controlNet model (.safetensors) shows this directory:
ComfyUI/models/controlnet/
Two pits blocked earlier:
1. The model must have a base model—ControlNet for SD 1.5 with SD 1.5 base model, SDXL with SDXL. Combining with LoRA will Blackchart / Invalid, page Base Model label will be accepted before。
2. Don't be too big— WITH 512, WITH 1024 4 TIMES TIME + 3 TIMES VISIBILITY, 8G CARD IS EASY TO OOM。
Trail experience:For the first time, I was thinking of stacking three ControlNets, all of which were original 1024, and then the CUDA OOM, and it was stuck. Later on -- a smaller reference, a smaller CN, less folding, and an 8G card。
III. Hand-to-hand nodes (continuous)
For example, OpenPose:
Reference Diagram OpenPose Preprocessor
Checkpoint (MODEL)
Phrasing
1. Add Node: Right-key search for Apple ControlNet, ControlNet Loader, Load Image, and then OpenPose。
2. Select Model:ControlNet Loader contorl_net_name chooses .safetensors。
3. Connect: Reference Diagram →Control for Loader; Model → Loader for CHeckpoint; CONTROL_NET → Apply ControlNet; Prostive →Apply ControlNet's commanding, Apply Output KSampler。
4. setstrength: Apply ControlNet default 1.0 (lock to death), adjustable。
One sentence:Apply ControlNet, then KSampler, Reference Map Preprocessor fed ControlNet Loader。
Clean-up ring: 10 minutes, lock character positions with OpenPose (do now)
The principles of the previous sections are true. After you've got an extra "Personal pose and reference map" in your hand, you'll be able to confirm that ControlNet really works -- not just reading it, but making it。
Preconditions: You've got the Load Checkpoint, two CLIP Text Encode, KSampler, VAE Decode, Save Image. If you lose it, press this five nodes again。
Target: Take a profile of the person's position, and let the person who's created do the same thing, and don't move with the hints。
Step (sight, don't jump):
1. Down ModelOpenPose ControlNet (SD1.5 for control_v11p_sd15_openpose, SDXL for SDXL version), dropped into ComfyUI/models/controlnet/, F5 to refresh。
2. Add Node: Double-click Load Image + OpenPose + ControlNet Loader + Apply ControlNet, one each。
3. Reference Diagram: IMAGE →image for OpenPose preprocessor for Load Image; control for POSE → ControlNet Loader for preprocessor。
4. Main Chain: MODEL → ControlNet Loader of CHeckpoint; CONTROL_NET → Apply ControlNet _net of Loader; Control_net of your positive tipping word → Apply ControlNet = Conditioning, Apply CONDIIONG Output → KSampler。
5. Set parameters: Strength fill 1.0 for Apply ControlNet (locked in position), start_percent / end_percent for 0/ 1.0。
6. Fixed Feed + OutSeed for KSampler has fixed numbers (e. g. 12345), point Queue Prompt。
Checkpoint1 Generating a person's position and reference diagrams consistent, with a no-altering gesture = effective; 2 completely disordered = preprocessor failed or reference did not enter Loader (return section III); 3 Report “Expected 1 channels but got 3 channels” = reference diagrams directly linked to Apply ControlNet, without preprocessor (repeated pre-processing, see section VI); 4 Total unchanged = Apply ControlNet was not connected to the patent chain。
the job (chosen): to drop the stength from 1.0 to 0.6 and then run, the character becomes loose, the model becomes more active, and feels the rope is loose。
IV. Three scenario formulations (OpenPose/Dept/Canny)
A ControlNet ripe, three together is normal. Multi-control method: The last Apply ControlNet CONDITIONG output goes to the next input (positive → Apply CN#1 → Apply CN#2 → KSampler). Here's the beginning of a fine-tuning exercise
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
end_percent techniques: 1.0 whole-step control; 0.7-0.85 front-end control diagrams, post-stage free-colouring of models (canny rewind permanent 0.7, OpenPose set 1.0)。
Trail experience:I wanted the picture to be "both in position and in depth" and set it directly at 0.0 for Apply ControlNet, and the result was to create a completely ignored reference -- and later I knew that 0.0 was " hung and uncontrollable" and to control it was to give value greater than zero。
V. Common pits / mined areas
1. Canny will insert pre-treatment: Reference diagrams directly linked to ControlNet "Expected 1 channels but got 3 channels". The Canny Edge Test Node (low_threshold 100/ high_threshold 200 applies 95% scenes) must be inserted between Load Image and Apply. OpenPose/Depth has a preprocessor, too, except for。
2. settings 0.0 = Generates but completely ignores reference maps, often with errors。
3. Reference scale mismatch: USING 512 INSTEAD OF 1024, OR 4 TIMES TIME + 3 TIMES VISIBILITY, EASY OOM。
4. Base model does not match: CN with SDXL base model for SD1.5, black map for light and invalid weight. Allows the Base Model tag before the next。
5. The constraints are contradictory: Too many ControlNets and asked to fight, the scenes are falling apart. First, double, then single, then group。
VI. QUIREMENTS OF PARLIAMENTS
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
VII. 4060 / 8G SPECIALIZED & I STEPPED ON PITS
Skills:Union SDXL is the 8G User Gospel- A 2.5-GB model from xinsir contains 10+CN types, which, in deduction, inserts the SetUnionControlNetType node type (canny/dispth/pose/tile)Toggle type without reloading model, save memory + save time, the production environment is preferred. In the early days of SDXL, only Canny/Dept, Union completely changed the situation。
More CN superclaves visible warning: each CN is an additional U-Net level reasoning, much more visible。3 CN SUPERIMPOSED 16GB+ VRAM; 2 ARE DESSERT- We have up to two 8G users and suggest –lowvram at startup. The more restraints, the easier they are to contradict each other, the more the picture collapses, the more they are, the more they are, the more they are, the more they are, the more they are, the more they are, the more they are, the more they are, the more they are, the more they are, the more they are, the more they are, the more they are, the more they are, the more they are, the more they are, the more they are, the more they are, the more they are, the more they are, the more they are, the more they are, the more they are, the more they are, the more they are, the more they are, the more they are, the more they are, the more they are, the more they are, the more they are, the more they are, the more they are, the more they are, the more they are, the more they are, the more they are, the more they are, the more they are, the more they are, the more they are, the more they are, the more they are, the more they are, the more they are, the more, the more they are the more they are。
Five pits say all at once: Canny, no pre-treatment(No. 1 red-word pit) 2 settings 0.0 Equal to uncontrolled;3 Reference figure 1024 Visible explosive; 4 Base model does not match Black Chart 5 3 CNS STACKED 8G CARP. OOM。
ControlNet will work, and you've been able to put the picture in command -- position, space, contours. But there is a need for both:To lock a specific face and make it look the same in every episode。It's gonna have to be a high-resolution zoom in on IPAdapter and back。Page 11 Deep + High Zoom, add clarity and detail。