Difference ControlNets

Original ControlNet Difference Models HuggingFace Repository

STOP! THESE MODELS ARE NOT FOR PROMPTING/IMAGE GENERATION AND ARE FOR USE WITH ANY STABLE DIFFUSION 1.5 BASED MODELS (which are most models unless stated to be SD 2.X) AS CONTROL NETWORKS. THE ONES SHARED AT https://civitai.com/models/9251/controlnet-pre-trained-models ARE ControlNets FOR THE BASE SD 1.5 MODEL ITSELF, NOT DIFFERENCE ONES.

Control Nets allow you to guide the structure of an image based on existing ones and preprocessing them, or by using images that match the expected format for a given control net. More information regarding ControlNet and the different types can be found at https://github.com/lllyasviel/ControlNet.

All examples generated using preprocessors, with the AbyssOrangeMix2 - Hardcore model, with the prompt used being the same as the original input image (but with different seeds). All settings default except where mentioned on a specific control net type. Canvas width set to 512 x 768 to match the input image.

You only need to download the models you want to use, you do not need to download all of them.

These are the models required for the ControlNet extension, converted to Safetensor and "pruned" to extract the ControlNet neural network. I have tested them with AOM2, and they work. Download these models and place them in the \stable-diffusion-webui\extensions\sd-webui-controlnet\models directory.

Diagram was shared by Kohya and attempts to visually explain the difference between the original controlnet models, and the difference ones.