How to use ControlNet

Locates all faces in the input image, analyzes each face to detect facial features, and generates a facial landmarks map that can be used in conjunction with the OpenPose CN-model. This provides you with the ability to transfer facial expressions from one image to another. The Face Capture and Pose Capture preprocessors can be used simultaneously.

This module locates human subjects in the input image, analyzes each one to capture their pose, and generates a 'bones-joints' type map that can be used in combination with the OpenPose CN-model. This gives you the ability to control the pose of subjects in your generated images.

Creates a sketch-style image from the reference image with the help of an AI model.

Utilizes the MiDaS Depth model to generate a depth-map from the reference image.

Utilizes the IS-Net and U2Net Segmentation models to generate a segmentation map from the reference image based on its distinct areas and subjects. When you select the Segmentation preprocessor, you also have the option to mask only subjects or backgrounds. (IS-Net is used for single images while U2Net is applied to video frames)

Will invert the colors of the active image (original, depth-map, sketch).

Works with the outlines of an image. Best if the reference image is an outline-style image.

Reproduces depth relationships from the reference image. Works well with original images and depth-maps.

Utilize masks to define and modify specific areas. Employ reference images with clear transparency areas to be filled with the InPaint model. To zoom out an image and fill the empty areas, use InPaint. Just add the image to ControlNet, activate 'Camera: zoom, pan, roll', zoom out to your desired level, select the InPaint model, and click the "Imagine" button.

Capable of making direct changes to the reference image from text instructions. Use the prompt and style fields to instruct ControlNet to change something in the image. You can type things like "Make them look like robots", "Add boats on the water", "Change sunglasses for ski goggles", "Make it look like winter", etc.

Find and reuse small outlines.

LineArt optimized for anime.

(Mobile Line Segment Detection) is a straight-line detector. It is useful for extracting outlines with straight edges like interior designs, buildings, street scenes, picture frames, and paper edges.

Reproduce depth relationships using surface normal depth maps. The Bae normal map tends to render details in both background and foreground, while the Depth model might ignore more background objects.

A fast human key-point detection model that can extract human poses like positions of hands, legs, and head. Use it in conjunction with the face and body capture preprocessors.

Generate images from scribbles or freehand drawings. Works great with the Sketch preprocessor.

Find and reuse distinct areas and subjects. Works well with the Segmentation and Depth preprocessors.

Find and reorder major elements.

In contrast with the LineArt models, SoftEdge finds and reuses soft edges to generate the new image. Works great without preprocessing.

Used for adding details to an image.