Chinese ControlNet Model (Depth)

简介 Brief Introduction

我们开源了一个中文 ControlNet 模型，该模型适配 diffusion 模型 alibaba-pai/pai-diffusion-artist-large-zh，您可以使用该模型控制 diffusion 模型生成的图像。

We release a Chinese ControlNet model, which works with diffusion model alibaba-pai/pai-diffusion-artist-large-zh. You can use this model to control the diffusion model generating images as you wish.

• Github: EasyNLP

使用 Usage

from diffusers import StableDiffusionControlNetPipeline, ControlNetModel
from transformers import pipeline
from PIL import Image
import numpy as np
import cv2


def to_depth(image):
    depth_estimator = pipeline('depth-estimation')
    image = depth_estimator(image)['depth']
    image = np.array(image)
    image = image[:, :, None]
    image = np.concatenate([image, image, image], axis=2)
    image = Image.fromarray(image)
    return image


controlnet_id = "alibaba-pai/pai-diffusion-artist-large-zh-controlnet-depth"
controlnet = ControlNetModel.from_pretrained(controlnet_id)
model_id = "alibaba-pai/pai-diffusion-artist-large-zh"
pipe = StableDiffusionControlNetPipeline.from_pretrained(model_id, controlnet=controlnet)
pipe = pipe.to("cuda")

image = Image.open("image.png")
controlnet_image = to_depth(image)
prompt = "白色羽毛的小鸟"
image = pipe(prompt, controlnet_image).images[0]

controlnet_image.save("image_depth.png")
image.save("image_depth_output.png")

样例 Example

Prompt: "白色的小鸟"

原图	Depth 图	生成图

使用须知 Notice for Use

使用上述模型需遵守AIGC模型开源特别条款。

If you want to use this model, please read this document carefully and abide by the terms.