Create README.md
Browse files
README.md
ADDED
@@ -0,0 +1,65 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
1 |
+
---
|
2 |
+
license: other
|
3 |
+
language:
|
4 |
+
- en
|
5 |
+
base_model:
|
6 |
+
- black-forest-labs/FLUX.1-dev
|
7 |
+
pipeline_tag: text-to-image
|
8 |
+
tags:
|
9 |
+
- diffusers
|
10 |
+
- controlnet
|
11 |
+
- Flux
|
12 |
+
- image-generation
|
13 |
+
---
|
14 |
+
|
15 |
+
# Description
|
16 |
+
This repository provides a Diffusers version of FLUX.1-dev Canny ControlNet checkpoint by Xlabs AI, [original repo](https://huggingface.co/XLabs-AI/flux-controlnet-canny-v3).
|
17 |
+
|
18 |
+
![Example Picture 1](canny_result.png?raw=true)
|
19 |
+
|
20 |
+
# How to use
|
21 |
+
This model can be used directly with the diffusers library
|
22 |
+
|
23 |
+
```
|
24 |
+
import torch
|
25 |
+
from diffusers.utils import load_image
|
26 |
+
from diffusers import FluxControlNetModel
|
27 |
+
from diffusers.pipelines import FluxControlNetPipeline
|
28 |
+
from PIL import Image
|
29 |
+
import numpy as np
|
30 |
+
|
31 |
+
generator = torch.Generator(device="cuda").manual_seed(87544357)
|
32 |
+
|
33 |
+
controlnet = FluxControlNetModel.from_pretrained(
|
34 |
+
"Xlabs-AI/flux-controlnet-canny-diffusers",
|
35 |
+
torch_dtype=torch.bfloat16,
|
36 |
+
use_safetensors=True,
|
37 |
+
)
|
38 |
+
pipe = FluxControlNetPipeline.from_pretrained(
|
39 |
+
"black-forest-labs/FLUX.1-dev",
|
40 |
+
controlnet=controlnet,
|
41 |
+
torch_dtype=torch.bfloat16
|
42 |
+
)
|
43 |
+
pipe.to("cuda")
|
44 |
+
|
45 |
+
control_image = load_image("https://huggingface.co/Xlabs-AI/flux-controlnet-hed-diffusers/resolve/main/canny_example.png")
|
46 |
+
prompt = "handsome girl with rainbow hair, anime"
|
47 |
+
|
48 |
+
image = pipe(
|
49 |
+
prompt,
|
50 |
+
control_image=control_image,
|
51 |
+
controlnet_conditioning_scale=0.7,
|
52 |
+
num_inference_steps=25,
|
53 |
+
guidance_scale=3.5,
|
54 |
+
height=1024,
|
55 |
+
width=768,
|
56 |
+
generator=generator,
|
57 |
+
num_images_per_prompt=1,
|
58 |
+
).images[0]
|
59 |
+
|
60 |
+
image.save("output_test_controlnet.png")
|
61 |
+
```
|
62 |
+
|
63 |
+
## License
|
64 |
+
|
65 |
+
Our weights fall under the [FLUX.1 [dev]](https://huggingface.co/black-forest-labs/FLUX.1-dev/blob/main/LICENSE.md) Non-Commercial License<br/>
|