Added training info
Browse files
README.md
CHANGED
@@ -18,9 +18,9 @@ language:
|
|
18 |
library_name: diffusers
|
19 |
---
|
20 |
|
21 |
-
#
|
22 |
|
23 |
-
These are controlnet weights trained on runwayml/stable-diffusion-v1-5 with new type of conditioning. You can find some example images in the following.
|
24 |
|
25 |
**prompt**: contemporary living room of a house
|
26 |
|
@@ -35,4 +35,34 @@ These are controlnet weights trained on runwayml/stable-diffusion-v1-5 with new
|
|
35 |
**prompt**: contemporary living room, high quality, 4k, realistic
|
36 |
|
37 |
**negative prompt**: low quality, monochrome, low res
|
38 |
-
![images_2)](./images_2.png)
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
18 |
library_name: diffusers
|
19 |
---
|
20 |
|
21 |
+
# ControlNet - mfidabel/controlnet-segment-anything
|
22 |
|
23 |
+
These are controlnet weights trained on runwayml/stable-diffusion-v1-5 with a new type of conditioning. You can find some example images in the following.
|
24 |
|
25 |
**prompt**: contemporary living room of a house
|
26 |
|
|
|
35 |
**prompt**: contemporary living room, high quality, 4k, realistic
|
36 |
|
37 |
**negative prompt**: low quality, monochrome, low res
|
38 |
+
![images_2)](./images_2.png)
|
39 |
+
|
40 |
+
## Training
|
41 |
+
|
42 |
+
**Training Data** This model was trained using a Segmented dataset based on the [COYO-700M Dataset](https://huggingface.co/datasets/kakaobrain/coyo-700m).
|
43 |
+
[Stable Diffusion v1.5](https://huggingface.co/runwayml/stable-diffusion-v1-5) checkpoint was used as the base model for the controlnet.
|
44 |
+
|
45 |
+
|
46 |
+
The model was trained as follows:
|
47 |
+
|
48 |
+
- 25k steps with the [SAM-COYO-2k](https://huggingface.co/datasets/mfidabel/sam-coyo-2k) dataset
|
49 |
+
- 28k steps with the [SAM-COYO-2.5k](https://huggingface.co/datasets/mfidabel/sam-coyo-2.5k) dataset
|
50 |
+
- 38k steps with the [SAM-COYO-3k](https://huggingface.co/datasets/mfidabel/sam-coyo-3k) dataset
|
51 |
+
|
52 |
+
In that particular order.
|
53 |
+
|
54 |
+
- **Hardware**: Google Cloud TPUv4-8 VM
|
55 |
+
|
56 |
+
- **Optimizer**: AdamW
|
57 |
+
|
58 |
+
- **Train Batch Size**: 2 x 4 = 8
|
59 |
+
|
60 |
+
- **Learning rate**: 0.00001 constant
|
61 |
+
|
62 |
+
- **Gradient Accumulation Steps**: 1
|
63 |
+
|
64 |
+
- **Resolution**: 512
|
65 |
+
|
66 |
+
Learning rate: warmup to 0.0001 for 10,000 steps and then kept constant
|
67 |
+
|
68 |
+
|