Spaces:
Build error
Build error
Update README.md
Browse files
README.md
CHANGED
@@ -8,6 +8,27 @@ sdk_version: 3.28.2
|
|
8 |
app_file: app.py
|
9 |
pinned: false
|
10 |
license: cc-by-4.0
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
11 |
---
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
12 |
|
13 |
Check out the configuration reference at https://huggingface.co/docs/hub/spaces-config-reference
|
|
|
8 |
app_file: app.py
|
9 |
pinned: false
|
10 |
license: cc-by-4.0
|
11 |
+
tags:
|
12 |
+
- stable-diffusion
|
13 |
+
- stable-diffusion-diffusers
|
14 |
+
- text-to-image
|
15 |
+
- diffusers
|
16 |
+
- controlnet
|
17 |
+
- jax-diffusers-event
|
18 |
---
|
19 |
+
|
20 |
+
# ControlLight: Light control through ControlNet and Depth Maps conditioning
|
21 |
+
We propose a ControlNet using depth maps conditioning that is capable of controlling the light direction in a scene while trying to maintain the scene integrity.
|
22 |
+
The model was trained on [VIDIT dataset](https://huggingface.co/datasets/Nahrawy/VIDIT-Depth-ControlNet) and [
|
23 |
+
A Dataset of Flash and Ambient Illumination Pairs from the Crowd](https://huggingface.co/datasets/Nahrawy/FAID-Depth-ControlNet) as a part of the [Jax Diffusers Event](https://huggingface.co/jax-diffusers-event).
|
24 |
+
|
25 |
+
Due to the limited available data the model is clearly overfit, but it serves as a proof of concept to what can be further achieved using enough data.
|
26 |
+
|
27 |
+
A large part of the training data is synthetic so we encourage further training using synthetically generated scenes, using Unreal engine for example.
|
28 |
+
|
29 |
+
The WandB training logs can be found [here](https://wandb.ai/hassanelnahrawy/controlnet-VIDIT-FAID), it's worth noting that the model was left to overfit for experimentation and it's advised to use the 8K steps weights or prior weights.
|
30 |
+
|
31 |
+
This project is a joint work between [ParityError](https://huggingface.co/ParityError) and [Nahrawy](https://huggingface.co/Nahrawy).
|
32 |
+
|
33 |
|
34 |
Check out the configuration reference at https://huggingface.co/docs/hub/spaces-config-reference
|