Nahrawy commited on
Commit
2e36d78
1 Parent(s): a46c388

Update README.md

Browse files
Files changed (1) hide show
  1. README.md +21 -0
README.md CHANGED
@@ -8,6 +8,27 @@ sdk_version: 3.28.2
8
  app_file: app.py
9
  pinned: false
10
  license: cc-by-4.0
 
 
 
 
 
 
 
11
  ---
 
 
 
 
 
 
 
 
 
 
 
 
 
 
12
 
13
  Check out the configuration reference at https://huggingface.co/docs/hub/spaces-config-reference
 
8
  app_file: app.py
9
  pinned: false
10
  license: cc-by-4.0
11
+ tags:
12
+ - stable-diffusion
13
+ - stable-diffusion-diffusers
14
+ - text-to-image
15
+ - diffusers
16
+ - controlnet
17
+ - jax-diffusers-event
18
  ---
19
+
20
+ # ControlLight: Light control through ControlNet and Depth Maps conditioning
21
+ We propose a ControlNet using depth maps conditioning that is capable of controlling the light direction in a scene while trying to maintain the scene integrity.
22
+ The model was trained on [VIDIT dataset](https://huggingface.co/datasets/Nahrawy/VIDIT-Depth-ControlNet) and [
23
+ A Dataset of Flash and Ambient Illumination Pairs from the Crowd](https://huggingface.co/datasets/Nahrawy/FAID-Depth-ControlNet) as a part of the [Jax Diffusers Event](https://huggingface.co/jax-diffusers-event).
24
+
25
+ Due to the limited available data the model is clearly overfit, but it serves as a proof of concept to what can be further achieved using enough data.
26
+
27
+ A large part of the training data is synthetic so we encourage further training using synthetically generated scenes, using Unreal engine for example.
28
+
29
+ The WandB training logs can be found [here](https://wandb.ai/hassanelnahrawy/controlnet-VIDIT-FAID), it's worth noting that the model was left to overfit for experimentation and it's advised to use the 8K steps weights or prior weights.
30
+
31
+ This project is a joint work between [ParityError](https://huggingface.co/ParityError) and [Nahrawy](https://huggingface.co/Nahrawy).
32
+
33
 
34
  Check out the configuration reference at https://huggingface.co/docs/hub/spaces-config-reference