README.md

---
license: openrail
pipeline_tag: text-to-image
---
# Fine-tuning [Stable Diffusion XL](https://huggingface.co/stabilityai/stable-diffusion-xl-base-1.0) for custom face and thumbs up gesture
Colab Notebook implementation of **LoRA** based Dreambooth technique, utilizing **Huggingface diffusers**, for fine-tuning the model to reliably predict thumbs up gesture with a custom face.

## Datasets
Use the following links to download and view the thumbs-up and selfie images respectively
1. [Thumbs Up](https://www.dropbox.com/scl/fi/yye56ojn7aebbdf45u1nb/thumbs_up.zip?rlkey=ixu3srz5ks2ybgzl0056xree1&dl=0)
2. [Selfies](https://www.dropbox.com/scl/fi/8rdxsvyc3r6hf18b9u4gq/selfies.zip?rlkey=b7q6xn1wj90id2ix834wpwaa3&dl=0)

## Checkpoints
1. *lora-trained-thumbs-up-pw-0.5-steps-800* contains the weights for the model finetuned for the thumbs-up gesture
2. *lora-trained-thumbs-up-pw-0.5-steps-800-uttam-pw-0.5-steps-1600* contains the weights for the model finetuned for the custom face

## Results
Here are some of the quantitative results
![](results/best_ug2146_simple.png)
*trigger keyword:* **a photo of ug2146 giving thumbs up**

![](results/best_ug2146_joker.png)
*trigger keyword:*  + **in a joker costume**