AI & ML interests

Instruction tuning, Diffusion models

We explore instruction-tuning to teach Stable Diffusion to follow instructions to translate or process input images. With this method, we can prompt our model using an input image and an “instruction”, such as - Apply a cartoon filter to the natural image.

Please follow our blog post for more details: https://huggingface.co/blog/instruction-tuning-sd.

Below we provide links to our training code, models, and datasets:

Happy instruction-tuning 🤗