tags:
- text-to-image
- stable-diffusion
- lora
- diffusers
- template:sd-lora
base_model: stabilityai/stable-diffusion-xl-base-1.0
license: cc-by-nc-nd-4.0
⚡ Flash Diffusion: FlashSDXL ⚡
Flash Diffusion is a diffusion distillation method proposed in Flash Diffusion: Accelerating Any Conditional Diffusion Model for Few Steps Image Generation by Clément Chadebec, Onur Tasar, Eyal Benaroche, and Benjamin Aubin from Jasper Research. This model is a 108M LoRA distilled version of SDXL model that is able to generate images in 4 steps. The main purpose of this model is to reproduce the main results of the paper. See our live demo and official Github repo.
How to use?
The model can be used using the DiffusionPipeline from diffusers library directly. It can allow reducing the number of required sampling steps to 4 steps.
from diffusers import DiffusionPipeline, LCMScheduler
adapter_id = "jasperai/flash-sdxl"
pipe = DiffusionPipeline.from_pretrained(
"stabilityai/stable-diffusion-xl-base-1.0",
use_safetensors=True,
)
pipe.scheduler = LCMScheduler.from_pretrained(
"stabilityai/stable-diffusion-xl-base-1.0",
subfolder="scheduler",
timestep_spacing="trailing",
)
pipe.to("cuda")
# Fuse and load LoRA weights
pipe.load_lora_weights(adapter_id)
pipe.fuse_lora()
prompt = "A raccoon reading a book in a lush forest."
image = pipe(prompt, num_inference_steps=4, guidance_scale=0).images[0]
Training Details
The model was trained for 20k iterations on 4 H100 GPUs (representing approximately a total of 176 GPU hours of training). Please refer to the paper for further parameters details.
Metrics on COCO 2014 validation (Table 3)
- FID-10k: 21.62 (4 NFE)
- CLIP Score: 0.327 (4 NFE)
Citation
If you find this work useful or use it in your research, please consider citing us
@misc{chadebec2024flash,
title={Flash Diffusion: Accelerating Any Conditional Diffusion Model for Few Steps Image Generation},
author={Clement Chadebec and Onur Tasar and Eyal Benaroche and Benjamin Aubin},
year={2024},
eprint={2406.02347},
archivePrefix={arXiv},
primaryClass={cs.CV}
}
License
This model is released under the the Creative Commons BY-NC license.