| <!--Copyright 2024 The HuggingFace Team. All rights reserved. | |
| Licensed under the Apache License, Version 2.0 (the "License"); you may not use this file except in compliance with | |
| the License. You may obtain a copy of the License at | |
| http://www.apache.org/licenses/LICENSE-2.0 | |
| Unless required by applicable law or agreed to in writing, software distributed under the License is distributed on | |
| an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the License for the | |
| specific language governing permissions and limitations under the License. | |
| --> | |
| # Schedulers | |
| 🤗 Diffusers provides many scheduler functions for the diffusion process. A scheduler takes a model's output (the sample which the diffusion process is iterating on) and a timestep to return a denoised sample. The timestep is important because it dictates where in the diffusion process the step is; data is generated by iterating forward *n* timesteps and inference occurs by propagating backward through the timesteps. Based on the timestep, a scheduler may be *discrete* in which case the timestep is an `int` or *continuous* in which case the timestep is a `float`. | |
| Depending on the context, a scheduler defines how to iteratively add noise to an image or how to update a sample based on a model's output: | |
| - during *training*, a scheduler adds noise (there are different algorithms for how to add noise) to a sample to train a diffusion model | |
| - during *inference*, a scheduler defines how to update a sample based on a pretrained model's output | |
| Many schedulers are implemented from the [k-diffusion](https://github.com/crowsonkb/k-diffusion) library by [Katherine Crowson](https://github.com/crowsonkb/), and they're also widely used in A1111. To help you map the schedulers from k-diffusion and A1111 to the schedulers in 🤗 Diffusers, take a look at the table below: | |
| | A1111/k-diffusion | 🤗 Diffusers | Usage | | |
| |---------------------|-------------------------------------|---------------------------------------------------------------------------------------------------------------| | |
| | DPM++ 2M | [`DPMSolverMultistepScheduler`] | | | |
| | DPM++ 2M Karras | [`DPMSolverMultistepScheduler`] | init with `use_karras_sigmas=True` | | |
| | DPM++ 2M SDE | [`DPMSolverMultistepScheduler`] | init with `algorithm_type="sde-dpmsolver++"` | | |
| | DPM++ 2M SDE Karras | [`DPMSolverMultistepScheduler`] | init with `use_karras_sigmas=True` and `algorithm_type="sde-dpmsolver++"` | | |
| | DPM++ 2S a | N/A | very similar to `DPMSolverSinglestepScheduler` | | |
| | DPM++ 2S a Karras | N/A | very similar to `DPMSolverSinglestepScheduler(use_karras_sigmas=True, ...)` | | |
| | DPM++ SDE | [`DPMSolverSinglestepScheduler`] | | | |
| | DPM++ SDE Karras | [`DPMSolverSinglestepScheduler`] | init with `use_karras_sigmas=True` | | |
| | DPM2 | [`KDPM2DiscreteScheduler`] | | | |
| | DPM2 Karras | [`KDPM2DiscreteScheduler`] | init with `use_karras_sigmas=True` | | |
| | DPM2 a | [`KDPM2AncestralDiscreteScheduler`] | | | |
| | DPM2 a Karras | [`KDPM2AncestralDiscreteScheduler`] | init with `use_karras_sigmas=True` | | |
| | DPM adaptive | N/A | | | |
| | DPM fast | N/A | | | |
| | Euler | [`EulerDiscreteScheduler`] | | | |
| | Euler a | [`EulerAncestralDiscreteScheduler`] | | | |
| | Heun | [`HeunDiscreteScheduler`] | | | |
| | LMS | [`LMSDiscreteScheduler`] | | | |
| | LMS Karras | [`LMSDiscreteScheduler`] | init with `use_karras_sigmas=True` | | |
| | N/A | [`DEISMultistepScheduler`] | | | |
| | N/A | [`UniPCMultistepScheduler`] | | | |
| All schedulers are built from the base [`SchedulerMixin`] class which implements low level utilities shared by all schedulers. | |
| ## SchedulerMixin | |
| [[autodoc]] SchedulerMixin | |
| ## SchedulerOutput | |
| [[autodoc]] schedulers.scheduling_utils.SchedulerOutput | |
| ## KarrasDiffusionSchedulers | |
| [`KarrasDiffusionSchedulers`] are a broad generalization of schedulers in 🤗 Diffusers. The schedulers in this class are distinguished at a high level by their noise sampling strategy, the type of network and scaling, the training strategy, and how the loss is weighed. | |
| The different schedulers in this class, depending on the ordinary differential equations (ODE) solver type, fall into the above taxonomy and provide a good abstraction for the design of the main schedulers implemented in 🤗 Diffusers. The schedulers in this class are given [here](https://github.com/huggingface/diffusers/blob/a69754bb879ed55b9b6dc9dd0b3cf4fa4124c765/src/diffusers/schedulers/scheduling_utils.py#L32). | |
| ## PushToHubMixin | |
| [[autodoc]] utils.PushToHubMixin | |