Core ML Stable Diffusion Mixed-Bit Palettization Resources
This repository contains resources related to the use of mixed-bit palettization techniques for Stable Diffusion. As explained in the original repo, this is an advanced compression technique that applies palettization using a varying number of bits per layer to keep quality above the desired threshold (as measured by the PSNR with respect to the original model). This can achieve strong space savings without resorting to learned quantization tables during fine-tuning.
The repository contains the following resources:
recipes/. JSON files with results from the analysis phase (Step 1 of the instructions) on three popular Stable Diffusion models:- Stable Diffusion 1.5
- Stable Diffusion 2.1 base
- Stable Diffusion XL 1.0 base
You can apply any of the available recipes on the appropriate model to experiment and compare results.
unet-mbp-sdxl-1-base/. Three pre-applied recipes on the UNet of Stable Diffusion XL 1.0 base:
| Recipe | Effective bits | UNet size (GB) | Size reduction |
|---|---|---|---|
| recipe_3_41_bit_mixedpalette | 3.41 | 1.1 | 77% |
| recipe_4_50_bit_mixedpalette | 4.50 | 1.4 | 71% |
| recipe_6_55_bit_mixedpalette | 6.55 | 2.0 | 58% |
[
coreml-stable-diffusion-xl-base_mbp_4_50_palettized]. A full pipeline of a Core ML version of Stable Diffusion XL 1.0 base, with therecipe_4_50_bit_mixedpaletterecipe pre-applied to the UNet. You can download it to use in your own Core ML apps.[
coreml-stable-diffusion-xl-base_mbp_4_50_palettized.zip]. An archived version of the same pipeline, for use with Hugging Face demo app and other third party tools.
This repository was prepared by Apple and Hugging Face in July 2023, from experiments conducted on July 2023 using public beta versions of iOS 17.0, iPadOS 17.0 and macOS 14.0.