|
--- |
|
license: apache-2.0 |
|
base_model: |
|
- genmo/mochi-1-preview |
|
--- |
|
|
|
**Please change bf16 to fp16 in your code. I've just changed the checkpoint to fp16, which appears more coherent with mochi** |
|
|
|
This is a nf4 quantization of mochi-1-preview. It excludes specific layers that allow it to stay coherent: |
|
|
|
bf16 |
|
|
|
<video controls autoplay src="https://cdn-uploads.huggingface.co/production/uploads/630406731dd5d3c62486140a/tnCaH4bMgvjYi6Wfjk3vb.mp4"></video> |
|
|
|
nf4mix-small (this one): |
|
|
|
<video controls autoplay src="https://cdn-uploads.huggingface.co/production/uploads/630406731dd5d3c62486140a/DqXbARydKh0ts-bgzSGrt.mp4"></video> |
|
|
|
nf4: |
|
|
|
<video controls autoplay src="https://cdn-uploads.huggingface.co/production/uploads/630406731dd5d3c62486140a/gVMUZNcAPxGSoiPRximAN.mp4"></video> |
|
|
|
**Diffusers main branch is required** |
|
|
|
`pip install git+https://github.com/huggingface/diffusers` |
|
|
|
To use: |
|
|
|
``` |
|
from diffusers import MochiPipeline, MochiTransformer3DModel |
|
from diffusers.utils import export_to_video |
|
import torch |
|
transformer = MochiTransformer3DModel.from_pretrained("imnotednamode/mochi-1-preview-mix-nf4-small", torch_dtype=torch.float16) |
|
pipe = MochiPipeline.from_pretrained("genmo/mochi-1-preview", revision="refs/pr/18", torch_dtype=torch.float16, transformer=transformer) |
|
pipe.enable_model_cpu_offload() |
|
pipe.enable_vae_tiling() |
|
frames = pipe("A camera follows a squirrel running around on a tree branch", num_inference_steps=100, guidance_scale=4.5, height=480, width=848, num_frames=161).frames[0] |
|
export_to_video(frames, "mochi.mp4", fps=15) |
|
|
|
``` |
|
|
|
To reproduce: |
|
|
|
``` |
|
from diffusers import MochiPipeline, MochiTransformer3DModel, BitsAndBytesConfig |
|
import torch |
|
quantization_config = BitsAndBytesConfig(load_in_4bit=True, bnb_4bit_compute_dtype=torch.float16, bnb_4bit_quant_type="nf4", llm_int8_skip_modules=["final_layer", "x_embedder.proj", "t_embedder", "pos_frequencies", "t5"]) |
|
# Please convert mochi to diffusers first |
|
transformer = MochiTransformer3DModel.from_pretrained("genmo/mochi-1-preview", revision="refs/pr/18", subfolder="transformer", quantization_config=quantization_config, torch_dtype=torch.float16) |
|
transformer.save_pretrained("mochi-1-preview-nf4") |
|
``` |
