PAIR
/

Text-to-Video
Diffusers
StableDiffusionPipeline
text-to-image
Inference Endpoints
robhen commited on
Commit
650a721
1 Parent(s): 6d91dfd

Update README.md

Browse files
Files changed (1) hide show
  1. README.md +56 -2
README.md CHANGED
@@ -1,14 +1,68 @@
1
  ---
2
  license: creativeml-openrail-m
3
  library_name: diffusers
 
4
  tags:
5
  - text-to-video
 
6
  ---
 
7
 
 
 
 
 
 
8
 
9
- This model is used for our [huggingface demo](https://huggingface.co/spaces/PAIR/Text2Video-Zero) of our work [text-to-video zero](https://github.com/Picsart-AI-Research/Text2Video-Zero).
10
 
11
- The model is from [CIVITAI](https://civitai.com). The license from CIVITAI thus applies.
12
 
 
 
13
 
14
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
  ---
2
  license: creativeml-openrail-m
3
  library_name: diffusers
4
+ inference: true
5
  tags:
6
  - text-to-video
7
+ - text-to-image
8
  ---
9
+ # ControlNet Canny Anime Model Card
10
 
11
+ [Text2Video-Zero](https://arxiv.org/abs/2303.13439) is a zero-shot text to video generator. It can perform `zero-shot text-to-video generation`, `Video Instruct Pix2Pix` (instruction-guided video editing),
12
+ `text and pose conditional video generation`, `text and canny-edge conditional video generation`, and
13
+ `text, canny-edge and dreambooth conditional video generation`. For more information about this work,
14
+ please have a look at our [paper](https://arxiv.org/abs/2303.13439) and our demo: [![Hugging Face Spaces](https://img.shields.io/badge/%F0%9F%A4%97%20Hugging%20Face-Spaces-blue)](https://huggingface.co/spaces/PAIR/Text2Video-Zero)
15
+ Our [code](https://github.com/Picsart-AI-Research/Text2Video-Zero) works with any StableDiffusion base model.
16
 
17
+ This model provides [DreamBooth](https://arxiv.org/abs/2208.12242) weights for the `Anime style` to be used with edge guidance (using [ControlNet](https://arxiv.org/abs/2302.05543)) in text2video zero.
18
 
 
19
 
20
+ ## Weights for Text2Video-Zero
21
+ We converted the original weights into diffusers and made them usable for [ControlNet](https://arxiv.org/abs/2302.05543) with edge guidance using: https://github.com/lllyasviel/ControlNet/discussions/12.
22
 
23
 
24
+ ### Model Details
25
+ - **Developed by:** Levon Khachatryan, Andranik Movsisyan, Vahram Tadevosyan, Roberto Henschel, Zhangyang Wang, Shant Navasardyan and Humphrey Shi
26
+ - **Model type:** Dreambooth text-to-image and text-to-video generation model with edge control for text2video zero
27
+ - **Language(s):** English
28
+ - **License:** [The CreativeML OpenRAIL M license](https://huggingface.co/spaces/CompVis/stable-diffusion-license).
29
+ - **Model Description:** This is a model for [text2video zero](https://github.com/Picsart-AI-Research/Text2Video-Zero) with edge guidance and anime style.
30
+ It can be used also with ControlNet in a text-to-image setup with edge guidance.
31
+ - **DreamBoth Keyword:** anime style
32
+ - **Resources for more information:** [GitHub](https://github.com/Picsart-AI-Research/Text2Video-Zero), [Paper](https://arxiv.org/abs/2303.13439), [CIVITAI](https://civitai.com/models/8740/superanime-viper).
33
+ - **Cite as:**
34
+
35
+ @article{text2video-zero,
36
+ title={Text2Video-Zero: Text-to-Image Diffusion Models are Zero-Shot Video Generators},
37
+ author={Khachatryan, Levon and Movsisyan, Andranik and Tadevosyan, Vahram and Henschel, Roberto and Wang, Zhangyang and Navasardyan, Shant and Shi, Humphrey},
38
+ journal={arXiv preprint arXiv:2303.13439},
39
+ year={2023}
40
+ }
41
+
42
+
43
+
44
+
45
+ ## Original Weights
46
+ The Dreambooth weights for the Anime style were taken from [CIVITAI](https://civitai.com/models/8740/superanime-viper).
47
+
48
+ ### Model Details
49
+ - **Developed by:** Quiet_Joker (Username listed on CIVITAI)
50
+ - **Model type:** Dreambooth text-to-image generation model
51
+ - **Language(s):** English
52
+ - **License:** [The CreativeML OpenRAIL M license](https://huggingface.co/spaces/CompVis/stable-diffusion-license).
53
+ - **Model Description:** This is a model that was created using [DreamBooth](https://arxiv.org/abs/2208.12242) to generate images with Anime style, based on text prompts.
54
+ - **DreamBoth Keyword:** anime style
55
+ - **Resources for more information:** [CIVITAI](https://civitai.com/models/8740/superanime-viper).
56
+
57
+
58
+ ## Biases content acknowledgement:
59
+ Beware that Text2Video-Zero may output content that reinforces or exacerbates societal biases, as well as realistic faces, pornography, and violence. Text2Video-Zero in this demo is meant only for research purposes.
60
+
61
+
62
+ # Citation
63
+ @article{text2video-zero,
64
+ title={Text2Video-Zero: Text-to-Image Diffusion Models are Zero-Shot Video Generators},
65
+ author={Khachatryan, Levon and Movsisyan, Andranik and Tadevosyan, Vahram and Henschel, Roberto and Wang, Zhangyang and Navasardyan, Shant and Shi, Humphrey},
66
+ journal={arXiv preprint arXiv:2303.13439},
67
+ year={2023}
68
+ }