|
--- |
|
license: mit |
|
language: |
|
- en |
|
base_model: |
|
- Efficient-Large-Model/VILA1.5-40b |
|
pipeline_tag: video-text-to-text |
|
--- |
|
|
|
# LiFT: Leveraging Human Feedback for Text-to-Video Model Alignment |
|
|
|
|
|
## Summary |
|
This is the model checkpoint proposed in our paper "LiFT: Leveraging Human Feedback for Text-to-Video Model Alignment". LiFT-Critic is a novel Video-Text-to-Text Reward Model for synthesized video evaluation. |
|
|
|
Project: https://codegoat24.github.io/LiFT/ |
|
|
|
Code: https://github.com/CodeGoat24/LiFT |
|
|
|
## π§ Installation |
|
|
|
1. Clone the github repository and navigate to LiFT folder |
|
```bash |
|
git clone https://github.com/CodeGoat24/LiFT.git |
|
cd LiFT |
|
``` |
|
2. Install packages |
|
``` |
|
bash ./environment_setup.sh lift |
|
``` |
|
|
|
## π Inference |
|
|
|
### Run |
|
Please download this public [LiFT-Critic-40b-lora-v1.5](https://huggingface.co/Fudan-FUXI/LiFT-Critic-13b-lora-v1.5) checkpoints. |
|
|
|
We provide some synthesized videos for quick inference in `./demo` directory. |
|
|
|
```bash |
|
python LiFT-Critic/test/run_critic_40b.py --model-path ./LiFT-Critic-40b-lora-v1.5 |
|
``` |
|
|
|
# ποΈ Citation |
|
|
|
If you find our work helpful, please cite our paper. |
|
```bibtex |
|
@article{LiFT, |
|
title={LiFT: Leveraging Human Feedback for Text-to-Video Model Alignment.}, |
|
author={Wang, Yibin and Tan, Zhiyu, and Wang, Junyan and Yang, Xiaomeng and Jin, Cheng and Li, Hao}, |
|
journal={arXiv preprint arXiv:2412.04814}, |
|
year={2024} |
|
} |
|
``` |
|
|