ali-vilab

/

text-to-video-ms-1.7b

like 593

Follow

ali-vilab 152

I get a similar error. Though I installed imageio and imageio-ffmpeg for exporting the video:

import torch
from diffusers import DiffusionPipeline, DPMSolverMultistepScheduler
from diffusers.utils import export_to_video

# load pipeline
pipe = DiffusionPipeline.from_pretrained("damo-vilab/text-to-video-ms-1.7b", torch_dtype=torch.float16, variant="fp16")
pipe.scheduler = DPMSolverMultistepScheduler.from_config(pipe.scheduler.config)

# optimize for GPU memory
pipe.enable_model_cpu_offload()
pipe.enable_vae_slicing()

# generate
prompt=input("Provide text to convert into video: ")
video_frames=pipe(prompt,num_inference_steps=35).frames
# video_frames = pipe(prompt, num_inference_steps=25, num_frames=200).frames

# convert to video
vid_path= export_to_video(video_frames)
vid_name=vid_path.replace('./tmp','')
print('Name: ', vid_name)
torch.cuda.empty_cache()

The resulting error is similar:

100%|█████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 35/35 [00:10<00:00,  3.40it/s]
Traceback (most recent call last):
  File "B:\source\AI\vidgen\vidgen.py", line 19, in <module>
    vid_path= export_to_video(video_frames)
              ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "B:\source\AI\vidgen\Lib\site-packages\diffusers\utils\export_utils.py", line 182, in export_to_video
    writer.append_data(frame)
  File "B:\source\AI\vidgen\Lib\site-packages\imageio\core\format.py", line 590, in append_data
    return self._append_data(im, total_meta)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "B:\source\AI\vidgen\Lib\site-packages\imageio\plugins\ffmpeg.py", line 584, in _append_data
    raise ValueError("Image must have 1, 2, 3 or 4 channels")
ValueError: Image must have 1, 2, 3 or 4 channels

I have solved this problem by installing the lower virsion of diffusers.you can try to down the virsion 0.25-diffusers.hope you will solve it.

In higher versions of diffusers, you can try using pipe(prompt, num_inference_steps=25, num_frames=200).frames[0]. Alternatively, you can choose to use an earlier version of diffusers.