Spaces:

tonychenxyz

/

emo-knob

like 5

Running on T4

Hi @tonychenxyz , we've assigned ZeroGPU to this Space. Please check the compatibility and usage sections of this page so your Space can run on ZeroGPU.

@hysts Thank you very much! I checked out some other spaces using ZeroGPU, they seem to load model to cuda after checking torch.cuda.is_available(). I did the same, but I got the error:

RuntimeError: CUDA must not be initialized in the main process on Spaces with Stateless GPU environment.

The relevant part of code in my app.py is

if torch.cuda.is_available():
    if not debug:
        model_name = "metavoiceio/metavoice-1B-v0.1"
        seed = 1337
        output_dir = "outputs"
        _dtype = get_default_dtype()
        # _device = 'cuda:0'
        _model_dir = snapshot_download(repo_id=model_name)
        first_stage_adapter = FlattenedInterleavedEncodec2Codebook(end_of_audio_token=1024)
        output_dir = output_dir
        os.makedirs(output_dir, exist_ok=True)

        second_stage_ckpt_path = f"{_model_dir}/second_stage.pt"
        config_second_stage = InferenceConfig(
            ckpt_path=second_stage_ckpt_path,
            num_samples=1,
            seed=seed,
            device=_device,
            dtype=_dtype,
            compile=False,
            init_from="resume",
            output_dir=output_dir,
        )
        data_adapter_second_stage = TiltedEncodec(end_of_audio_token=1024)
        llm_second_stage = Model(
            config_second_stage, TrainedBPETokeniser, EncodecDecoder, data_adapter_fn=data_adapter_second_stage.decode
        )
        enhancer = get_enhancer("df")

        precision = {"float16": torch.float16, "bfloat16": torch.bfloat16}[_dtype]
        model, tokenizer, smodel, model_size = build_model(
            precision=precision,
            checkpoint_path=Path(f"{_model_dir}/first_stage.pt"),
            spk_emb_ckpt_path=Path(f"{_model_dir}/speaker_encoder.pt"),
            device=_device,
            compile=True,
            compile_prefill=True,
        )

I was wondering how can I fix it. Thanks!

@hysts Our method is built based on MetaVoice. I just checked their huggingface Space. Seems like they had some issue with ZeroGPU too and got assigned T4. I was wondering if we can be assigned T4 too. Thanks!
https://huggingface.co/spaces/mrfakename/MetaVoice-1B-v0.1/discussions/1

@tonychenxyz

In the discussion thread you mentioned, they said:

An unofficial demo of MetaVoice 1B. For some reason it doesn't work on ZERO GPU (gets an error when installing flash-attn). Doesn't work right now on CPU, it needs CUDA to run.

But I'm not sure if it really doesn't work on ZeroGPU. If the issue is installation of flash-attn, it can be installed like this in ZeroGPU Spaces. Maybe they just couldn't figure out how to install it.

@hysts
Thanks for your reply. Do you know what's potential solution for my error? (I don't think it's flash-attn)

Traceback (most recent call last):
  File "/home/user/app/app.py", line 110, in <module>
    model, tokenizer, smodel, model_size = build_model(
  File "/home/user/app/fam/llm/fast_inference_utils.py", line 331, in build_model
    device_sync(device=device)  # MKG
  File "/home/user/app/fam/llm/fast_inference_utils.py", line 39, in device_sync
    torch.cuda.synchronize()
  File "/usr/local/lib/python3.10/site-packages/torch/cuda/__init__.py", line 781, in synchronize
    _lazy_init()
  File "/usr/local/lib/python3.10/site-packages/torch/cuda/__init__.py", line 298, in _lazy_init
    torch._C._cuda_init()
  File "/usr/local/lib/python3.10/site-packages/spaces/zero/torch/patching.py", line 254, in _cuda_init_raise
    raise RuntimeError(
RuntimeError: CUDA must not be initialized in the main process on Spaces with Stateless GPU environment.
You can look at this Stacktrace to find out which part of your code triggered a CUDA init
/usr/local/lib/python3.10/site-packages/huggingface_hub/file_download.py:1150: FutureWarning: `resume_download` is deprecated and will be removed in version 1.0.0. Downloads always resume when possible. If you want to force a new download, use `force_download=True`.
  warnings.warn(
/usr/local/lib/python3.10/site-packages/torch/nn/utils/weight_norm.py:30: UserWarning: torch.nn.utils.weight_norm is deprecated in favor of torch.nn.utils.parametrizations.weight_norm.
  warnings.warn("torch.nn.utils.weight_norm is deprecated in favor of torch.nn.utils.parametrizations.weight_norm.")

Ah, OK, well, the fam library doesn't seem to be compatible with ZeroGPU for other reasons. I just assigned t4-small.

Thanks very much!