Spaces:

ggml-org

/

gguf-my-repo

like 954

Running on A10G

When i try to make q8 gguf for this model (https://huggingface.co/tiiuae/falcon-mamba-7b-instruct), i get this message: Error converting to fp16: b'INFO:hf-to-gguf:Loading model: falcon-mamba-7b-instruct\nERROR:hf-to-gguf:Model FalconMambaForCausalLM is not supported\n'

Llama.cpp release which supports Falcon Mamba is here: https://github.com/ggerganov/llama.cpp/releases/tag/b3612