bartowski

/

DeepSeek-V2.5-GGUF

like 15

DeepSeek-V2.5-IQ1_M-00001-of-00002.gguf

DeepSeek-V2.5-IQ1_M-00002-of-00002.gg

not run on colab

You just have to load the first and then any llama.cpp tool will look for and load the second

from llama_cpp import Llama

llm = Llama.from_pretrained(
repo_id="bartowski/DeepSeek-V2.5-GGUF",
filename="DeepSeek-V2.5-IQ1_M/DeepSeek-V2.5-IQ1_M-00001-of-00002.gguf",
)

llm.create_chat_completion(
messages = [
{
"role": "user",
"content": "What is the capital of France?"
}
]
) Should I use the code and it will download the two parts by itself and run them without me merging them together?

!./llama-gguf-split --merge DeepSeek-V2.5-IQ1_M-00001-of-00002.gguf DeepSeek-V2.5-IQ1_M-00002-of-00002.gguf DeepSeek-V2.5-IQ1_M.gguf

Every time I merge, the second part is deleted.

I think you're meant to only pass the first part like this:

./llama-gguf-split --merge DeepSeek-V2.5-IQ1_M-00001-of-00002.gguf DeepSeek-V2.5-IQ1_M.gguf

https://huggingface.co/goodasdgood/dd/tree/main