Hugging Face's logo

RESMPDEV
/
Meta-Llama-3.1-405B-Instruct-GGUF-FIXED

Model card Files Community
Meta-Llama-3.1-405B-Instruct-GGUF-FIXED
File size: 987 Bytes 
4148344
 
 
ada1037
 
67e2562
f90118e
7353d44
f90118e
d7145ee
 
0e09e34
 
 
 
f90118e
 
 
 
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
 
---
license: llama3.1
---
More updates. Going to be attempting a imatrix quant at Q3 so hopefully if that goes well and runs right on latest master will upload asap!

Not uploading Q6. This Quant may not work with your current llama.cpp build as there were breaking changes in the latest master.

Q4_K_M IS UP!

For those of you who do not know how to reassamble the parts `cat Meta-Llama-3.1-405B-Instruct-Q4_K_M.gguf.part_* > Meta-Llama-3.1-405B-Instruct-Q4_K_M.gguf`

Am amusing generation to kill the time. Is the instruct tuned to think it can ONLY be run on the cloud? 

![image/png](https://cdn-uploads.huggingface.co/production/uploads/655dc641accde1bbc8b41aec/krAj94J-WfaDr68GDohpr.png)

![image/png](https://cdn-uploads.huggingface.co/production/uploads/655dc641accde1bbc8b41aec/h03HVIi53D6F6Uo0Qiz3Z.png)

Why download these GGUF's? I made some modifications to the llama.cpp quantification process to for SURE use the right tokenizer vs the Smaug BPE GGUF's that are out now.