gbueno86
/

Meta-LLama-3-Cat-A-LLama-70b-exl2-8.0bpw

Text Generation

text-generation-inference

Inference Endpoints

8-bit precision

Model card Files Files and versions Community

gbueno86 commited on May 25

Commit

b3f63bf

•

1 Parent(s): e36cfff

Update README.md

Files changed (1) hide show

README.md +1 -1

README.md CHANGED Viewed

@@ -1,7 +1,7 @@
 ![image/jpeg](https://cdn-uploads.huggingface.co/production/uploads/649dc85249ae3a68334adcc6/q3MkfQNU9gAw8Bn-OHHpK.jpeg)
 Most intelligent merge yet. This became my new daily driver, last was 120b version.
-This needs 67GB VRAM, loads in 3x 3090s with cache_4bit and 8192 context on exllamav2 0.0.20 as 12tk/s.
 ---
 base_model: []

 ![image/jpeg](https://cdn-uploads.huggingface.co/production/uploads/649dc85249ae3a68334adcc6/q3MkfQNU9gAw8Bn-OHHpK.jpeg)
 Most intelligent merge yet. This became my new daily driver, last was 120b version.
+This needs 67GB VRAM, loads in 3x 3090s with cache_4bit and 8192 context on exllamav2 0.0.20 at 12tk/s.
 ---
 base_model: []