Update README.md

README.md

@@ -27,11 +27,42 @@ datasets:
 language:
 - en
 library_name: transformers
+inference: false
 tags:
 - code
 - art
 ---
 
+# NinjaMouse-3B-40L-danube-exl2
+
+Original model: [NinjaMouse-3B-40L-danube](https://huggingface.co/trollek/NinjaMouse-3B-40L-danube)  
+Model creator: [trollek](https://huggingface.co/trollek)  
+
+## Quants
+
+[4bpw h6](https://huggingface.co/cgus/NinjaMouse-3B-40L-danube-exl2/tree/main)  
+[4.25bpw h6](https://huggingface.co/cgus/NinjaMouse-3B-40L-danube-exl2/tree/4.25bpw-h6)  
+[4.65bpw h6](https://huggingface.co/cgus/NinjaMouse-3B-40L-danube-exl2/tree/4.65bpw-h6)  
+[5bpw h6](https://huggingface.co/cgus/NinjaMouse-3B-40L-danube-exl2/tree/5bpw-h6)  
+[6bpw h6](https://huggingface.co/cgus/NinjaMouse-3B-40L-danube-exl2/tree/6bpw-h6)  
+[8bpw h8](https://huggingface.co/cgus/NinjaMouse-3B-40L-danube-exl2/tree/8bpw-h8)  
+
+## Quantization notes
+Made with exllamav2 0.0.15 with default dataset. I'm very unsure about this model.  
+For me it breaks past 3000 context, at about 3500 or so. Both with these quants and the creator's GGUF files.  
+At first I thought I had some quantization issues but it's probably just the model itself.
+
+## How to run
+This quantization method uses GPU and requires Exllamav2 loader which can be found in following applications:
+
+[Text Generation Webui](https://github.com/oobabooga/text-generation-webui)  
+
+[KoboldAI](https://github.com/henk717/KoboldAI)  
+
+[ExUI](https://github.com/turboderp/exui)  
+
+# Original model card
+
 This is [NinjaMouse](https://huggingface.co/trollek/NinjaMouse-2.4B-32L-danube) extended even further. Instead of Cosmopedia I used different coding datasets. 
 
 I have learned a lot during this process, and if you got a GPU capable of training your own you should try it. I made some mistakes, like using the pure_bf16 at some point among other things, but the second version will slap the leaderboard for its weight class.