Hugging Face's logo

dranger003
/
c4ai-command-r-plus-iMat.GGUF

Text Generation
GGUF
Inference Endpoints
Model card Files Community
19
dranger003 commited on
Commit
32cc406
•
1 Parent(s): 234e584

Update README.md

Browse files
Files changed (1) hide show
  1. README.md +2 -0
README.md CHANGED
@@ -7,6 +7,8 @@ base_model: CohereForAI/c4ai-command-r-plus
7
  **2024-04-05**: Support for this model is still being worked on - [`PR#6491`](https://github.com/ggerganov/llama.cpp/pull/6491).
8
  For now, you can test the model using this fork: [https://github.com/Noeda/llama.cpp/tree/commandr-plus](https://github.com/Noeda/llama.cpp/tree/commandr-plus)
9
 
 
 
10
  * GGUF importance matrix (imatrix) quants for https://huggingface.co/CohereForAI/c4ai-command-r-plus
11
  * The importance matrix was trained for ~100K tokens (200 batches of 512 tokens) using [wiki.train.raw](https://huggingface.co/datasets/wikitext).
12
  * [Which GGUF is right for me? (from Artefact2)](https://gist.github.com/Artefact2/b5f810600771265fc1e39442288e8ec9)
 
7
  **2024-04-05**: Support for this model is still being worked on - [`PR#6491`](https://github.com/ggerganov/llama.cpp/pull/6491).
8
  For now, you can test the model using this fork: [https://github.com/Noeda/llama.cpp/tree/commandr-plus](https://github.com/Noeda/llama.cpp/tree/commandr-plus)
9
 
10
+ **<u>NOTE</u>**: There may be a need to re-quantize all the weights once support for this model is finalized.
11
+
12
  * GGUF importance matrix (imatrix) quants for https://huggingface.co/CohereForAI/c4ai-command-r-plus
13
  * The importance matrix was trained for ~100K tokens (200 batches of 512 tokens) using [wiki.train.raw](https://huggingface.co/datasets/wikitext).
14
  * [Which GGUF is right for me? (from Artefact2)](https://gist.github.com/Artefact2/b5f810600771265fc1e39442288e8ec9)