|
--- |
|
base_model: CohereForAI/c4ai-command-r-08-2024 |
|
extra_gated_fields: |
|
Affiliation: text |
|
Country: |
|
options: |
|
- Aruba |
|
- Afghanistan |
|
- Angola |
|
- Anguilla |
|
- Åland Islands |
|
- Albania |
|
- Andorra |
|
- United Arab Emirates |
|
- Argentina |
|
- Armenia |
|
- American Samoa |
|
- Antarctica |
|
- French Southern Territories |
|
- Antigua and Barbuda |
|
- Australia |
|
- Austria |
|
- Azerbaijan |
|
- Burundi |
|
- Belgium |
|
- Benin |
|
- Bonaire Sint Eustatius and Saba |
|
- Burkina Faso |
|
- Bangladesh |
|
- Bulgaria |
|
- Bahrain |
|
- Bahamas |
|
- Bosnia and Herzegovina |
|
- Saint Barthélemy |
|
- Belarus |
|
- Belize |
|
- Bermuda |
|
- Plurinational State of Bolivia |
|
- Brazil |
|
- Barbados |
|
- Brunei-Darussalam |
|
- Bhutan |
|
- Bouvet-Island |
|
- Botswana |
|
- Central African Republic |
|
- Canada |
|
- Cocos (Keeling) Islands |
|
- Switzerland |
|
- Chile |
|
- China |
|
- Côte-dIvoire |
|
- Cameroon |
|
- Democratic Republic of the Congo |
|
- Cook Islands |
|
- Colombia |
|
- Comoros |
|
- Cabo Verde |
|
- Costa Rica |
|
- Cuba |
|
- Curaçao |
|
- Christmas Island |
|
- Cayman Islands |
|
- Cyprus |
|
- Czechia |
|
- Germany |
|
- Djibouti |
|
- Dominica |
|
- Denmark |
|
- Dominican Republic |
|
- Algeria |
|
- Ecuador |
|
- Egypt |
|
- Eritrea |
|
- Western Sahara |
|
- Spain |
|
- Estonia |
|
- Ethiopia |
|
- Finland |
|
- Fiji |
|
- Falkland Islands (Malvinas) |
|
- France |
|
- Faroe Islands |
|
- Federated States of Micronesia |
|
- Gabon |
|
- United Kingdom |
|
- Georgia |
|
- Guernsey |
|
- Ghana |
|
- Gibraltar |
|
- Guinea |
|
- Guadeloupe |
|
- Gambia |
|
- Guinea Bissau |
|
- Equatorial Guinea |
|
- Greece |
|
- Grenada |
|
- Greenland |
|
- Guatemala |
|
- French Guiana |
|
- Guam |
|
- Guyana |
|
- Hong Kong |
|
- Heard Island and McDonald Islands |
|
- Honduras |
|
- Croatia |
|
- Haiti |
|
- Hungary |
|
- Indonesia |
|
- Isle of Man |
|
- India |
|
- British Indian Ocean Territory |
|
- Ireland |
|
- Islamic Republic of Iran |
|
- Iraq |
|
- Iceland |
|
- Israel |
|
- Italy |
|
- Jamaica |
|
- Jersey |
|
- Jordan |
|
- Japan |
|
- Kazakhstan |
|
- Kenya |
|
- Kyrgyzstan |
|
- Cambodia |
|
- Kiribati |
|
- Saint-Kitts-and-Nevis |
|
- South Korea |
|
- Kuwait |
|
- Lao-Peoples-Democratic-Republic |
|
- Lebanon |
|
- Liberia |
|
- Libya |
|
- Saint-Lucia |
|
- Liechtenstein |
|
- Sri Lanka |
|
- Lesotho |
|
- Lithuania |
|
- Luxembourg |
|
- Latvia |
|
- Macao |
|
- Saint Martin (French-part) |
|
- Morocco |
|
- Monaco |
|
- Republic of Moldova |
|
- Madagascar |
|
- Maldives |
|
- Mexico |
|
- Marshall Islands |
|
- North Macedonia |
|
- Mali |
|
- Malta |
|
- Myanmar |
|
- Montenegro |
|
- Mongolia |
|
- Northern Mariana Islands |
|
- Mozambique |
|
- Mauritania |
|
- Montserrat |
|
- Martinique |
|
- Mauritius |
|
- Malawi |
|
- Malaysia |
|
- Mayotte |
|
- Namibia |
|
- New Caledonia |
|
- Niger |
|
- Norfolk Island |
|
- Nigeria |
|
- Nicaragua |
|
- Niue |
|
- Netherlands |
|
- Norway |
|
- Nepal |
|
- Nauru |
|
- New Zealand |
|
- Oman |
|
- Pakistan |
|
- Panama |
|
- Pitcairn |
|
- Peru |
|
- Philippines |
|
- Palau |
|
- Papua New Guinea |
|
- Poland |
|
- Puerto Rico |
|
- North Korea |
|
- Portugal |
|
- Paraguay |
|
- State of Palestine |
|
- French Polynesia |
|
- Qatar |
|
- Réunion |
|
- Romania |
|
- Russia |
|
- Rwanda |
|
- Saudi Arabia |
|
- Sudan |
|
- Senegal |
|
- Singapore |
|
- South Georgia and the South Sandwich Islands |
|
- Saint Helena Ascension and Tristan da Cunha |
|
- Svalbard and Jan Mayen |
|
- Solomon Islands |
|
- Sierra Leone |
|
- El Salvador |
|
- San Marino |
|
- Somalia |
|
- Saint Pierre and Miquelon |
|
- Serbia |
|
- South Sudan |
|
- Sao Tome and Principe |
|
- Suriname |
|
- Slovakia |
|
- Slovenia |
|
- Sweden |
|
- Eswatini |
|
- Sint Maarten (Dutch-part) |
|
- Seychelles |
|
- Syrian Arab Republic |
|
- Turks and Caicos Islands |
|
- Chad |
|
- Togo |
|
- Thailand |
|
- Tajikistan |
|
- Tokelau |
|
- Turkmenistan |
|
- Timor Leste |
|
- Tonga |
|
- Trinidad and Tobago |
|
- Tunisia |
|
- Turkey |
|
- Tuvalu |
|
- Taiwan |
|
- United Republic of Tanzania |
|
- Uganda |
|
- Ukraine |
|
- United States Minor Outlying Islands |
|
- Uruguay |
|
- United-States |
|
- Uzbekistan |
|
- Holy See (Vatican City State) |
|
- Saint Vincent and the Grenadines |
|
- Bolivarian Republic of Venezuela |
|
- Virgin Islands British |
|
- Virgin Islands U.S. |
|
- VietNam |
|
- Vanuatu |
|
- Wallis and Futuna |
|
- Samoa |
|
- Yemen |
|
- South Africa |
|
- Zambia |
|
- Zimbabwe |
|
type: select |
|
I agree to use this model for non-commercial use ONLY: checkbox |
|
Name: text |
|
Receive email updates on C4AI and Cohere research, events, products and services?: |
|
options: |
|
- Yes |
|
- No |
|
type: select |
|
extra_gated_prompt: By submitting this form, you agree to the [License Agreement](https://cohere.com/c4ai-cc-by-nc-license) and |
|
acknowledge that the information you provide will be collected, used, and shared |
|
in accordance with Cohere’s [Privacy Policy]( https://cohere.com/privacy). |
|
language: |
|
- en |
|
- fr |
|
- de |
|
- es |
|
- it |
|
- pt |
|
- ja |
|
- ko |
|
- zh |
|
- ar |
|
library_name: transformers |
|
license: cc-by-nc-4.0 |
|
quantized_by: mradermacher |
|
--- |
|
## About |
|
|
|
<!-- ### quantize_version: 2 --> |
|
<!-- ### output_tensor_quantised: 1 --> |
|
<!-- ### convert_type: hf --> |
|
<!-- ### vocab_type: --> |
|
<!-- ### tags: nicoboss --> |
|
weighted/imatrix quants of https://huggingface.co/CohereForAI/c4ai-command-r-08-2024 |
|
|
|
<!-- provided-files --> |
|
static quants are available at https://huggingface.co/mradermacher/c4ai-command-r-08-2024-GGUF |
|
## Usage |
|
|
|
If you are unsure how to use GGUF files, refer to one of [TheBloke's |
|
READMEs](https://huggingface.co/TheBloke/KafkaLM-70B-German-V0.1-GGUF) for |
|
more details, including on how to concatenate multi-part files. |
|
|
|
## Provided Quants |
|
|
|
(sorted by size, not necessarily quality. IQ-quants are often preferable over similar sized non-IQ quants) |
|
|
|
| Link | Type | Size/GB | Notes | |
|
|:-----|:-----|--------:|:------| |
|
| [GGUF](https://huggingface.co/mradermacher/c4ai-command-r-08-2024-i1-GGUF/resolve/main/c4ai-command-r-08-2024.i1-IQ1_S.gguf) | i1-IQ1_S | 7.9 | for the desperate | |
|
| [GGUF](https://huggingface.co/mradermacher/c4ai-command-r-08-2024-i1-GGUF/resolve/main/c4ai-command-r-08-2024.i1-IQ1_M.gguf) | i1-IQ1_M | 8.5 | mostly desperate | |
|
| [GGUF](https://huggingface.co/mradermacher/c4ai-command-r-08-2024-i1-GGUF/resolve/main/c4ai-command-r-08-2024.i1-IQ2_XXS.gguf) | i1-IQ2_XXS | 9.5 | | |
|
| [GGUF](https://huggingface.co/mradermacher/c4ai-command-r-08-2024-i1-GGUF/resolve/main/c4ai-command-r-08-2024.i1-IQ2_XS.gguf) | i1-IQ2_XS | 10.4 | | |
|
| [GGUF](https://huggingface.co/mradermacher/c4ai-command-r-08-2024-i1-GGUF/resolve/main/c4ai-command-r-08-2024.i1-IQ2_S.gguf) | i1-IQ2_S | 10.9 | | |
|
| [GGUF](https://huggingface.co/mradermacher/c4ai-command-r-08-2024-i1-GGUF/resolve/main/c4ai-command-r-08-2024.i1-IQ2_M.gguf) | i1-IQ2_M | 11.7 | | |
|
| [GGUF](https://huggingface.co/mradermacher/c4ai-command-r-08-2024-i1-GGUF/resolve/main/c4ai-command-r-08-2024.i1-Q2_K.gguf) | i1-Q2_K | 12.9 | IQ3_XXS probably better | |
|
| [GGUF](https://huggingface.co/mradermacher/c4ai-command-r-08-2024-i1-GGUF/resolve/main/c4ai-command-r-08-2024.i1-IQ3_XXS.gguf) | i1-IQ3_XXS | 13.1 | lower quality | |
|
| [GGUF](https://huggingface.co/mradermacher/c4ai-command-r-08-2024-i1-GGUF/resolve/main/c4ai-command-r-08-2024.i1-IQ3_XS.gguf) | i1-IQ3_XS | 14.1 | | |
|
| [GGUF](https://huggingface.co/mradermacher/c4ai-command-r-08-2024-i1-GGUF/resolve/main/c4ai-command-r-08-2024.i1-Q3_K_S.gguf) | i1-Q3_K_S | 14.8 | IQ3_XS probably better | |
|
| [GGUF](https://huggingface.co/mradermacher/c4ai-command-r-08-2024-i1-GGUF/resolve/main/c4ai-command-r-08-2024.i1-IQ3_S.gguf) | i1-IQ3_S | 14.9 | beats Q3_K* | |
|
| [GGUF](https://huggingface.co/mradermacher/c4ai-command-r-08-2024-i1-GGUF/resolve/main/c4ai-command-r-08-2024.i1-IQ3_M.gguf) | i1-IQ3_M | 15.3 | | |
|
| [GGUF](https://huggingface.co/mradermacher/c4ai-command-r-08-2024-i1-GGUF/resolve/main/c4ai-command-r-08-2024.i1-Q3_K_M.gguf) | i1-Q3_K_M | 16.3 | IQ3_S probably better | |
|
| [GGUF](https://huggingface.co/mradermacher/c4ai-command-r-08-2024-i1-GGUF/resolve/main/c4ai-command-r-08-2024.i1-Q3_K_L.gguf) | i1-Q3_K_L | 17.7 | IQ3_M probably better | |
|
| [GGUF](https://huggingface.co/mradermacher/c4ai-command-r-08-2024-i1-GGUF/resolve/main/c4ai-command-r-08-2024.i1-IQ4_XS.gguf) | i1-IQ4_XS | 17.9 | | |
|
| [GGUF](https://huggingface.co/mradermacher/c4ai-command-r-08-2024-i1-GGUF/resolve/main/c4ai-command-r-08-2024.i1-Q4_0.gguf) | i1-Q4_0 | 18.9 | fast, low quality | |
|
| [GGUF](https://huggingface.co/mradermacher/c4ai-command-r-08-2024-i1-GGUF/resolve/main/c4ai-command-r-08-2024.i1-Q4_K_S.gguf) | i1-Q4_K_S | 18.9 | optimal size/speed/quality | |
|
| [GGUF](https://huggingface.co/mradermacher/c4ai-command-r-08-2024-i1-GGUF/resolve/main/c4ai-command-r-08-2024.i1-Q4_K_M.gguf) | i1-Q4_K_M | 19.9 | fast, recommended | |
|
| [GGUF](https://huggingface.co/mradermacher/c4ai-command-r-08-2024-i1-GGUF/resolve/main/c4ai-command-r-08-2024.i1-Q5_K_S.gguf) | i1-Q5_K_S | 22.6 | | |
|
| [GGUF](https://huggingface.co/mradermacher/c4ai-command-r-08-2024-i1-GGUF/resolve/main/c4ai-command-r-08-2024.i1-Q5_K_M.gguf) | i1-Q5_K_M | 23.2 | | |
|
| [GGUF](https://huggingface.co/mradermacher/c4ai-command-r-08-2024-i1-GGUF/resolve/main/c4ai-command-r-08-2024.i1-Q6_K.gguf) | i1-Q6_K | 26.6 | practically like static Q6_K | |
|
|
|
Here is a handy graph by ikawrakow comparing some lower-quality quant |
|
types (lower is better): |
|
|
|
 |
|
|
|
And here are Artefact2's thoughts on the matter: |
|
https://gist.github.com/Artefact2/b5f810600771265fc1e39442288e8ec9 |
|
|
|
## FAQ / Model Request |
|
|
|
See https://huggingface.co/mradermacher/model_requests for some answers to |
|
questions you might have and/or if you want some other model quantized. |
|
|
|
## Thanks |
|
|
|
I thank my company, [nethype GmbH](https://www.nethype.de/), for letting |
|
me use its servers and providing upgrades to my workstation to enable |
|
this work in my free time. Additional thanks to [@nicoboss](https://huggingface.co/nicoboss) for giving me access to his private supercomputer, enabling me to provide many more imatrix quants, at much higher quality, than I would otherwise be able to. |
|
|
|
<!-- end --> |
|
|
