GGUF Quants with iMatrix for : https://huggingface.co/Undi95/Miqu-70B-Alpaca-DPO

Q3_K_M to be uploaded shortly.

Q3_K_S, IQ3_XXS, Q2_K, Q2_K_S, IQ2_XS, IQ2_XXS to follow.

LlamaCPP Benchs on a non iMatrix Q3_K_M released by Undi95 :

- Miqu-70B-DPO.q3_k_m.gguf,-,Hellaswag,84.5,400,,2024-02-07 00:00:00,,70b,Mistral_Medium,32768,,,GGUF,NeverSleep,NeverSleep,
- Miqu-70B-DPO.q3_k_m.gguf,-,Hellaswag,83.8,1000,,2024-02-07 00:00:00,,70b,Mistral_Medium,32768,,,GGUF,NeverSleep,NeverSleep,
- Miqu-70B-DPO.q3_k_m.gguf,-,Arc-Challenge,57.85953177,,299,2024-02-07 05:40:00,,70b,Mistral_Medium,32768,,,GGUF,NeverSleep,NeverSleep,
- Miqu-70B-DPO.q3_k_m.gguf,-,Arc-Easy,77.36842105,,570,2024-02-07 05:40:00,,70b,Mistral_Medium,32768,,,GGUF,NeverSleep,NeverSleep,
- Miqu-70B-DPO.q3_k_m.gguf,-,MMLU,50.15974441,,313,2024-02-07 05:40:00,,70b,Mistral_Medium,32768,,,GGUF,NeverSleep,NeverSleep,
- Miqu-70B-DPO.q3_k_m.gguf,-,Thruthful-QA,42.47246022,,817,2024-02-07 05:40:00,,70b,Mistral_Medium,32768,,,GGUF,NeverSleep,NeverSleep,
- Miqu-70B-DPO.q3_k_m.gguf,-,Winogrande,78.7687,,1267,2024-02-07 05:40:00,,70b,Mistral_Medium,32768,,,GGUF,NeverSleep,NeverSleep,
- Miqu-70B-DPO.q3_k_m.gguf,-,wikitext,4.3018,512,512,2024-02-07 00:00:00,,70b,Mistral_Medium,32768,,,GGUF,NeverSleep,NeverSleep,81
- Miqu-70B-DPO.q3_k_m.gguf,-,wikitext,3.8469,512,512,2024-02-07 00:00:00,,70b,Mistral_Medium,32768,,,GGUF,NeverSleep,NeverSleep,655

Quite convincing compared to the original Miqu.. with iMatrix :

- Miqu-1-70b-Requant-b1989-iMat-c32_ch400-Q3_K_M.gguf,-,Arc-Challenge,57.19063545,,299,2024-01-29 05:40:00,,70b,Mistral_Medium,32768,,,GGUF,- Miqudev,Nexesenex,
- Miqu-1-70b-Requant-b1989-iMat-c32_ch400-Q3_K_M.gguf,-,Arc-Easy,77.19298246,,570,2024-01-29 05:40:00,,70b,Mistral_Medium,32768,,,GGUF,- Miqudev,Nexesenex,
- Miqu-1-70b-Requant-b1989-iMat-c32_ch400-Q3_K_M.gguf,-,MMLU,50.15974441,,313,2024-01-29 05:40:00,,70b,Mistral_Medium,32768,,,GGUF,- Miqudev,Nexesenex,
- Miqu-1-70b-Requant-b1989-iMat-c32_ch400-Q3_K_M.gguf,-,Thruthful-QA,41.49326805,,817,2024-01-29 05:40:00,,70b,Mistral_Medium,32768,,,GGUF,- Miqudev,Nexesenex,
- Miqu-1-70b-Requant-b1989-iMat-c32_ch400-Q3_K_M.gguf,-,Winogrande,78.8477,,1267,2024-01-29 05:40:00,,70b,Mistral_Medium,32768,,,GGUF,- Miqudev,Nexesenex,
- Miqu-1-70b-Requant-b1989-iMat-c32_ch400-Q3_K_M.gguf,-,wikitext,4.2957,512,512,2024-01-29 00:00:00,RBF1000000,70b,Mistral_Medium,32768,,,GGUF,- Miqudev,Nexesenex,81
- Miqu-1-70b-Requant-b1989-iMat-c32_ch400-Q3_K_M.gguf,-,wikitext,3.8380,512,512,2024-01-29 00:00:00,RBF1000000,70b,Mistral_Medium,32768,,,GGUF,- Miqudev,Nexesenex,655

The TQA shows a slight bonus, thanks to the DPO training I believe.