My GGUF-IQ-ARM-Imatrix quants for inflatebot/MN-12B-Mag-Mell-R1.

Mag Mell R1 was tested with Temp 1.25 and MinP 0.2. This was fairly stable up to 10K, but this might be too "hot". If issues with coherency occur, try increasing MinP or decreasing Temperature.

Use the ChatML prompt format.

Downloads last month: 2,426

GGUF

Model size

12.2B params

Architecture

llama

3-bit

4-bit

5-bit

6-bit

8-bit

16-bit

View +2 files

Inference API

Inference API (serverless) has been turned off for this model.

Model tree for Lewdiculous/MN-12B-Mag-Mell-R1-GGUF-IQ-ARM-Imatrix

Base model

inflatebot/MN-12B-Mag-Mell-R1

Quantized

(15)

this model

Collection including Lewdiculous/MN-12B-Mag-Mell-R1-GGUF-IQ-ARM-Imatrix

Quantized Models (GGUF, IQ, Imatrix)

Collection

Various quantizations of models in the GGUF format. Models with a "checkmark" are personal favorites. An "orange arrow" means it's being uploaded. • 90 items • Updated 11 days ago • 49