Model quantization request

#1
by Nexesenex - opened

Hey Mradermacher !

Could you add this model to your list?

https://huggingface.co/ycros/airoboros-65b-gpt4-1.4.1-PI-8192-fp16

(I currently lack of RAM to make a proper imatrix of this beast).

https://huggingface.co/bhenrym14 and Ycros have been releasing long context Airoboros last year, when the scaled and NTK ropes were a thing : those models were very good at the time, especially this one, and they are classics!

Course I can. Having good memory of airoboros, too. If I overlooked other classic models, feel free to drop me a note.

You can watch the model go through its stages at http://hf.tst.eu/status.html

mradermacher changed discussion status to closed

Of course, llama.cpp already has trouble with this model (I had to remove a config setting).

Thank you very much.

Indeed, these older models config has to be checked, I remember that now (it's been a while I didn't quantize any of them).

It's more like llama.cpp stopped caring about them. Now let me try it out...

Sign up or log in to comment