Model quantization request
Hey Mradermacher !
Could you add this model to your list?
https://huggingface.co/ycros/airoboros-65b-gpt4-1.4.1-PI-8192-fp16
(I currently lack of RAM to make a proper imatrix of this beast).
https://huggingface.co/bhenrym14 and Ycros have been releasing long context Airoboros last year, when the scaled and NTK ropes were a thing : those models were very good at the time, especially this one, and they are classics!
Course I can. Having good memory of airoboros, too. If I overlooked other classic models, feel free to drop me a note.
You can watch the model go through its stages at http://hf.tst.eu/status.html
Of course, llama.cpp already has trouble with this model (I had to remove a config setting).
Thank you very much.
Indeed, these older models config has to be checked, I remember that now (it's been a while I didn't quantize any of them).
It's more like llama.cpp stopped caring about them. Now let me try it out...