Q6 quant?
id like to get to know this model a bit more. it looks like a wild mix of ingredients and im curios what its capable of especially being both V5 and gold, my two favorites. could i request a Q6? i can run 8, but its just slightly too big for my 16gb and id rather offload all layers into GPU with 6.
and my i also ask for an elaboration on the base for this merge?
I remember testing it and not liking it, so I didn't bother asking for quants.
For the base, I had one failed merge on top of another failed merge on top of another, just trying to make it better by adding more finetunes. That obviously didn't work out very well.
Siskin v0.1 and v0.2 is a more successful models and I would recommend using those.
If you still want to try this one, I've tried making a gguf myself for the first time since gguf-my-repo doesn't seem to quantize the same model twice.
Not sure if it works, but here you go:
https://huggingface.co/Nohobby/RunningAround1-Q8_0-GGUF/resolve/main/RunningAround1_Q6_K.gguf?download=true