Any chance that this might get a 4bit?
#1
by
ShatteredBlue
- opened
A quantized version of this would be amazing! Shygmalion has been my fav and V8P2 has been my fav pygmalion branch but my gpu can't run the full model. I'd be eternally grateful ;n;