Why is this model so sloooooooow in text-generation-webui?
#6
by
cleverest
- opened
This is the only 30b model that is unusable due to this incredible slowdown...What do I need to do to speed this up to a usable speed? I have a 4090, 96GB of Ram, I9-13900k system.
Same here, my system us: Windows 11, RTX4090, 65GB RAM, i7 13700KF.
I have around 0.2 t/s. I wish someone could make not 128g version :(
Glad it's not just me! I wish I could try this out and actually use it. It's mind-numbingly slow and tries to be verbose while being that way...
Yeah, I get this (0.60) and it's mind-numbingly annoyingly unusable. It's clearly broken. No other 30b is even close to being this slow.
I guess they don't care to respond with a fix or a solution upcoming, etc...seems random and rambly anyways of a model.