Set "use_cache": true for a speedup.

#4
by autobots - opened

Not much of one but at least it makes it somewhat usable.

Same here!
It gave about 1 token/s.
However, Text genertion web UI gave me about 7 tokens/s.

Your need to confirm your account before you can post a new comment.

Sign up or log in to comment