infer speed so slow and mutil-gpu infer bug

#17
by YuYuyanzu - opened

use sentence_trancsfomer encode sentence.

simple A100 gpu , batch size = 6 need to 6s!!!

Alibaba-NLP org

It seems a bit odd, could you upload your code?

Sign up or log in to comment