question about tokens limit
#4
by
sunzx0810
- opened
when testing the model in a local environment by directly running the code, I input a 567 tokens' question and raise error 'list index out of range' which seems to tell me that the output exceeds in 2000 tokens. However, the question works well both on demo and text generation webui. why webui can generate answers word by word? And I also find that the response from the local model will try to edit the instructions but the webui and demo will not. It's very confusing.