Spaces:
Running
on
Zero
Update app.py
since this model takes in conversation history, not restricting it to 100 tokens will occupy a lot of context window and I feel like 100 tokens is pretty chatty already
since this model takes in conversation history, not restricting it to 100 tokens will occupy a lot of context window and I feel like 100 tokens is pretty chatty already
A 100-token context window is quite limited, often insufficient to complete examples. An expansion of context window would be beneficial.
if you can open a PR to add slider for max_new_tokens I will merge. max_new_tokens per se isn't about context window but rather how many tokens are generated by the model, so having less max_new_tokens means more memorization for previous responses, and this is a chatty model. moreover, more max_new_tokens often results in hallucination and weird responses. thus I don't want to hardcode a large value