interleaved_sliding_window vs sliding_window for TextConfig
#4
by
jitanus
- opened
Did someone at vLLM decided to make that change for gemma2, breaking MERaLION?
Hi, did you upgraded vllm? They might have changed the gemma2model code from 0.6.5. For now, setting vllm==0.6.4.post1 still can work.
Yes I did! It was two lines of code that I locally edited in vllm back to sliding_window to make it work. I just wanted to alert you. Thanks.
Thanks for sharing this with us! Btw the latest hf code/vllm plugin support both fast and slow tokenizers now :)
jitanus
changed discussion status to
closed