maximum context length?
#1
by
hiauiarau
- opened
Good day, have you expanded the context for this model? What is the maximum context length now supported? I want to test it by increasing the inference in vllm.
Okay, I see the length is 16384.
But neither vllm nor sglang are starting up, I'm getting an error:
otImplementedError: Disabling sliding window is not supported for models with rope_scaling. Please raise an issue so we can investigate.
Even if I change the model config to "attn_logit_softcapping": null and "final_logit_softcapping": null, "rope_scaling": {
"type": "linear",
"factor": 2.0
} and
"sliding_window": 16384,
"sliding_window_size": 16384,