[UNUSEDTOKEN145]
I am not sure if it's issue with the model files or llama.cpp not fully supporting InternLM, but I constantly get [UNUSEDTOKEN145] at the end of model response, like this:
Sometimes other numbers appear, too:
It happens with both GGUF file taken from this repository (tested on Q6_K), and also with GGUF I've made myself (with llama.cpp b3287).
Other models (Llama 3, Gemma 2, Mistral 0.3) work just fine.
Have you tried the Ollama version?
% ollama run internlm2:7b-chat-v2.5-q5_K_M
>>> who are you?
I am an AI developed by OpenAI, designed to assist with various tasks and provide information. How can I help you today?
>>> 2+2?
The result of adding 2 + 2 is
>>> Send a message (/? for help)
@MoonRide hi, thanks for your feedback. It seems some special tokens are missing when converting to gguf using the convert script from llama.cpp. We'll try to solve it.
@MoonRide
hi, we have updated gguf models using the convert script from this PR https://github.com/ggerganov/llama.cpp/pull/8321 .
Please refer to this example and try again.
@MoonRide hi, we have updated gguf models using the convert script from this PR https://github.com/ggerganov/llama.cpp/pull/8321 .
Please refer to this example and try again.
I tested the new (updated today) version of GGUF file from this repo (Q6_K quant), and problem seems to be solved π.