Inference seems to be broken with latest llama.cpp llama-server
#2
by
AIWintermuteAI
- opened
The model outputs gibberish. Tried both./llama-server --model qwen2-0_5b-instruct-q5_0.gguf
and./llama-server --model qwen2-0_5b-instruct-q5_0.gguf --interactive-first --in-prefix '<|im_start|>user\n' --in-suffix '<|im_end|>\n<|im_start|>assistant\n' --reverse-prompt '<|im_end|>' -p '<|im_start|>system\nYou are a helpful assistant<|im_end|>'
Any ideas?
AIWintermuteAI
changed discussion title from
Inference seems to be broken with latest llamap.cpp llama-server
to Inference seems to be broken with latest llama.cpp llama-server