Repetition Loops

#1
by chrisoutwright - opened

This model seems to repeat a lot on specific prompts or parts in the prompt:

image

Or is this only happening in specific circumstances?

Have you tried adjusting Repetition penalty/Frequency penalty/Presence penalty? Sometimes this help but if it is that extreme it could also just be the that the model wasn't trained/merged well.

These are my params:

{
"repeat_penalty": 1.05,
"stop": ["<|im_start|>", "<|im_end|>", "<|endoftext|>"],
"temperature": 0.7,
"top_k": 20,
"top_p": 0.8,
"num_gpu":91,
"num_ctx":90000
}

Sign up or log in to comment