Prompt format different in dataset compared to model card
In the model card it says the expected format is [INST]\n<text>\n[/INST]\n\n however after looking at the dataset and the code it appears that the actual format it was trained on was [INST] <text> [/INST]
Is the model card incorrect or is there something I'm missing? It seems like it was never trained on the \n version of the prompt. Sort of works with the \n variant, but I seem to be getting better consistency and instruction following with the variant found in code/dataset.
bump
Hi
@bhperry
, great catch! You are right, during training we used [INST] <text> [/INST], but we found that adding two newline characters during inference works best. If in your case however, omitting the newlines works best, then you can safely use the template without the newline characters.
Interesting... thanks! After more experimentation, I seem to be getting the best results with a mix of the two formats. [INST] <text> [/INST]\n\n