Create README.md

README.md

@@ -0,0 +1 @@
+It's Danube3 500M with <|im_start|> and <|im_end|> tags trained in. First, one LoRA finetune on the uninstruct dataset, then I modified the vocabulary and ran a second LoRA, this time including embed_tokens and lm_head. Useful for quick hyperparameter optimization where you don't want to train lm_head and embed_tokens but you want to have <|im_start|> and <|im_end|> tokens trained in already.