The training script missing
The training script for v0.2 seems to be missing. In v0.1 does have train_unsloth_7b.py.
I'm trying to figure out what special tokenizations and instruction formats were used for the different datasources/tasks, but can't seem to find info about it.
Kiitos paljon, että saadaan suomenkielisiäkin malleja! 🔥🇫🇮
We are soon releasing new 3b and 7b models.
I will later add the finetuning scripts once those are published and once I have prepared the finetuning tutorial notebooks.
Unsloth which I have mainly used on my own finetuning trials unfortunately does not work on Google Colab for our upcoming 3b model so I will need to do it with peft.
But probably I will also share a finetuning sample with Unsloth which works on newer GPUs.
You can find finetuning example from here: https://huggingface.co/Finnish-NLP/Ahma-3B/blob/main/Finetune_Ahma_3B_example.ipynb
Along with a video: https://www.youtube.com/watch?v=6mbgn9XzpS4