Can you help the efforts to integrate the fine-tuning of gemma-2 it in Torchtune ?
#5
by
almugabo
- opened
with their multilingual capabilities gemma-2 models are a good candidates to serve as base of models in low resources languages.
There are good frameworks for fine-tuning gemma-2 but they either
- don't offer full fine tuning (e.g. Unsloth)
- or are resources optimized (e.g. Axolotl )
- or are not very widely used in the community because they are tailored to gemma-2 (like the fine-tuning approach gemma-2 offered using Keras).
There is a need for a way to full-finetune gemma-2 using existing fine-tuning frameworks which work with different LLM and are resources optimized. A good candidate for that is Torchtune.
Torchtune already works well with gemma-1 but not yet with gemma-2 .
Could you help the efforts to integrate the fine-tuning of gemma-2 it in Torchtune ?
Here an issue which has been opened on that :