|
--- |
|
license: apache-2.0 |
|
language: |
|
- fr |
|
pipeline_tag: text-generation |
|
--- |
|
|
|
- Model : https://huggingface.co/GeneZC/MiniChat-1.5-3B |
|
- FT : @teilomillet |
|
|
|
- Instruction tune using QLoRA on a french dataset for 1 epoch. |
|
The aim was to test and try the dataset. Implementing a customization via a dataset and fine-tuning on it. |
|
The way to respond is also important to see if it's taken from the dataset and add to the customization. |
|
|
|
This is the first of a long serie of multiple models. Aimed to be minuscule as possible. |
|
|
|
- Batch : 6 |
|
- Gradient step : 1 |
|
- Epoch : 1 |
|
- Lr : 0.0002 |
