|
--- |
|
license: apache-2.0 |
|
language: |
|
- en |
|
library_name: transformers |
|
pipeline_tag: text-generation |
|
--- |
|
|
|
[Falcon 7b](https://huggingface.co/tiiuae/falcon-7b) model trained on [FAQ from an ecommerce website](https://www.kaggle.com/datasets/saadmakhdoom/ecommerce-faq-chatbot-dataset). |
|
|
|
## Tutorials |
|
|
|
- Text tutorial: https://www.mlexpert.io/prompt-engineering/fine-tuning-llm-on-custom-dataset-with-qlora |
|
- YouTube video: https://www.youtube.com/watch?v=DcBC4yGHV4Q |
|
|
|
## Training |
|
|
|
Fine-tuned using the QLoRA technique (only adapter uploaded). With the help of bitsandbytes, peft and transformers. Full reproduction is available in the tutorials. |
