README.md

---
datasets:
- tatsu-lab/alpaca
language:
- en
pipeline_tag: text2text-generation
library_name: transformers
license: other
---


# Model Details

- **Model name:** Flan-UL2-Alpaca-LoRA
- **Model type:** - Text2Text Generation
- **Parent Model:** [google/flan-ul2](https://huggingface.co/google/flan-ul2)
- **Training dataset:** [Alpaca](https://huggingface.co/datasets/tatsu-lab/alpaca) 
- **Language:** English
- **Framework:** PyTorch
- **Model version:** 1.0


We take the instruction-tuned Flan models (trained on Academic datasets) and perform style transfer using the Alpaca dataset.

We released the code for LORA fine-tuning Seq2Seq models along with code walkthrough medium article here:
- https://github.com/vmware-labs/research-and-development-artificial-intelligence-lab/tree/main/instruction-tuning/peft-seq2seq
- https://medium.com/vmware-data-ml-blog/lora-finetunning-of-ul-2-and-t5-models-35a08863593d

We fine-tuned the `google/flan-ul2` model on the Alpaca dataset using [PEFT-LORA](https://huggingface.co/docs/diffusers/main/en/training/lora). 

# License
- Parent model ([google/flan-ul2](https://huggingface.co/google/flan-ul2)): Apache 2.0
- Dataset ([Alpaca](https://huggingface.co/datasets/tatsu-lab/alpaca)) : cc-by-4.0
- text-davinci-003 (Used to generate Alpaca): [OpenAI Terms of Use](https://openai.com/policies/terms-of-use)



# How to Use

```
import torch
from transformers import pipeline

# Chose the model inference precision
dtype = torch.float16 # options are torch.float16, torch.bfloat16, torch.float32

model = pipeline(
    model = "VMware/flan-ul2-alpaca-lora",
    device_map = 'auto',
    torch_dtype=dtype
)

prompt = "YOUR PROMPT HERE"

output = model(prompt, max_length=2048, do_sample=True)
```


Using Alpaca prompt template might generate better outputs for certain prompts as the model was trained using the bellow template.

```
# Chose the model inference precision
import torch
from transformers import pipeline

dtype = torch.float16 # options are torch.float16, torch.bfloat16, torch.float32
model = pipeline(model="VMware/flan-ul2-alpaca-lora",device_map = 'auto',torch_dtype=dtype )

prompt_template = """
Below is an instruction that describes a task. Write a response that appropriately completes the request.

### Instruction:
{instruction}

### Response:"""

prompt = "YOUR INSTRUCTION HERE"

output = model(prompt_template.format(instruction=prompt), max_length=2048)
```

# Training Details

The model was trained on 3xV100 GPUs using PEFT-LORA and Deepspeed

* Hyperparameters:
* learning_rate = 3e-4
* per_device_batch_size = 2
* gradient_accumulation_steps = 21 
*  epochs = 3


# Limitations and Bias

The model is based on a large and diverse dataset, but it may still have limitations and biases in certain areas. Some limitations include:

- Language: The model is designed to work with English text only and may not perform as well in other languages.


In addition, the model may have some bias in terms of the data it was trained on. The dataset includes questions from a variety of sources, but it may not be representative of all populations or perspectives. As a result, the model may perform better or worse for certain types of questions or on certain types of texts.