---
library_name: transformers
tags:
- unsloth
license: mit
datasets:
- efederici/alpaca-vs-alpaca-orpo-dpo
---

# Model Card for Model ID

This is phi-3-mini-4k-instruct ORPO finetuning for the italian language over the Alpaca vs. Alpaca italian dataset: [efederici/alpaca-vs-alpaca-orpo-dpo](https://huggingface.co/datasets/efederici/alpaca-vs-alpaca-orpo-dpo)

## Model Details

### Model Description

- **Developed by:** Diego Giorgini
- **Funded by:** AI Technologies SRL - www.aitechnologies.it
- **Language(s) (NLP):** Italian
- **License:** llama3
- **Finetuned from model:** unsloth/Phi-3-mini-4k-instruct

## Training Details

### Environment

unsloth: 2024.5  
torch: 2.2

### Training Data

[efederici/alpaca-vs-alpaca-orpo-dpo](https://huggingface.co/datasets/efederici/alpaca-vs-alpaca-orpo-dpo): The Alpaca vs. Alpaca dataset is a curated blend of the Alpaca dataset and the Alpaca GPT-4 dataset, both available on HuggingFace Datasets. It uses the standard GPT dataset as the 'rejected' answer, steering the model towards the GPT-4 answer, which is considered as the 'chosen' one.

### Training Procedure

#### Preprocessing [optional]

- No preprocessing has been performed, except for formatting with the phi-3 chat_template from unsloth:

  ```tokenizer = get_chat_template(tokenizer, chat_template = "phi-3")```

#### Training Hyperparameters

- **Training regime:** bf16

- **Model loading parameters:**

```
max_seq_length = 8192
dtype = None
load_in_4bit = False
```

- **PEFT parameters:**

```
r = 64  
lora_alpha = 64  
lora_dropout = 0  
bias = "none"  
random_state = 3407  
use_rslora = False  
loftq_config = None
```

- **ORPOConfig parameters:**

```
max_length = 8192  
max_prompt_length = max_seq_length//2  
max_completion_length = max_seq_length//2  
warmup_ratio = 0.1  
weight_decay = 0.01  
per_device_train_batch_size = 1  
gradient_accumulation_steps = 16  
learning_rate=8e-6  
beta = 0.1  
optim = "paged_adamw_8bit"  
lr_scheduler_type = "linear"  
num_train_epochs = 1
```

#### Speeds, Sizes, Times

7h on an A100-40GB

## Model Card Contact

diego.giorgini@icloud.com