README.md

---
license: other
pipeline_tag: text-generation
datasets:
- cosimoiaia/Loquace-102k
language:
- it
---
# Model Card for Model raicrits/OpenLLama13b_Loquace_ITA

<!-- Provide a quick summary of what the model is/does. -->
An open-source LLaMa language model of 13b parameters fine-tuned to follow instructions in italian.


### Model Description

This model is an open-source LLM of 13b parameters based on [OpenLLaMA](https://github.com/openlm-research/open_llama), an open-source replica of Meta AI's LLaMA.
The model was fine-tuned in order to follow instructions, as proposed in [Alpaca](https://github.com/tatsu-lab/stanford_alpaca), 
but using [LoRA](https://arxiv.org/pdf/2106.09685.pdf) technique and a bigger dataset of instruction/answers in italian, [cosimoiaia/Loquace-102k](https://huggingface.co/datasets/cosimoiaia/Loquace-102k/viewer/cosimoiaia--Loquace-102k).

This repository contains the model merged with the LoRA adapters obtained in the fine-tuning procedure.


- **Developed by:** Stefano Scotta (stefano.scotta@rai.it)
- **Model type:** LLM fine-tuned to follow instructions
- **Language(s) (NLP):** Italian
- **License:** Other
- **Finetuned from model:** [openlm-research/open_llama_13b](https://huggingface.co/openlm-research/open_llama_13b)


## Uses

<!-- Address questions around how the model is intended to be used, including the foreseeable users of the model and those affected by the model. -->
The model can be used as is to respond to simple instructions in Italian or can be further fine-tuned to perform specific tasks.



## Bias, Risks, and Limitations

<!-- This section is meant to convey both technical and sociotechnical limitations. -->
As any other LLM it is possible that the model generates content which does not correspond to the reality as well as wrong, biased, offensive and inappropriate answers.


## How to Get Started with the Model
 **Prompt template:**

 ``` python
"Di seguito è riportata un'istruzione che descrive un compito, abbinata a un input che fornisce un ulteriore contesto. Scrivete una risposta che completi in modo appropriato la richiesta.

### Istruzione:
{instruction}

### Input:
{input}

### Risposta:"
```

 **Usage:**
Use the code below to get started with the model.
 ``` python
import os
import torch
import sys
from transformers import LlamaTokenizer, LlamaForCausalLM

if torch.cuda.is_available():
    device = "cuda"
else:
    device = "cpu"

def generate_prompt(instruction, input=None):
    if input:
        return f"""Di seguito è riportata un'istruzione che descrive un compito, abbinata a un input che fornisce un ulteriore contesto. Scrivete una risposta che completi in modo appropriato la richiesta.

### Istruzione:
{instruction}

### Input:
{input}

### Risposta:"""
    else:
        return f"""Di seguito è riportata un'istruzione che descrive un compito. Scrivete una risposta che completi in modo appropriato la richiesta..

### Istruzione:
{instruction}

### Risposta:"""

model_name = "raicrits/OpenLLama13b_Loquace_ITA"

model = LlamaForCausalLM.from_pretrained(
    model_name,
    torch_dtype=torch.float16,
    device_map="auto"
)

tokenizer = LlamaTokenizer.from_pretrained(model_name)

instruction = "qual'è la relazione tra i seguenti oggetti"
input = "sedia, tavolo, divano"
prompt = generate_prompt("instruction", input)
inputs = tokenizer(prompt, return_tensors="pt")
input_ids = inputs["input_ids"].to(device)

generation_output = model.generate(
    input_ids=input_ids,
    max_new_tokens=256,
)


output = tokenizer.decode(generation_output[0])
output = output.split("### Risposta:")[1].strip().replace("</s>","")
print(output)
```
``` python
"Sedia, tavolo e divano sono tutti oggetti che possono essere utilizzati per creare un'atmosfera rilassante in una stanza."
```

## Training Details

### Training Data

<!-- This should link to a Data Card, perhaps with a short stub of information on what the training data is all about as well as documentation related to data pre-processing or additional filtering. -->
The model was fine-tinuned on [cosimoiaia/Loquace-102k](https://huggingface.co/datasets/cosimoiaia/Loquace-102k/viewer/cosimoiaia--Loquace-102k), a dataset of 102k question/answer pairs in italian.


### Training Procedure 

<!-- This relates heavily to the Technical Specifications. Content here should link to that section when it is relevant to the training procedure. -->
The fine-tuning procedure was done using [LoRA](https://arxiv.org/pdf/2106.09685.pdf) approach following closely what done for fine-tuning models like [Alpaca-LoRA](https://github.com/tloen/alpaca-lora).


#### Training Hyperparameters

 **Training setting:**
- train epochs=3,

- learning_rate=3e-4,

- optimizer="adamw_hf"

- mixed precision training: float16 

 **LoRA configuration:**
-  r= 8
  
- lora_alpha=16
  
- target_modules=["q_proj","v_proj"]

- lora_dropout=0.05
  
- bias="none"
  
- task_type=TaskType.CAUSAL_LM
  


## Environmental Impact

<!-- Total emissions (in grams of CO2eq) and additional considerations, such as electricity usage, go here. Edit the suggested text below accordingly -->

Carbon emissions can be estimated using the [Machine Learning Impact calculator](https://mlco2.github.io/impact#compute) presented in [Lacoste et al. (2019)](https://arxiv.org/abs/1910.09700).

- **Hardware Type:** 1 NVIDIA A100/40Gb
- **Hours used:** 68
- **Cloud Provider:** Private Infrastructure
- **Carbon Emitted:** 7.34 kg eq. CO2



## Model Card Authors

Stefano Scotta (stefano.scotta@rai.it)

## Model Card Contact

stefano.scotta@rai.it