Update README.md

---
language: en
license: apache-2.0
tags:
- instruct
- gpt
- text-generation
- rubra-ai
- mistral
datasets:
- custom-dataset
metrics:
- accuracy
- perplexity
model-index:
- name: Mistral-7B-Instruct-v0.2
results:
- task:
type: text-generation
dataset:
name: custom-dataset
type: custom
metrics:
- name: Accuracy
type: accuracy
value: 98.5
- name: Perplexity
type: perplexity
value: 15.2
---

# Rubra Mistral-7B-Instruct-v0.2

## Model description
The model is the result of further post-training [mistralai/Mistral-7B-Instruct-v0.2](https://huggingface.co/mistralai/Mistral-7B-Instruct-v0.2). It is capable of complex tool/function calling.

## How to use

You can use the model with the Hugging Face `transformers` and the rubra library [rubra-tools](https://github.com/rubra-ai/rubra-tools) as follows:

```
pip install rubra_tools torch==2.3.0 transformers
```

```python
from transformers import AutoTokenizer, AutoModelForCausalLM
from rubra_tools import preprocess_input, postprocess_output
import torch

model_id = "rubra-ai/Mistral-7B-Instruct-v0.2"

tokenizer = AutoTokenizer.from_pretrained(model_id)
model = AutoModelForCausalLM.from_pretrained(
model_id,
torch_dtype=torch.bfloat16,
device_map="auto",
)

functions = [
{
'type': 'function',
'function': {
'name': 'addition',
'description': "Adds two numbers together",
'parameters': {
'type': 'object',
'properties': {
'a': {
'description': 'First number to add',
'type': 'string'
},
'b': {
'description': 'Second number to add',
'type': 'string'
}
},
'required': []
}
}
},
{
'type': 'function',
'function': {
'name': 'subtraction',
'description': "Subtracts two numbers",
'parameters': {
'type': 'object',
'properties': {
'a': {
'description': 'First number to be subtracted from',
'type': 'string'
},
'b': {
'description': 'Number to subtract',
'type': 'string'
}
},
'required': []
}
}
},
{
'type': 'function',
'function': {
'name': 'multiplication',
'description': "Multiply two numbers together",
'parameters': {
'type': 'object',
'properties': {
'a': {
'description': 'First number to multiply',
'type': 'string'
},
'b': {
'description': 'Second number to multiply',
'type': 'string'
}
},
'required': []
}
}
},
{
'type': 'function',
'function': {
'name': 'division',
'description': "Divide two numbers",
'parameters': {
'type': 'object',
'properties': {
'a': {
'description': 'First number to use as the dividend',
'type': 'string'
},
'b': {
'description': 'Second number to use as the divisor',
'type': 'string'
}
},
'required': []
}
}
},
]

messages = [
{"role": "system", "content": "You are a helpful assistant."},
{"role": "user", "content": "What is the result of four plus six? Take the result and add 2? Then multiply by 5 and then divide by two"},
]

def run_model(messages, functions):
## Format messages in Rubra's format
formatted_msgs = preprocess_input(msgs=messages, tools=tools)

input_ids = tokenizer.apply_chat_template(
formatted_msgs,
add_generation_prompt=True,
return_tensors="pt"
).to(model.device)

terminators = [
tokenizer.eos_token_id,
tokenizer.convert_tokens_to_ids("<|eot_id|>")
]

outputs = model.generate(
input_ids,
max_new_tokens=1000,
eos_token_id=terminators,
do_sample=True,
temperature=0.1,
top_p=0.9,
)
response = outputs[0][input_ids.shape[-1]:]
raw_output = tokenizer.decode(response, skip_special_tokens=True)
return raw_output

raw_output = run_model(messages, functions)
# Check if there's a function call
function_call = postprocess_output(raw_output)
if function_call:
print(function_call)
else:
print(raw_output)
```

You should see this output, which is a function call made by the ai assistant:
```
[{'id': 'fc65a533', 'function': {'name': 'addition', 'arguments': '{"a": "4", "b": "6"}'}, 'type': 'function'}]
```

You can continue the conversation by provide the function call result:

```python
if function_call:
# append the assistant tool call msg
messages.append({"role": "assistant", "tool_calls": function_call})
# append the result of the tool call in openai format, in this case, the value of add 6 to 4 is 10.
messages.append({'role': 'tool', 'tool_call_id': function_call[0]["id"], 'name': function_call[0]["function"]["name"], 'content': '10'})
raw_output = run_model(messages, functions)
# Check if there's a function call
function_call = postprocess_output(raw_output)
if function_call:
print(function_call)
else:
print(raw_output)
```

The AI will make another call
```
[{'id': '2ffc3de4', 'function': {'name': 'addition', 'arguments': '{"a": "10", "b": "2"}'}, 'type': 'function'}]
```


## Training Data

The model was post-trained (freeze tuned & DPO) on a proprietary dataset consisting of diverse function calling, chat, and instruct data.

## Training Hyperparameters

The following hyperparameters were used during training:
- learning_rate: 2e-05
- train_batch_size: 2
- eval_batch_size: 8
- seed: 42
- gradient_accumulation_steps: 12
- total_train_batch_size: 24
- optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
- lr_scheduler_type: cosine
- num_epochs: 1.0

## Framework Versions

- Transformers 4.41.2
- Pytorch 2.3.1+cu121
- Datasets 2.19.2
- Tokenizers 0.19.1

## Evaluation

<table>
<thead>
<tr>
<th rowspan="2">Model</th>
<th rowspan="2">Function Calling</th>
<th rowspan="2">MMLU</th>
<th rowspan="2">GPQA</th>
<th rowspan="2">GSM-8K</th>
<th rowspan="2">MATH</th>
<th rowspan="2">MT-bench</th>
<th colspan="6">MT-bench Pairwise Comparison</th>
</tr>
<tr>
<th>Win</th>
<th>Loss</th>
<th>Tie</th>
<th>Win Rate</th>
<th>Loss Rate</th>
<th>Adjusted Win Rate</th>
</tr>
</thead>
<tbody>
<tr>
<td>Mistral 7B Instruct v0.2</td>
<td>-</td>
<td>59.27</td>
<td>27.68</td>
<td>43.21</td>
<td>10.30</td>
<td>7.50</td>
<td>34</td>
<td>54</td>
<td>72</td>
<td>0.2125</td>
<td>0.3375</td>
<td>0.4375</td>
</tr>
<tr>
<td>Rubra Enhanced Mistral 7B Instruct v0.2</td>
<td>69.28%</td>
<td>58.90</td>
<td>29.91</td>
<td>34.12</td>
<td>8.36</td>
<td>7.36</td>
<td>59</td>
<td>43</td>
<td>58</td>
<td>0.36875</td>
<td>0.26875</td>
<td><strong>0.55</strong></td>
</tr>
</tbody>
</table>

## Limitations and Bias

While the model performs well on a wide range of tasks, it may still produce biased or incorrect outputs. Users should exercise caution and critical judgment when using the model in sensitive or high-stakes applications. The model's outputs are influenced by the data it was trained on, which may contain inherent biases.

## Ethical Considerations

Users should ensure that the deployment of this model adheres to ethical guidelines and consider the potential societal impact of the generated text. Misuse of the model for generating harmful or misleading content is strongly discouraged.

## Acknowledgements

We would like to thank Mistral for the model and LLaMA-Factory for training utilities.

## Contact Information

For questions or comments about the model, please reach out to [the rubra team](mailto:rubra@acorn.io).

## Citation

If you use this work, please cite it as:



@misc

{rubra202406mistral7binstructv02,
title = {Rubra-Mistral-7B-Instruct-v0.2},
author = {Sanjay Nadhavajhala and Yingbei Tong},
year = {2024},
publisher = {Hugging Face},
howpublished = {\url{https://huggingface.co/rubra-ai/Mistral-7B-Instruct-v0.2}},
}

README.md

@@ -1,3 +1,7 @@
+---
+language:
+- en
+---
 ---
 ---
 
@@ -27,4 +31,4 @@ The following hyperparameters were used during training:
 - Transformers 4.41.2
 - Pytorch 2.3.1+cu121
 - Datasets 2.19.2
-- Tokenizers 0.19.1
+- Tokenizers 0.19.1