---
library_name: transformers
tags: []
---

- this model was trained to classify whether input text comes from "chosen sentence" or "rejected sentence"
- the probability (logits after passing softmax function) in last layer of this model can be used to quantify the preference from user input
- fine-tuned [Rakuten/RakutenAI-7B-instruct](https://huggingface.co/Rakuten/RakutenAI-7B-instruct) via [LoRA](https://arxiv.org/abs/2106.09685) using [open-preference-v0.3](https://huggingface.co/datasets/ryota39/open_preference-v0.3)
- trained on bf16 format

## Metric

![image/png](https://cdn-uploads.huggingface.co/production/uploads/651e3f30ca333f3c8df692b8/-V76_Tj2ltNFn1YQWX8BJ.png)

- validation

|accuracy|recall|precision|f1-score|
|:---|:---|:---|:---|
|0.9694|0.9757|0.9636|0.9696|

- test

|accuracy|recall|precision|f1-score|
|:---|:---|:---|:---|
|0.5162|0.8822|0.5093|0.6458|

- confusion matrix
  - x-axis shows ground truth
  - y-axis shows prediction

![image/png](https://cdn-uploads.huggingface.co/production/uploads/651e3f30ca333f3c8df692b8/Rg2W9qdSAFumCkq9z1G7N.png)