File size: 1,194 Bytes
618e5ae 482bf4f 618e5ae 20194dd 618e5ae |
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 |
---
license: cc-by-nc-4.0
tags:
- DPO
---
# SJ-Donald/SJ-SOLAR-10.7b-DPO
SJ-Donald/SJ-SOLAR-10.7b-DPO is fine-tuned using DPO method.
## Environment
Using **Google CoLab A100**
## Base model
* [SJ-Donald/SOLAR-10.7B-slerp](https://huggingface.co/SJ-Donald/SOLAR-10.7B-slerp)
## Datasets
* [SJ-Donald/orca-dpo-pairs-ko](https://huggingface.co/datasets/SJ-Donald/orca-dpo-pairs-ko)
## Benchmark
### Open-LLM-Leaderboard(https://huggingface.co/spaces/HuggingFaceH4/open_llm_leaderboard)
| Average | ARC | HellaSwag | MMLU | TruthfulQA | Winogrande | GSM8K |
| ------: | -----: | -----------: | ------: | -----------: | ---------: | ------: |
| 72.67 | 68.26 | 86.95 | 66.73 | 67.74 | 84.21 | 62.03 |
## How to use
```Python
import torch
from transformers import AutoModelForCausalLM, AutoTokenizer
repo = 'SJ-Donald/SJ-SOLAR-10.7b-DPO'
tokenizer = AutoTokenizer.from_pretrained(repo)
model = AutoModelForCausalLM.from_pretrained(
repo,
return_dict=True,
torch_dtype=torch.float16,
device_map='auto'
)
```
## Chat Template
```Python
template = """### System:
{{system_content}}
### User:
{{question}}
### Assistant:
"""
``` |