|
--- |
|
language: |
|
- en |
|
license: apache-2.0 |
|
library_name: transformers |
|
tags: |
|
- orpo |
|
- trl |
|
datasets: |
|
- alvarobartt/dpo-mix-7k-simplified |
|
base_model: mistralai/Mistral-7B-v0.1 |
|
pipeline_tag: text-generation |
|
inference: false |
|
--- |
|
|
|
## ORPO fine-tune of Mistral 7B v0.1 with DPO Mix 7K |
|
|
|
 |
|
|
|
> Stable Diffusion XL "A capybara, a killer whale, and a robot named Ultra being friends" |
|
|
|
This is an ORPO fine-tune of [`mistralai/Mistral-7B-v0.1`](https://huggingface.co/mistralai/Mistral-7B-v0.1) with |
|
[`alvarobartt/dpo-mix-7k-simplified`](https://huggingface.co/datasets/alvarobartt/dpo-mix-7k-simplified). |
|
|
|
⚠️ Note that the code is still experimental, as the `ORPOTrainer` PR is still not merged, follow its progress |
|
at [🤗`trl` - `ORPOTrainer` PR](https://github.com/huggingface/trl/pull/1435). |
|
|
|
## Reference |
|
|
|
[`ORPO: Monolithic Preference Optimization without Reference Model`](https://huggingface.co/papers/2403.07691) |
