Update README.md
Browse files
README.md
CHANGED
@@ -143,7 +143,7 @@ model-index:
|
|
143 |
# 🧙 Coven 7B 128K ORPO
|
144 |
|
145 |
|
146 |
-
Coven 7B 128K is an improved iteration of Mistral-7B-Instruct-v0.2, refined to expand processing capabilities and refine language model preferences. This model includes a significantly increased context constraint of 128K tokens using the Yarn technique, which allows for more extensive data processing and understanding of complex language scenarios. In addition, the Coven 7B
|
147 |
|
148 |
|
149 |
### Eval
|
|
|
143 |
# 🧙 Coven 7B 128K ORPO
|
144 |
|
145 |
|
146 |
+
Coven 7B 128K is an improved iteration of Mistral-7B-Instruct-v0.2, refined to expand processing capabilities and refine language model preferences. This model includes a significantly increased context constraint of 128K tokens using the [Yarn](https://github.com/jquesnelle/yarn) technique, which allows for more extensive data processing and understanding of complex language scenarios. In addition, the Coven 7B ORPO 128K tokenization uses the innovative ORPO (Monolithic Preference Optimization without Reference Model) technology. ORPO simplifies the fine-tuning process by directly optimizing the odds ratio to distinguish between favorable and unfavorable generation styles, effectively improving model performance without the need for an additional preference alignment step.
|
147 |
|
148 |
|
149 |
### Eval
|