Update README.md
Browse files
README.md
CHANGED
@@ -122,6 +122,18 @@ I made it due to popular demand but I haven't tested it so use it at your own ri
|
|
122 |
|
123 |
It might be good for creative writing tasks. I recommend a context length of 32k but you can go up to 131,072 tokens in theory.
|
124 |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
125 |
## 🧩 Configuration
|
126 |
|
127 |
The following YAML configuration was used to produce this model:
|
@@ -176,17 +188,4 @@ pipeline = transformers.pipeline(
|
|
176 |
|
177 |
outputs = pipeline(prompt, max_new_tokens=256, do_sample=True, temperature=0.7, top_k=50, top_p=0.95)
|
178 |
print(outputs[0]["generated_text"])
|
179 |
-
```
|
180 |
-
# [Open LLM Leaderboard Evaluation Results](https://huggingface.co/spaces/open-llm-leaderboard/open_llm_leaderboard)
|
181 |
-
Detailed results can be found [here](https://huggingface.co/datasets/open-llm-leaderboard/details_mlabonne__BigQwen2.5-52B-Instruct)
|
182 |
-
|
183 |
-
| Metric |Value|
|
184 |
-
|-------------------|----:|
|
185 |
-
|Avg. |37.42|
|
186 |
-
|IFEval (0-Shot) |79.29|
|
187 |
-
|BBH (3-Shot) |59.81|
|
188 |
-
|MATH Lvl 5 (4-Shot)|17.82|
|
189 |
-
|GPQA (0-shot) | 6.94|
|
190 |
-
|MuSR (0-shot) |10.45|
|
191 |
-
|MMLU-PRO (5-shot) |50.22|
|
192 |
-
|
|
|
122 |
|
123 |
It might be good for creative writing tasks. I recommend a context length of 32k but you can go up to 131,072 tokens in theory.
|
124 |
|
125 |
+
## 🏆 Evaluation
|
126 |
+
|
127 |
+
| Metric |BigQwen2.5-Echo-47B-Instruct|**BigQwen2.5-52B-Instruct**|Qwen2.5-32B-Instruct|
|
128 |
+
|-------------------|----:|----:|----:|
|
129 |
+
|Avg. |30.31|37.42|36.17|
|
130 |
+
|IFEval (0-Shot) |73.57|79.29|83.46|
|
131 |
+
|BBH (3-Shot) |44.52|59.81|56.49|
|
132 |
+
|MATH Lvl 5 (4-Shot)| 3.47|17.82|0|
|
133 |
+
|GPQA (0-shot) | 8.61| 6.94|11.74|
|
134 |
+
|MuSR (0-shot) |10.19|10.45|13.5|
|
135 |
+
|MMLU-PRO (5-shot) |41.49|50.22|51.85|
|
136 |
+
|
137 |
## 🧩 Configuration
|
138 |
|
139 |
The following YAML configuration was used to produce this model:
|
|
|
188 |
|
189 |
outputs = pipeline(prompt, max_new_tokens=256, do_sample=True, temperature=0.7, top_k=50, top_p=0.95)
|
190 |
print(outputs[0]["generated_text"])
|
191 |
+
```
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|