mlabonne commited on
Commit
9eb644e
1 Parent(s): 580248d

Update README.md

Browse files
Files changed (1) hide show
  1. README.md +13 -14
README.md CHANGED
@@ -122,6 +122,18 @@ I made it due to popular demand but I haven't tested it so use it at your own ri
122
 
123
  It might be good for creative writing tasks. I recommend a context length of 32k but you can go up to 131,072 tokens in theory.
124
 
 
 
 
 
 
 
 
 
 
 
 
 
125
  ## 🧩 Configuration
126
 
127
  The following YAML configuration was used to produce this model:
@@ -176,17 +188,4 @@ pipeline = transformers.pipeline(
176
 
177
  outputs = pipeline(prompt, max_new_tokens=256, do_sample=True, temperature=0.7, top_k=50, top_p=0.95)
178
  print(outputs[0]["generated_text"])
179
- ```
180
- # [Open LLM Leaderboard Evaluation Results](https://huggingface.co/spaces/open-llm-leaderboard/open_llm_leaderboard)
181
- Detailed results can be found [here](https://huggingface.co/datasets/open-llm-leaderboard/details_mlabonne__BigQwen2.5-52B-Instruct)
182
-
183
- | Metric |Value|
184
- |-------------------|----:|
185
- |Avg. |37.42|
186
- |IFEval (0-Shot) |79.29|
187
- |BBH (3-Shot) |59.81|
188
- |MATH Lvl 5 (4-Shot)|17.82|
189
- |GPQA (0-shot) | 6.94|
190
- |MuSR (0-shot) |10.45|
191
- |MMLU-PRO (5-shot) |50.22|
192
-
 
122
 
123
  It might be good for creative writing tasks. I recommend a context length of 32k but you can go up to 131,072 tokens in theory.
124
 
125
+ ## 🏆 Evaluation
126
+
127
+ | Metric |BigQwen2.5-Echo-47B-Instruct|**BigQwen2.5-52B-Instruct**|Qwen2.5-32B-Instruct|
128
+ |-------------------|----:|----:|----:|
129
+ |Avg. |30.31|37.42|36.17|
130
+ |IFEval (0-Shot) |73.57|79.29|83.46|
131
+ |BBH (3-Shot) |44.52|59.81|56.49|
132
+ |MATH Lvl 5 (4-Shot)| 3.47|17.82|0|
133
+ |GPQA (0-shot) | 8.61| 6.94|11.74|
134
+ |MuSR (0-shot) |10.19|10.45|13.5|
135
+ |MMLU-PRO (5-shot) |41.49|50.22|51.85|
136
+
137
  ## 🧩 Configuration
138
 
139
  The following YAML configuration was used to produce this model:
 
188
 
189
  outputs = pipeline(prompt, max_new_tokens=256, do_sample=True, temperature=0.7, top_k=50, top_p=0.95)
190
  print(outputs[0]["generated_text"])
191
+ ```