victormiller
commited on
Commit
•
99e0342
1
Parent(s):
4d89b34
Update README.md
Browse files
README.md
CHANGED
@@ -17,31 +17,31 @@ We utilized the following datasets:
|
|
17 |
| | K2-Chat-060124 | K2-Chat |
|
18 |
|-------------------------|---------|----------|
|
19 |
| **Natural Language Benchmarks** | | |
|
20 |
-
| MMLU (0-shot) | 63.5 | 69.14 |
|
21 |
-
| RACE (0-shot) | 46.1 | 46.60 |
|
22 |
-
| HellaSwag (10-shot) | 81.7 | 80.80 |
|
23 |
-
| PIQA (5-shot) | 82.3 | 81.34 |
|
24 |
-
| ARC-easy (5-shot) | 84.6 | 79.00 |
|
25 |
-
| ARC-challenge (25-shot) | 61.3 | 61.09 |
|
26 |
-
| OpenBookQA (5-shot) | 48.0 | 47.00 |
|
27 |
-
| Winogrande (5-shot) | 79.5 | 78.30 |
|
28 |
-
| TruthfulQA (0-shot) | 44.7 | 57.32 |
|
29 |
-
| CrowS-Pairs (0-shot) | 64.2 | 65.32 |
|
30 |
-
| GSM8K (5-shot) | 60.7 | 77.10 |
|
31 |
-
| MathQA (5-shot) | 44.8 | 43.12 |
|
32 |
-
| LogiQA2.0 (0-shot) | 38.0 | 36.83 |
|
33 |
-
| BBH CoT (0-shot) | 64.9 | 70.37 |
|
34 |
| **Code Benchmarks** | | |
|
35 |
-
| HumanEval (pass@1) | 47.9 | 71.20 |
|
36 |
| **Domain Specific (Medical)** | | |
|
37 |
-
| MedQA (0-shot) | 53.6 | 52.87 |
|
38 |
-
| MedMCQA (5-shot) | 51.3 | 50.71 |
|
39 |
-
| PubMedQA (0-shot) | 75.0 | 71.20 |
|
40 |
| **Other** | | |
|
41 |
-
| MT-Bench | 6.87 | 7.55 |
|
42 |
-
| JSON-Mode-Eval | 77.21 | 90.09 |
|
43 |
| **Overall Average Score**| | |
|
44 |
-
| Avg Score | 58.88 | 61.30 |
|
45 |
|
46 |
|
47 |
# Function Calling
|
|
|
17 |
| | K2-Chat-060124 | K2-Chat |
|
18 |
|-------------------------|---------|----------|
|
19 |
| **Natural Language Benchmarks** | | |
|
20 |
+
| MMLU (0-shot) | 63.5 | **69.14** |
|
21 |
+
| RACE (0-shot) | 46.1 | **46.60** |
|
22 |
+
| HellaSwag (10-shot) | **81.7** | 80.80 |
|
23 |
+
| PIQA (5-shot) | **82.3** | 81.34 |
|
24 |
+
| ARC-easy (5-shot) | **84.6** | 79.00 |
|
25 |
+
| ARC-challenge (25-shot) | **61.3** | 61.09 |
|
26 |
+
| OpenBookQA (5-shot) | **48.0** | 47.00 |
|
27 |
+
| Winogrande (5-shot) | **79.5** | 78.30 |
|
28 |
+
| TruthfulQA (0-shot) | 44.7 | **57.32** |
|
29 |
+
| CrowS-Pairs (0-shot) | 64.2 | **65.32** |
|
30 |
+
| GSM8K (5-shot) | 60.7 | **77.10** |
|
31 |
+
| MathQA (5-shot) | **44.8** | 43.12 |
|
32 |
+
| LogiQA2.0 (0-shot) | **38.0** | 36.83 |
|
33 |
+
| BBH CoT (0-shot) | 64.9 | **70.37** |
|
34 |
| **Code Benchmarks** | | |
|
35 |
+
| HumanEval (pass@1) | 47.9 | **71.20** |
|
36 |
| **Domain Specific (Medical)** | | |
|
37 |
+
| MedQA (0-shot) | **53.6** | 52.87 |
|
38 |
+
| MedMCQA (5-shot) | **51.3** | 50.71 |
|
39 |
+
| PubMedQA (0-shot) | **75.0** | 71.20 |
|
40 |
| **Other** | | |
|
41 |
+
| MT-Bench | 6.87 | **7.55** |
|
42 |
+
| JSON-Mode-Eval | 77.21 | **90.09** |
|
43 |
| **Overall Average Score**| | |
|
44 |
+
| Avg Score | 58.88 | **61.30** |
|
45 |
|
46 |
|
47 |
# Function Calling
|