victormiller commited on
Commit
99e0342
1 Parent(s): 4d89b34

Update README.md

Browse files
Files changed (1) hide show
  1. README.md +21 -21
README.md CHANGED
@@ -17,31 +17,31 @@ We utilized the following datasets:
17
  | | K2-Chat-060124 | K2-Chat |
18
  |-------------------------|---------|----------|
19
  | **Natural Language Benchmarks** | | |
20
- | MMLU (0-shot) | 63.5 | 69.14 |
21
- | RACE (0-shot) | 46.1 | 46.60 |
22
- | HellaSwag (10-shot) | 81.7 | 80.80 |
23
- | PIQA (5-shot) | 82.3 | 81.34 |
24
- | ARC-easy (5-shot) | 84.6 | 79.00 |
25
- | ARC-challenge (25-shot) | 61.3 | 61.09 |
26
- | OpenBookQA (5-shot) | 48.0 | 47.00 |
27
- | Winogrande (5-shot) | 79.5 | 78.30 |
28
- | TruthfulQA (0-shot) | 44.7 | 57.32 |
29
- | CrowS-Pairs (0-shot) | 64.2 | 65.32 |
30
- | GSM8K (5-shot) | 60.7 | 77.10 |
31
- | MathQA (5-shot) | 44.8 | 43.12 |
32
- | LogiQA2.0 (0-shot) | 38.0 | 36.83 |
33
- | BBH CoT (0-shot) | 64.9 | 70.37 |
34
  | **Code Benchmarks** | | |
35
- | HumanEval (pass@1) | 47.9 | 71.20 |
36
  | **Domain Specific (Medical)** | | |
37
- | MedQA (0-shot) | 53.6 | 52.87 |
38
- | MedMCQA (5-shot) | 51.3 | 50.71 |
39
- | PubMedQA (0-shot) | 75.0 | 71.20 |
40
  | **Other** | | |
41
- | MT-Bench | 6.87 | 7.55 |
42
- | JSON-Mode-Eval | 77.21 | 90.09 |
43
  | **Overall Average Score**| | |
44
- | Avg Score | 58.88 | 61.30 |
45
 
46
 
47
  # Function Calling
 
17
  | | K2-Chat-060124 | K2-Chat |
18
  |-------------------------|---------|----------|
19
  | **Natural Language Benchmarks** | | |
20
+ | MMLU (0-shot) | 63.5 | **69.14** |
21
+ | RACE (0-shot) | 46.1 | **46.60** |
22
+ | HellaSwag (10-shot) | **81.7** | 80.80 |
23
+ | PIQA (5-shot) | **82.3** | 81.34 |
24
+ | ARC-easy (5-shot) | **84.6** | 79.00 |
25
+ | ARC-challenge (25-shot) | **61.3** | 61.09 |
26
+ | OpenBookQA (5-shot) | **48.0** | 47.00 |
27
+ | Winogrande (5-shot) | **79.5** | 78.30 |
28
+ | TruthfulQA (0-shot) | 44.7 | **57.32** |
29
+ | CrowS-Pairs (0-shot) | 64.2 | **65.32** |
30
+ | GSM8K (5-shot) | 60.7 | **77.10** |
31
+ | MathQA (5-shot) | **44.8** | 43.12 |
32
+ | LogiQA2.0 (0-shot) | **38.0** | 36.83 |
33
+ | BBH CoT (0-shot) | 64.9 | **70.37** |
34
  | **Code Benchmarks** | | |
35
+ | HumanEval (pass@1) | 47.9 | **71.20** |
36
  | **Domain Specific (Medical)** | | |
37
+ | MedQA (0-shot) | **53.6** | 52.87 |
38
+ | MedMCQA (5-shot) | **51.3** | 50.71 |
39
+ | PubMedQA (0-shot) | **75.0** | 71.20 |
40
  | **Other** | | |
41
+ | MT-Bench | 6.87 | **7.55** |
42
+ | JSON-Mode-Eval | 77.21 | **90.09** |
43
  | **Overall Average Score**| | |
44
+ | Avg Score | 58.88 | **61.30** |
45
 
46
 
47
  # Function Calling