Hugging Face's logo

Darkknight535
/
OpenCrystal-12B-L3

Text Generation
Transformers
Safetensors
llama
Not-For-All-Audiences
conversational
Eval Results
text-generation-inference
Inference Endpoints
Model card Files Community
2
Darkknight535 leaderboard-pr-bot commited on
Commit
7a4993e
•
1 Parent(s): 4bbee9c

Adding Evaluation Results (#2)

Browse files

- Adding Evaluation Results (e06452a57366f92afe377bd47905d421f537e84d)


Co-authored-by: Open LLM Leaderboard PR Bot <leaderboard-pr-bot@users.noreply.huggingface.co>

Files changed (1) hide show
  1. README.md +109 -1
README.md CHANGED
@@ -2,6 +2,101 @@
2
  library_name: transformers
3
  tags:
4
  - not-for-all-audiences
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
5
  ---
6
  ### OpenCrystal-12B-L3
7
  This is a finetuned language model. (I recommend using this one v2 and v2.1 are not good enough)
@@ -57,4 +152,17 @@ You're {{char}}, follow {{char}} personality and plot of the story, Don't impers
57
  ```
58
 
59
  ### FeedBack
60
- [FeedBack here](https://huggingface.co/Darkknight535/OpenCrystal-12B-L3/discussions/1)
 
 
 
 
 
 
 
 
 
 
 
 
 
 
2
  library_name: transformers
3
  tags:
4
  - not-for-all-audiences
5
+ model-index:
6
+ - name: OpenCrystal-12B-L3
7
+ results:
8
+ - task:
9
+ type: text-generation
10
+ name: Text Generation
11
+ dataset:
12
+ name: IFEval (0-Shot)
13
+ type: HuggingFaceH4/ifeval
14
+ args:
15
+ num_few_shot: 0
16
+ metrics:
17
+ - type: inst_level_strict_acc and prompt_level_strict_acc
18
+ value: 40.71
19
+ name: strict accuracy
20
+ source:
21
+ url: https://huggingface.co/spaces/open-llm-leaderboard/open_llm_leaderboard?query=Darkknight535/OpenCrystal-12B-L3
22
+ name: Open LLM Leaderboard
23
+ - task:
24
+ type: text-generation
25
+ name: Text Generation
26
+ dataset:
27
+ name: BBH (3-Shot)
28
+ type: BBH
29
+ args:
30
+ num_few_shot: 3
31
+ metrics:
32
+ - type: acc_norm
33
+ value: 31.84
34
+ name: normalized accuracy
35
+ source:
36
+ url: https://huggingface.co/spaces/open-llm-leaderboard/open_llm_leaderboard?query=Darkknight535/OpenCrystal-12B-L3
37
+ name: Open LLM Leaderboard
38
+ - task:
39
+ type: text-generation
40
+ name: Text Generation
41
+ dataset:
42
+ name: MATH Lvl 5 (4-Shot)
43
+ type: hendrycks/competition_math
44
+ args:
45
+ num_few_shot: 4
46
+ metrics:
47
+ - type: exact_match
48
+ value: 7.93
49
+ name: exact match
50
+ source:
51
+ url: https://huggingface.co/spaces/open-llm-leaderboard/open_llm_leaderboard?query=Darkknight535/OpenCrystal-12B-L3
52
+ name: Open LLM Leaderboard
53
+ - task:
54
+ type: text-generation
55
+ name: Text Generation
56
+ dataset:
57
+ name: GPQA (0-shot)
58
+ type: Idavidrein/gpqa
59
+ args:
60
+ num_few_shot: 0
61
+ metrics:
62
+ - type: acc_norm
63
+ value: 7.49
64
+ name: acc_norm
65
+ source:
66
+ url: https://huggingface.co/spaces/open-llm-leaderboard/open_llm_leaderboard?query=Darkknight535/OpenCrystal-12B-L3
67
+ name: Open LLM Leaderboard
68
+ - task:
69
+ type: text-generation
70
+ name: Text Generation
71
+ dataset:
72
+ name: MuSR (0-shot)
73
+ type: TAUR-Lab/MuSR
74
+ args:
75
+ num_few_shot: 0
76
+ metrics:
77
+ - type: acc_norm
78
+ value: 5.74
79
+ name: acc_norm
80
+ source:
81
+ url: https://huggingface.co/spaces/open-llm-leaderboard/open_llm_leaderboard?query=Darkknight535/OpenCrystal-12B-L3
82
+ name: Open LLM Leaderboard
83
+ - task:
84
+ type: text-generation
85
+ name: Text Generation
86
+ dataset:
87
+ name: MMLU-PRO (5-shot)
88
+ type: TIGER-Lab/MMLU-Pro
89
+ config: main
90
+ split: test
91
+ args:
92
+ num_few_shot: 5
93
+ metrics:
94
+ - type: acc
95
+ value: 29.34
96
+ name: accuracy
97
+ source:
98
+ url: https://huggingface.co/spaces/open-llm-leaderboard/open_llm_leaderboard?query=Darkknight535/OpenCrystal-12B-L3
99
+ name: Open LLM Leaderboard
100
  ---
101
  ### OpenCrystal-12B-L3
102
  This is a finetuned language model. (I recommend using this one v2 and v2.1 are not good enough)
 
152
  ```
153
 
154
  ### FeedBack
155
+ [FeedBack here](https://huggingface.co/Darkknight535/OpenCrystal-12B-L3/discussions/1)
156
+ # [Open LLM Leaderboard Evaluation Results](https://huggingface.co/spaces/open-llm-leaderboard/open_llm_leaderboard)
157
+ Detailed results can be found [here](https://huggingface.co/datasets/open-llm-leaderboard/details_Darkknight535__OpenCrystal-12B-L3)
158
+
159
+ | Metric |Value|
160
+ |-------------------|----:|
161
+ |Avg. |20.51|
162
+ |IFEval (0-Shot) |40.71|
163
+ |BBH (3-Shot) |31.84|
164
+ |MATH Lvl 5 (4-Shot)| 7.93|
165
+ |GPQA (0-shot) | 7.49|
166
+ |MuSR (0-shot) | 5.74|
167
+ |MMLU-PRO (5-shot) |29.34|
168
+