yinsong1986
commited on
Commit
•
2348608
1
Parent(s):
f4c8132
Update README.md
Browse files
README.md
CHANGED
@@ -85,6 +85,20 @@ there were some limitations on its performance on longer context. Motivated by i
|
|
85 |
- **Contact:** [GitHub issues](https://github.com/awslabs/extending-the-context-length-of-open-source-llms/issues)
|
86 |
- **Inference Code** [Github Repo](https://github.com/awslabs/extending-the-context-length-of-open-source-llms/blob/main/MistralLite/)
|
87 |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
88 |
## How to Use MistralLite from Python Code (HuggingFace transformers) ##
|
89 |
|
90 |
**Important** - For an end-to-end example Jupyter notebook, please refer to [this link](https://github.com/awslabs/extending-the-context-length-of-open-source-llms/blob/main/MistralLite/huggingface-transformers/example_usage.ipynb).
|
|
|
85 |
- **Contact:** [GitHub issues](https://github.com/awslabs/extending-the-context-length-of-open-source-llms/issues)
|
86 |
- **Inference Code** [Github Repo](https://github.com/awslabs/extending-the-context-length-of-open-source-llms/blob/main/MistralLite/)
|
87 |
|
88 |
+
## MistralLite LM-Eval Results
|
89 |
+
|
90 |
+
### Methodology
|
91 |
+
|
92 |
+
- Please see https://huggingface.co/spaces/HuggingFaceH4/open_llm_leaderboard
|
93 |
+
- revision=4ececff
|
94 |
+
- Note: we used --model hf-causal-experimental instead of --model hf-causal
|
95 |
+
|
96 |
+
### Results
|
97 |
+
|
98 |
+
|Average|hellaswag| arc_challenge|truthful_qa (mc2)| MMLU (acc)|
|
99 |
+
|----------|-------------:|------------:|-----------:|-----------:|
|
100 |
+
| 0.57221 | 0.81617 | 0.58874 | 0.38275 | 0.5012 |
|
101 |
+
|
102 |
## How to Use MistralLite from Python Code (HuggingFace transformers) ##
|
103 |
|
104 |
**Important** - For an end-to-end example Jupyter notebook, please refer to [this link](https://github.com/awslabs/extending-the-context-length-of-open-source-llms/blob/main/MistralLite/huggingface-transformers/example_usage.ipynb).
|