Update README.md
Browse files
README.md
CHANGED
@@ -3,44 +3,35 @@
|
|
3 |
<!-- markdownlint-disable no-duplicate-header -->
|
4 |
|
5 |
<div align="center">
|
6 |
-
<img src="https://github.com/deepseek-ai/DeepSeek-V2/blob/main/figures/logo.svg" width="60%" alt="DeepSeek LLM" />
|
7 |
</div>
|
8 |
<hr>
|
9 |
<div align="center">
|
10 |
|
11 |
<a href="https://www.deepseek.com/" target="_blank">
|
12 |
-
<img alt="Homepage" src="https://github.com/deepseek-ai/DeepSeek-V2/blob/main/figures/badge.svg" />
|
13 |
</a>
|
14 |
<a href="https://chat.deepseek.com/" target="_blank">
|
15 |
-
<img alt="Chat" src="https://img.shields.io/badge/🤖%20Chat-DeepSeek%20LLM-536af5?color=536af5&logoColor=white" />
|
16 |
</a>
|
17 |
<a href="https://huggingface.co/deepseek-ai" target="_blank">
|
18 |
-
<img alt="Hugging Face" src="https://img.shields.io/badge/%F0%9F%A4%97%20Hugging%20Face-DeepSeek%20AI-ffc107?color=ffc107&logoColor=white" />
|
19 |
</a>
|
20 |
|
21 |
-
</div>
|
22 |
-
|
23 |
-
<div align="center">
|
24 |
-
|
25 |
<a href="https://discord.gg/Tc7c45Zzu5" target="_blank">
|
26 |
-
<img alt="Discord" src="https://img.shields.io/badge/Discord-DeepSeek%20AI-7289da?logo=discord&logoColor=white&color=7289da" />
|
27 |
</a>
|
28 |
<a href="https://github.com/deepseek-ai/DeepSeek-V2/blob/main/figures/qr.jpeg" target="_blank">
|
29 |
-
<img alt="Wechat" src="https://img.shields.io/badge/WeChat-DeepSeek%20AI-brightgreen?logo=wechat&logoColor=white" />
|
30 |
</a>
|
31 |
<a href="https://twitter.com/deepseek_ai" target="_blank">
|
32 |
-
<img alt="Twitter Follow" src="https://img.shields.io/badge/Twitter-deepseek_ai-white?logo=x&logoColor=white" />
|
33 |
</a>
|
34 |
-
|
35 |
-
</div>
|
36 |
-
|
37 |
-
<div align="center">
|
38 |
-
|
39 |
<a href="LICENSE-CODE">
|
40 |
-
<img alt="Code License" src="https://img.shields.io/badge/Code_License-MIT-f5de53?&color=f5de53">
|
41 |
</a>
|
42 |
<a href="LICENSE-MODEL">
|
43 |
-
<img alt="Model License" src="https://img.shields.io/badge/Model_License-Model_Agreement-f5de53?&color=f5de53">
|
44 |
</a>
|
45 |
</div>
|
46 |
|
@@ -66,8 +57,8 @@ Today, we’re introducing DeepSeek-V2, a strong Mixture-of-Experts (MoE) langua
|
|
66 |
<p align="center">
|
67 |
|
68 |
<div style="display: flex; justify-content: center;">
|
69 |
-
<img src="https://github.com/deepseek-ai/DeepSeek-V2/blob/main/figures/activationparameters.png" style="height:300px; width:auto; margin-right:10px">
|
70 |
-
<img src="https://github.com/deepseek-ai/DeepSeek-V2/blob/main/figures/trainingcost.png" style="height:300px; width:auto; margin-left:10px">
|
71 |
</div>
|
72 |
</p>
|
73 |
We pretrained DeepSeek-V2 on a diverse and high-quality corpus comprising 8.1 trillion tokens. This comprehensive pretraining was followed by a process of Supervised Fine-Tuning (SFT) and Reinforcement Learning (RL) to fully unleash the model's capabilities. The evaluation results validate the effectiveness of our approach as DeepSeek-V2 achieves remarkable performance on both standard benchmarks and open-ended generation evaluation.
|
@@ -107,7 +98,7 @@ For more evaluation details, such as few-shot settings and prompts, please check
|
|
107 |
|
108 |
#### Context Window
|
109 |
<p align="center">
|
110 |
-
<img width="80%" src="https://github.com/deepseek-ai/DeepSeek-V2/blob/main/figures/niah.png">
|
111 |
</p>
|
112 |
|
113 |
Evaluation results on the ``Needle In A Haystack`` (NIAH) tests. DeepSeek-V2 performs well across all context window lengths up to **128K**.
|
@@ -133,7 +124,7 @@ Evaluation results on the ``Needle In A Haystack`` (NIAH) tests. DeepSeek-V2 pe
|
|
133 |
#### English Open Ended Generation Evaluation
|
134 |
We evaluate our model on AlpacaEval 2.0 and MTBench, showing the competitive performance of DeepSeek-V2-Chat-RL on English conversation generation.
|
135 |
<p align="center">
|
136 |
-
<img width="50%" src="https://github.com/deepseek-ai/DeepSeek-V2/blob/main/figures/mtbench.png" />
|
137 |
</p>
|
138 |
|
139 |
#### Chinese Open Ended Generation Evaluation
|
@@ -160,7 +151,7 @@ We evaluate our model on AlpacaEval 2.0 and MTBench, showing the competitive per
|
|
160 |
We evaluate our model on LiveCodeBench (0901-0401), a benchmark designed for live coding challenges. As illustrated, DeepSeek-V2 demonstrates considerable proficiency in LiveCodeBench, achieving a Pass@1 score that surpasses several other sophisticated models. This performance highlights the model's effectiveness in tackling live coding tasks.
|
161 |
|
162 |
<p align="center">
|
163 |
-
<img width="50%" src="https://github.com/deepseek-ai/DeepSeek-V2/blob/main/figures/code_benchmarks.png">
|
164 |
</p>
|
165 |
|
166 |
## 4. Model Architecture
|
@@ -169,7 +160,7 @@ DeepSeek-V2 adopts innovative architectures to guarantee economical training and
|
|
169 |
- For Feed-Forward Networks (FFNs), we adopt DeepSeekMoE architecture, a high-performance MoE architecture that enables training stronger models at lower costs.
|
170 |
|
171 |
<p align="center">
|
172 |
-
<img width="90%" src="https://github.com/deepseek-ai/DeepSeek-V2/blob/main/figures/architecture.png" />
|
173 |
</p>
|
174 |
|
175 |
## 5. Chat Website
|
@@ -180,7 +171,7 @@ We also provide OpenAI-Compatible API at DeepSeek Platform: [platform.deepseek.c
|
|
180 |
|
181 |
|
182 |
<p align="center">
|
183 |
-
<img width="40%" src="https://github.com/deepseek-ai/DeepSeek-V2/blob/main/figures/model_price.png">
|
184 |
</p>
|
185 |
|
186 |
|
|
|
3 |
<!-- markdownlint-disable no-duplicate-header -->
|
4 |
|
5 |
<div align="center">
|
6 |
+
<img src="https://github.com/deepseek-ai/DeepSeek-V2/blob/main/figures/logo.svg?raw=true" width="60%" alt="DeepSeek LLM" />
|
7 |
</div>
|
8 |
<hr>
|
9 |
<div align="center">
|
10 |
|
11 |
<a href="https://www.deepseek.com/" target="_blank">
|
12 |
+
<img alt="Homepage" src="https://github.com/deepseek-ai/DeepSeek-V2/blob/main/figures/badge.svg?raw=true" style="display: inline-block; vertical-align: middle;"/>
|
13 |
</a>
|
14 |
<a href="https://chat.deepseek.com/" target="_blank">
|
15 |
+
<img alt="Chat" src="https://img.shields.io/badge/🤖%20Chat-DeepSeek%20LLM-536af5?color=536af5&logoColor=white?raw=true" style="display: inline-block; vertical-align: middle;"/>
|
16 |
</a>
|
17 |
<a href="https://huggingface.co/deepseek-ai" target="_blank">
|
18 |
+
<img alt="Hugging Face" src="https://img.shields.io/badge/%F0%9F%A4%97%20Hugging%20Face-DeepSeek%20AI-ffc107?color=ffc107&logoColor=white?raw=true" style="display: inline-block; vertical-align: middle;"/>
|
19 |
</a>
|
20 |
|
|
|
|
|
|
|
|
|
21 |
<a href="https://discord.gg/Tc7c45Zzu5" target="_blank">
|
22 |
+
<img alt="Discord" src="https://img.shields.io/badge/Discord-DeepSeek%20AI-7289da?logo=discord&logoColor=white&color=7289da?raw=true" style="display: inline-block; vertical-align: middle;"/>
|
23 |
</a>
|
24 |
<a href="https://github.com/deepseek-ai/DeepSeek-V2/blob/main/figures/qr.jpeg" target="_blank">
|
25 |
+
<img alt="Wechat" src="https://img.shields.io/badge/WeChat-DeepSeek%20AI-brightgreen?logo=wechat&logoColor=white?raw=true"style="display: inline-block; vertical-align: middle;" />
|
26 |
</a>
|
27 |
<a href="https://twitter.com/deepseek_ai" target="_blank">
|
28 |
+
<img alt="Twitter Follow" src="https://img.shields.io/badge/Twitter-deepseek_ai-white?logo=x&logoColor=white?raw=true" style="display: inline-block; vertical-align: middle;"/>
|
29 |
</a>
|
|
|
|
|
|
|
|
|
|
|
30 |
<a href="LICENSE-CODE">
|
31 |
+
<img alt="Code License" src="https://img.shields.io/badge/Code_License-MIT-f5de53?&color=f5de53?raw=true"style="display: inline-block; vertical-align: middle;">
|
32 |
</a>
|
33 |
<a href="LICENSE-MODEL">
|
34 |
+
<img alt="Model License" src="https://img.shields.io/badge/Model_License-Model_Agreement-f5de53?&color=f5de53?raw=true"style="display: inline-block; vertical-align: middle;">
|
35 |
</a>
|
36 |
</div>
|
37 |
|
|
|
57 |
<p align="center">
|
58 |
|
59 |
<div style="display: flex; justify-content: center;">
|
60 |
+
<img src="https://github.com/deepseek-ai/DeepSeek-V2/blob/main/figures/activationparameters.png?raw=true" style="height:300px; width:auto; margin-right:10px">
|
61 |
+
<img src="https://github.com/deepseek-ai/DeepSeek-V2/blob/main/figures/trainingcost.png?raw=true" style="height:300px; width:auto; margin-left:10px">
|
62 |
</div>
|
63 |
</p>
|
64 |
We pretrained DeepSeek-V2 on a diverse and high-quality corpus comprising 8.1 trillion tokens. This comprehensive pretraining was followed by a process of Supervised Fine-Tuning (SFT) and Reinforcement Learning (RL) to fully unleash the model's capabilities. The evaluation results validate the effectiveness of our approach as DeepSeek-V2 achieves remarkable performance on both standard benchmarks and open-ended generation evaluation.
|
|
|
98 |
|
99 |
#### Context Window
|
100 |
<p align="center">
|
101 |
+
<img width="80%" src="https://github.com/deepseek-ai/DeepSeek-V2/blob/main/figures/niah.png?raw=true">
|
102 |
</p>
|
103 |
|
104 |
Evaluation results on the ``Needle In A Haystack`` (NIAH) tests. DeepSeek-V2 performs well across all context window lengths up to **128K**.
|
|
|
124 |
#### English Open Ended Generation Evaluation
|
125 |
We evaluate our model on AlpacaEval 2.0 and MTBench, showing the competitive performance of DeepSeek-V2-Chat-RL on English conversation generation.
|
126 |
<p align="center">
|
127 |
+
<img width="50%" src="https://github.com/deepseek-ai/DeepSeek-V2/blob/main/figures/mtbench.png?raw=true" />
|
128 |
</p>
|
129 |
|
130 |
#### Chinese Open Ended Generation Evaluation
|
|
|
151 |
We evaluate our model on LiveCodeBench (0901-0401), a benchmark designed for live coding challenges. As illustrated, DeepSeek-V2 demonstrates considerable proficiency in LiveCodeBench, achieving a Pass@1 score that surpasses several other sophisticated models. This performance highlights the model's effectiveness in tackling live coding tasks.
|
152 |
|
153 |
<p align="center">
|
154 |
+
<img width="50%" src="https://github.com/deepseek-ai/DeepSeek-V2/blob/main/figures/code_benchmarks.png?raw=true">
|
155 |
</p>
|
156 |
|
157 |
## 4. Model Architecture
|
|
|
160 |
- For Feed-Forward Networks (FFNs), we adopt DeepSeekMoE architecture, a high-performance MoE architecture that enables training stronger models at lower costs.
|
161 |
|
162 |
<p align="center">
|
163 |
+
<img width="90%" src="https://github.com/deepseek-ai/DeepSeek-V2/blob/main/figures/architecture.png?raw=true" />
|
164 |
</p>
|
165 |
|
166 |
## 5. Chat Website
|
|
|
171 |
|
172 |
|
173 |
<p align="center">
|
174 |
+
<img width="40%" src="https://github.com/deepseek-ai/DeepSeek-V2/blob/main/figures/model_price.png?raw=true">
|
175 |
</p>
|
176 |
|
177 |
|