Commit
•
8d514e8
1
Parent(s):
bdea023
Update README.md
Browse files
README.md
CHANGED
|
@@ -2,6 +2,7 @@
|
|
| 2 |
license: llama2
|
| 3 |
datasets:
|
| 4 |
- pkupie/mc2_corpus
|
|
|
|
| 5 |
language:
|
| 6 |
- en
|
| 7 |
- bo
|
|
@@ -18,4 +19,4 @@ We use the **Tibetan texts** in MC^2 and **English texts** in RedPajama with a p
|
|
| 18 |
* batch size: 1M (2K*512)
|
| 19 |
* lr scheduler: cosine
|
| 20 |
* min lr: 1e-6
|
| 21 |
-
* lr decay iters: 10240
|
|
|
|
| 2 |
license: llama2
|
| 3 |
datasets:
|
| 4 |
- pkupie/mc2_corpus
|
| 5 |
+
- togethercomputer/RedPajama-Data-1T
|
| 6 |
language:
|
| 7 |
- en
|
| 8 |
- bo
|
|
|
|
| 19 |
* batch size: 1M (2K*512)
|
| 20 |
* lr scheduler: cosine
|
| 21 |
* min lr: 1e-6
|
| 22 |
+
* lr decay iters: 10240
|