Update README.md
Browse files
README.md
CHANGED
@@ -34,15 +34,20 @@ inference: true
|
|
34 |
<p><em style="color: black; font-weight: bold;">This repo contains the 7M version.</em></p>
|
35 |
</center>
|
36 |
|
37 |
-
###
|
38 |
-
Sasando-1 is a tiny, highly experimental text generator built using the Phi-3 architecture. It comes with two variations of microscopic sizes: 7M and 25M parameters. It is trained on a tightly-controlled Indo4B dataset filtered to only have 18000 unique words. The method is inspired by Microsoft's TinyStories paper which demonstrates that a tiny language model can produce fluent text when trained on tightly-controlled dataset.
|
39 |
|
40 |
-
### Specs
|
41 |
- Comes with 7M and 25M parameters
|
42 |
- Based on Phi-3 architecture
|
43 |
- Embedding vocab 4096
|
44 |
- Trained on ~257M tokens * 4 epoch
|
45 |
|
|
|
|
|
|
|
|
|
|
|
46 |
### Acknowledgments
|
47 |
|
48 |
- **Developed by:** Afrizal Hasbi Azizy
|
|
|
34 |
<p><em style="color: black; font-weight: bold;">This repo contains the 7M version.</em></p>
|
35 |
</center>
|
36 |
|
37 |
+
### 🎻 Welcome!
|
38 |
+
Sasando-1 is a tiny, highly experimental Indonesian text generator built using the Phi-3 architecture. It comes with two variations of microscopic sizes: 7M and 25M parameters. It is trained on a tightly-controlled Indo4B dataset filtered to only have 18000 unique words. The method is inspired by Microsoft's TinyStories paper which demonstrates that a tiny language model can produce fluent text when trained on tightly-controlled dataset.
|
39 |
|
40 |
+
### ✨ Specs
|
41 |
- Comes with 7M and 25M parameters
|
42 |
- Based on Phi-3 architecture
|
43 |
- Embedding vocab 4096
|
44 |
- Trained on ~257M tokens * 4 epoch
|
45 |
|
46 |
+
### 🔭 Out-of-Scope Use
|
47 |
+
This is a research preview base model. It is not intruction-tuned and has minimal safety curation. It is not intended for commercial or practical applications.
|
48 |
+
|
49 |
+
You are also not allowed to use this model without having fun.
|
50 |
+
|
51 |
### Acknowledgments
|
52 |
|
53 |
- **Developed by:** Afrizal Hasbi Azizy
|