Update README.md
Browse files
README.md
CHANGED
|
@@ -3,4 +3,10 @@ license: apache-2.0
|
|
| 3 |
language:
|
| 4 |
- ru
|
| 5 |
pipeline_tag: text-generation
|
| 6 |
-
---
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 3 |
language:
|
| 4 |
- ru
|
| 5 |
pipeline_tag: text-generation
|
| 6 |
+
---
|
| 7 |
+
|
| 8 |
+
# BulgakovLM 3B
|
| 9 |
+
|
| 10 |
+
A language model trained on Russian. May be suitable for further tuning. The 100 gigabyte dataset consisted primarily of web pages, books, poems, and prose. The model was trained over 2 epochs.
|
| 11 |
+
|
| 12 |
+
Uses GPT-J architecture with a context window of 4k tokens.
|