File size: 391 Bytes
fec2a86 e87c086 bea7332 11789e6 |
1 2 3 4 5 6 7 8 9 10 11 12 13 14 |
---
license: apache-2.0
language:
- ru
pipeline_tag: text-generation
---
# BulgakovLM 3B
A language model trained on Russian. May be suitable for further tuning. The 100 gigabyte dataset consisted primarily of web pages, books, poems, and prose. The model was trained over 2 epochs.
Uses GPT-J architecture with a context window of 4k tokens.
Trained thanks to a TRC grant on TPU-VM v3-8 |