kazzand commited on
Commit
a54ae14
1 Parent(s): 02c87f7

Update README.md

Browse files
Files changed (1) hide show
  1. README.md +8 -3
README.md CHANGED
@@ -3,9 +3,14 @@ language:
3
  - ru
4
  ---
5
 
6
- This is a base version of Russian Longformer model created from [blinoff/roberta-base-russian-v0](https://huggingface.co/blinoff/roberta-base-russian-v0) weights with the length of context expanded to 4096 tokens.
7
- The model was fine-tuned on russian books dataset but also supports English as its source model.
8
- For a more comprehensive overview, please refer to this Habr post, which is available in Russian.
 
 
 
 
 
9
 
10
  The model can be used as-is to produce text embeddings or it can be further fine-tuned for a specific downstream task.
11
 
 
3
  - ru
4
  ---
5
 
6
+ This is a base Longformer model designed for Russian language.
7
+ It was initialized from [blinoff/roberta-base-russian-v0](https://huggingface.co/blinoff/roberta-base-russian-v0) weights and has been modified to support a context length of up to 4096 tokens.
8
+ We fine-tuned it on a dataset of Russian books. For a detailed information check out our post on Habr.
9
+
10
+ Model attributes:
11
+ * 12 attention heads
12
+ * 12 hidden layers
13
+ * 4096 tokens length of context
14
 
15
  The model can be used as-is to produce text embeddings or it can be further fine-tuned for a specific downstream task.
16