readme: minor fixes
Browse files
README.md
CHANGED
@@ -6,7 +6,7 @@ license: mit
|
|
6 |
---
|
7 |
|
8 |
# German GPT-2 model
|
9 |
-
In this repository we release (yet another) GPT-2 model, that was trained on ~90 GB from the ["German colossal, clean Common Crawl corpus"
|
10 |
|
11 |
The model is meant to be an entry point for fine-tuning on other texts, and it is definitely not as good or "dangerous" as the English GPT-3 model. We do not plan extensive PR or staged releases for this model 😉
|
12 |
|
@@ -85,8 +85,8 @@ This results in a total training corpus size of 90GB.
|
|
85 |
|
86 |
# Training Details
|
87 |
|
88 |
-
We use the recently re-trained `dbmdz/german-gpt2` (version 2!)
|
89 |
-
Thus, the tokenizer and vocab is the same as used in the `dbmdz/german-gpt2` model.
|
90 |
|
91 |
The model was trained on a v3-8 TPU, with the following parameters:
|
92 |
|
|
|
6 |
---
|
7 |
|
8 |
# German GPT-2 model
|
9 |
+
In this repository we release (yet another) GPT-2 model, that was trained on ~90 GB from the ["German colossal, clean Common Crawl corpus"](https://german-nlp-group.github.io/projects/gc4-corpus.html) (GC4).
|
10 |
|
11 |
The model is meant to be an entry point for fine-tuning on other texts, and it is definitely not as good or "dangerous" as the English GPT-3 model. We do not plan extensive PR or staged releases for this model 😉
|
12 |
|
|
|
85 |
|
86 |
# Training Details
|
87 |
|
88 |
+
We use the recently re-trained `dbmdz/german-gpt2` ([version 2](https://huggingface.co/dbmdz/german-gpt2)!)
|
89 |
+
model as back-bone model. Thus, the tokenizer and vocab is the same as used in the `dbmdz/german-gpt2` model.
|
90 |
|
91 |
The model was trained on a v3-8 TPU, with the following parameters:
|
92 |
|