Edit readme model spec
Browse files
README.md
CHANGED
|
@@ -11,4 +11,7 @@ tags:
|
|
| 11 |
- Administrative documents
|
| 12 |
---
|
| 13 |
|
| 14 |
-
# AdminBERT 16GB: A French Language Model adapted to administrative documents
|
|
|
|
|
|
|
|
|
|
|
|
| 11 |
- Administrative documents
|
| 12 |
---
|
| 13 |
|
| 14 |
+
# AdminBERT 16GB: A French Language Model adapted to administrative documents
|
| 15 |
+
|
| 16 |
+
[AdminBERT-16GB](example) is a French language model adapted on a large corpus of 50 millions of French administrative text. It is a derivative of CamemBERT model, which is based on the RoBERTa architecture. AdminBERT-16GB is trained using the Masked Language Modeling (MLM) objective with 30% mask rate for 3 epochs on 24 A100 GPUs. The dataset used for training is [Adminset](https://huggingface.co/datasets/taln-ls2n/Adminset).
|
| 17 |
+
|