Update README.md
Browse files
README.md
CHANGED
|
@@ -2,3 +2,5 @@
|
|
| 2 |
This model is based on [nicoladecao/msmarco-word2vec256000-distilbert-base-uncased](https://huggingface.co/nicoladecao/msmarco-word2vec256000-distilbert-base-uncased) with a 256k sized vocabulary initialized with word2vec.
|
| 3 |
|
| 4 |
This model has been trained with MLM on the MS MARCO corpus collection for 210k steps. See train_mlm.py for the train script. It was run on 2x V100 GPUs.
|
|
|
|
|
|
|
|
|
| 2 |
This model is based on [nicoladecao/msmarco-word2vec256000-distilbert-base-uncased](https://huggingface.co/nicoladecao/msmarco-word2vec256000-distilbert-base-uncased) with a 256k sized vocabulary initialized with word2vec.
|
| 3 |
|
| 4 |
This model has been trained with MLM on the MS MARCO corpus collection for 210k steps. See train_mlm.py for the train script. It was run on 2x V100 GPUs.
|
| 5 |
+
|
| 6 |
+
**Note: Token embeddings where updated!**
|