

TITLE
XLM-R XL/XXL

COMMENTS
12

REACTIONS
+1: 1
-1: 0
laugh: 0
hooray: 0
heart: 0
rocket: 0
eyes: 0

BODY
# 🌟 New model addition

## Model description

The larger version of XLMR.

[Source](https://github.com/pytorch/fairseq/tree/master/examples/xlmr)

Model | Description | #params | vocab size | Download
---|---|---|---|---
`xlmr.xl` | XLM-R (`layers=36, model_dim=2560`) | 3.5B | 250k | [xlm.xl.tar.gz](https://dl.fbaipublicfiles.com/fairseq/models/xlmr/xlmr.xl.tar.gz)
`xlmr.xxl` | XLM-R (`layers=48, model_dim=4096`) | 10.7B | 250k | [xlm.xxl.tar.gz](https://dl.fbaipublicfiles.com/fairseq/models/xlmr/xlmr.xxl.tar.gz)



<!-- Important information -->

## Open source status

* [x] the model implementation is available: (give details) -> Already available in huggingface
* [x] the model weights are available: (give details) -> link + source provided.
* [ ] who are the authors: (mention them, if possible by @gh-username)
