

TITLE
ADD BORT

COMMENTS
3

REACTIONS
+1: 1
-1: 0
laugh: 0
hooray: 0
heart: 0
rocket: 0
eyes: 0

BODY
Hi,

this is a "clean" follow-up PR to the first attempt of adding Bort to Transformers (see #9112).

As Bort is based on the BERT architecture, there's no need to define own model classes, such as `BortModel`. This is done in the main Bort configuration via:

```json
"model_type": "bert"
```

Bort uses the same vocab as RoBERTa, so the tokenizer instance is also configured in the model configuration:

```json
"tokenizer_class": "RobertaTokenizer"
```

Basic integration tests and a (hopefully verbose) conversion script are also included in this PR.