

TITLE
Tokenization in run_mlm is not correct (maybe)?

COMMENTS
2

REACTIONS
+1: 0
-1: 0
laugh: 0
hooray: 0
heart: 0
rocket: 0
eyes: 0

BODY
### System Info

```shell
- `transformers` version: 4.17.0.dev0
- Platform: Linux-5.13.0-28-generic-x86_64-with-glibc2.17
- Python version: 3.8.12
- PyTorch version (GPU?): 1.10.2 (True)
- Tensorflow version (GPU?): not installed (NA)
- Flax version (CPU?/GPU?/TPU?): not installed (NA)
- Jax version: not installed
- JaxLib version: not installed
- Using GPU in script?: True
```


### Who can help?

@sgugger 

### Information

- [X] The official example scripts
- [ ] My own modified scripts

### Tasks

- [X] An officially supported task in the `examples` folder (such as GLUE/SQuAD, ...)
- [ ] My own task or dataset (give details below)

### Reproduction

In the `run_mlm.py` unless I am missing something which is completely possible, the two functions `tokenize_function` and `group_texts` end up tokenizing sentences in this way (for BERT for example):

The sentence "One sentence. Another sentence. One last sentence." is tokenized as 
[101, id_One, id_sentence, 102, 101, id_Another, id_sentece, 102, 101, id_One, id_last, id_sentence, 102]
instead of 
[101, id_One, id_sentence, 102, id_Another, id_sentece, 102, id_One, id_last, id_sentence, 102]

I believe this ends up being in conflict (not erroring but behaving differently) with the default tokenizer behaviour on fine tuning tasks with pairs of sentences such as mnli. Is this intended?

### Expected behavior

```shell
Maybe this is the expected tokenization
[101, id_One, id_sentence, 102, id_Another, id_sentece, 102, id_One, id_last, id_sentence, 102]?
```
