

TITLE
Strange start token in MT5 generation

COMMENTS
3

REACTIONS
+1: 0
-1: 0
laugh: 0
hooray: 0
heart: 0
rocket: 0
eyes: 0

BODY
## Environment info

- `transformers` version: 4.2.2
- Platform: Linux
- Python version: 3.6
- PyTorch version (GPU?):1.7.1

### Who can help
 Text Generation: @patrickvonplaten @TevenLeScao
 T5: @patrickvonplaten
 
## Information

Model I am using (Bert, XLNet ...): MT5

## To reproduce

Steps to reproduce the behavior:
```python
from transformers import MT5ForConditionalGeneration, MT5Tokenizer

model = MT5ForConditionalGeneration.from_pretrained("google/mt5-small")
tokenizer = MT5Tokenizer.from_pretrained("google/mt5-small")

text = 'summarize: Bidirectional Encoder Representations from Transformers is a Transformer-based machine learning technique for natural language processing pre-training developed by Google'
inputs = tokenizer([text], max_length=512, truncation=True, return_tensors='pt')
summary_ids = model.generate(inputs['input_ids'])
print([tokenizer.decode(g, skip_special_tokens=True, clean_up_tokenization_spaces=True) for g in summary_ids])
```
The output I got is ['<extra_id_0>.']
## Expected behavior
I tried a few input texts. The generated output always start with <extra_id_0>, which doesn't happen in t5 generation. Anyone knows how to solve it?
