

TITLE
Error in mBART50 fast tokenizer `convert_tokens_to_string` method

COMMENTS
2

REACTIONS
+1: 0
-1: 0
laugh: 0
hooray: 0
heart: 0
rocket: 0
eyes: 0

BODY
### System Info

```shell
- `transformers` version: 4.18.0
- Platform: Linux-5.13.0-40-generic-x86_64-with-glibc2.29
- Python version: 3.8.10
- Huggingface_hub version: 0.5.1
- PyTorch version (GPU?): 1.11.0+cu113 (True)
- Tensorflow version (GPU?): not installed (NA)
- Flax version (CPU?/GPU?/TPU?): not installed (NA)
- Jax version: not installed
- JaxLib version: not installed
- Using GPU in script?: Yes
- Using distributed or parallel set-up in script?: No
```


### Who can help?

@patil-suraj @SaulLu 

### Information

- [ ] The official example scripts
- [X] My own modified scripts

### Tasks

- [ ] An officially supported task in the `examples` folder (such as GLUE/SQuAD, ...)
- [X] My own task or dataset (give details below)

### Reproduction

```python
>>> from transformers import AutoTokenizer

# Problematic behavior with default fast tokenizer
>>> t = AutoTokenizer.from_pretrained("facebook/mbart-large-50-one-to-many-mmt", src_lang="en_XX")

>>> t.convert_tokens_to_string(["This", " is", " a", " test"])
['This', ' is', ' a', ' test'] # Conversion didn't work

>>> t.convert_tokens_to_string([["This", " is", " a", " test"], ["Other", " test", "!"]])
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/home/gsarti/.cache/pypoetry/virtualenvs/inseq-PzwjmCYf-py3.8/lib/python3.8/site-packages/transformers/tokenization_utils_fast.py", line 534, in convert_tokens_to_string
    return self.backend_tokenizer.decoder.decode(tokens)
TypeError: Can't convert ['This', 'is', 'a', 'test'] to PyString


# Normal behavior with slow tokenizer
>>> t = AutoTokenizer.from_pretrained("facebook/mbart-large-50-one-to-many-mmt", , src_lang="en_XX", use_fast=False)

>>> t.convert_tokens_to_string(["This", " is", " a", " test"])
"This is a test" # Correctly converted

>>> t.convert_tokens_to_string([["This", " is", " a", " test"], ["Other", " test", "!"]])
["This is a test", "Other test!"] # Correctly converted
```

### Expected behavior

I would expect the behavior of the fast tokenizer to be consistent with the one of the original slow tokenizer. At the moment, the `convert_tokens_to_string` method doesn't achieve the expected results.

I am not facing the same issues when using Marian models, but it is likely that the problem is not restricted only to the `mBART50` class.