

TITLE
BertTokenizer can't split the string in the form of "word+special_token"correctly.

COMMENTS
5

REACTIONS
+1: 0
-1: 0
laugh: 0
hooray: 0
heart: 0
rocket: 0
eyes: 0

BODY
## Environment info
<!-- You can run the command `transformers-cli env` and copy-and-paste its output below.
     Don't forget to fill out the missing fields in that output! -->

- `transformers` version: 4.2.1
- Platform: Linux-3.10.0-957.el7.x86_64-x86_64-with-debian-buster-sid
- Python version: 3.6.13
- PyTorch version (GPU?): 1.10.0+cu102 (True)
- Tensorflow version (GPU?): not installed (NA)
- Using GPU in script?: <yes>
- Using distributed or parallel set-up in script?: <no>

## To reproduce

Steps to reproduce the behavior: just run some tokenizing examples.

1.tokenizer = BertTokenizer.from_pretrained("models/pretrained_models/bert-base-chinese")
2.tokenizer.tokenize("↗↖")
`from transformers import BertTokenizer

if __name__ == "__main__":
    tokenizer = BertTokenizer.from_pretrained("models/pretrained_models/bert-base-chinese")
    # tokenizer.tokenize("↗↖")
    # tokenizer.tokenize("happy↖")
    tokenizer.tokenize("中↖")
`
more examples：
tokenizer.tokenize("中↖")
Out[58]: ['中', '[UNK]']
tokenizer.tokenize("a↖")
Out[59]: ['[UNK]']
tokenizer.tokenize("b↖")
Out[60]: ['[UNK]']
tokenizer.tokenize("abc↖")
Out[61]: ['[UNK]']
tokenizer.tokenize("happy↖")
Out[62]: ['[UNK]']
tokenizer.tokenize("↗↖")
Out[63]: ['[UNK]']
tokenizer.tokenize(",↖")
Out[64]: [',', '[UNK]']
tokenizer.tokenize("。↖")
Out[65]: ['。', '[UNK]']
tokenizer.tokenize("咕咕↖")
Out[66]: ['咕', '咕', '[UNK]']
tokenizer.tokenize("|↖")
Out[67]: ['|', '[UNK]']
tokenizer.tokenize("ك↖")
Out[68]: ['[UNK]']
tokenizer.tokenize("happy😚")
Out[69]: ['[UNK]']

<!-- If you have code snippets, error messages, stack traces please provide them here as well.
     Important! Use code tags to correctly format your code. See https://help.github.com/en/github/writing-on-github/creating-and-highlighting-code-blocks#syntax-highlighting
     Do not use screenshots, as they are hard to read and (more importantly) don't allow others to copy-and-paste your code.-->

## Expected behavior
tokenizer.tokenize("happy↖")-->[happy, unk]
But the tokenizer now treat "happy↖" as a whole token, so return only one unk.
And the tokenizer will add white spaces when tokenizing Chinese strings, so it can split "中↖" to [中，unk].
<!-- A clear and concise description of what you would expect to happen. -->

