dqnguyen commited on
Commit
c4b2482
1 Parent(s): 1a0be64

Update README.md

Browse files
Files changed (1) hide show
  1. README.md +1 -1
README.md CHANGED
@@ -65,7 +65,7 @@ with torch.no_grad():
65
 
66
  Before applying BPE to the pre-training corpus of English Tweets, we tokenized these Tweets using `TweetTokenizer` from the NLTK toolkit and used the `emoji` package to translate emotion icons into text strings (here, each icon is referred to as a word token). We also normalized the Tweets by converting user mentions and web/url links into special tokens `@USER` and `HTTPURL`, respectively. Thus it is recommended to also apply the same pre-processing step for BERTweet-based downstream applications w.r.t. the raw input Tweets.
67
 
68
- For "`vinai/bertweet-large`", given the raw input Tweets, to obtain the same pre-processing output, users could employ our [TweetNormalizer](https://github.com/VinAIResearch/BERTweet/blob/master/TweetNormalizer.py) module.
69
 
70
  - Installation: `pip3 install nltk emoji`
71
 
 
65
 
66
  Before applying BPE to the pre-training corpus of English Tweets, we tokenized these Tweets using `TweetTokenizer` from the NLTK toolkit and used the `emoji` package to translate emotion icons into text strings (here, each icon is referred to as a word token). We also normalized the Tweets by converting user mentions and web/url links into special tokens `@USER` and `HTTPURL`, respectively. Thus it is recommended to also apply the same pre-processing step for BERTweet-based downstream applications w.r.t. the raw input Tweets.
67
 
68
+ For `vinai/bertweet-large`, given the raw input Tweets, to obtain the same pre-processing output, users could employ our [TweetNormalizer](https://github.com/VinAIResearch/BERTweet/blob/master/TweetNormalizer.py) module.
69
 
70
  - Installation: `pip3 install nltk emoji`
71