paraphrastic_test / tokenizer_config.json
jwieting's picture
test
5960170
raw
history blame
177 Bytes
{
"eos_token": "</s>",
"name_or_path": "./transformer_tokenizer",
"special_tokens_map_file": "./transformer_tokenizer/special_tokens_map.json",
"unk_token": "UUUNKKK"
}