ChemGPT-19M / tokenizer_config.json
ncfrey's picture
add tokenizer
37fbf2e
raw
history blame
260 Bytes
{"special_tokens_map_file": "/data1/groups/ai4bio_shared/molecules/tokenizers/pubchem10M_tokenizer/special_tokens_map.json", "name_or_path": "/data1/groups/ai4bio_shared/molecules/tokenizers/pubchem10M_tokenizer/", "tokenizer_class": "PreTrainedTokenizerFast"}