ChemGPT-1.2B / tokenizer_config.json
ncfrey's picture
add tokenizer
0164ca1
raw
history blame contribute delete
260 Bytes
{"special_tokens_map_file": "/data1/groups/ai4bio_shared/molecules/tokenizers/pubchem10M_tokenizer/special_tokens_map.json", "name_or_path": "/data1/groups/ai4bio_shared/molecules/tokenizers/pubchem10M_tokenizer/", "tokenizer_class": "PreTrainedTokenizerFast"}