transcriptome-gpt-1024-8-16-128 / tokenizer_config.json
lixc
initial commit
494522f
raw
history blame contribute delete
158 Bytes
{"model_max_length": 1024, "pad_token": "<pad>", "eos_token": "</s>", "bos_token": "</b>", "unk_token": "<unk>", "tokenizer_class": "PreTrainedTokenizerFast"}