Update tokenizer_config.json

#2
by stevenaaa1207 - opened

same as the tokenizer config from hf-internal-testing of llama-1

source : https://huggingface.co/hf-internal-testing/llama-tokenizer/blob/main/tokenizer_config.json

Ready to merge
This branch is ready to get merged automatically.

Sign up or log in to comment