Spaces:
Sleeping
Sleeping
prasanna kumar
added llama3 models for tokenize and show as well as pass token ids will give back text
5dca0b0
| { | |
| "bos_token": "<|begin_of_text|>", | |
| "eos_token": "<|end_of_text|>" | |
| } | |