Spaces:

vpkprasanna
/

TokenizerViz

Sleeping

App Files Files Community

TokenizerViz / models /Meta-Llama-3.1-8B /original

4 contributors

History: 2 commits

prasanna kumar

added GIT LFS support

eb89467 4 months ago

params.json

199 Bytes
LFS

added GIT LFS support 4 months ago
tokenizer.model

2.18 MB

added llama3 models for tokenize and show as well as pass token ids will give back text 4 months ago