Regarding Source Code For Tokenizer
#11
by
mattm1005
- opened
Hey there,
Any chance the source code with the pretrained weights for the tokenizer.model file will be made public?
mattm1005
changed discussion status to
closed
It looks like the tokenzier.model is not needed when running Offline with tokenizer.json. Please ignore my question.