it seems that the model repo does not contain a tokenizer which makes it hard to use in common inference code
· Sign up or log in to comment
