Adds the tokenizer configuration file

#7
by lysandre HF staff - opened

The tokenizer configuration file is missing and therefore leading to unforeseen errors after the migration of the canonical models.

Refer to the following issue for moreinformation: transformers#29050

lysandre changed pull request status to open
julien-c changed pull request status to merged

merging with offline approval from Louis Martin

ALMAnaCH (Inria) org

I did merge https://huggingface.co/almanach/camembert-base/discussions/8 which i think was a duplicate request?

indeed, thanks for merging #8 @wissamantoun !

Sign up or log in to comment