Remove `added_tokens` json file

#44
by LucileSaulnier - opened
Mistral AI_ org
No description provided.
LucileSaulnier changed pull request status to open

LGTM! As discussed offline I don't think this should be kept. For transformers 4.34 it will be ignored, for earlier versions it will break. For the next patch it will no longer be saved.

LucileSaulnier changed pull request status to merged

Sign up or log in to comment