Upload tokenizer.json
Browse filesMade using:
```python
from transformers import AutoTokenizer
tokenizer = AutoTokenizer.from_pretrained("facebook/mbart-large-50")
assert tokenizer.is_fast
tokenizer.save_pretrained("...")
```
- .gitattributes +1 -0
- tokenizer.json +3 -0
.gitattributes
CHANGED
@@ -14,3 +14,4 @@
|
|
14 |
*.pb filter=lfs diff=lfs merge=lfs -text
|
15 |
*.pt filter=lfs diff=lfs merge=lfs -text
|
16 |
*.pth filter=lfs diff=lfs merge=lfs -text
|
|
|
|
14 |
*.pb filter=lfs diff=lfs merge=lfs -text
|
15 |
*.pt filter=lfs diff=lfs merge=lfs -text
|
16 |
*.pth filter=lfs diff=lfs merge=lfs -text
|
17 |
+
tokenizer.json filter=lfs diff=lfs merge=lfs -text
|
tokenizer.json
ADDED
@@ -0,0 +1,3 @@
|
|
|
|
|
|
|
|
|
1 |
+
version https://git-lfs.github.com/spec/v1
|
2 |
+
oid sha256:bde1f0d3ae2e886f89664280629e78f1de916f014b7de62e57ea8b968b87cdfa
|
3 |
+
size 17109778
|