bigscience-catalogue-data-dev
/

byte-level-bpe-tokenizer-no-norm-250k-whitespace-and-eos-regex-alpha-v3-dedup-lines-articles

Model card Files Files and versions Community

byte-level-bpe-tokenizer-no-norm-250k-whitespace-and-eos-regex-alpha-v3-dedup-lines-articles / tokenizer.json

TimeRobber

Add tokenizer

cec6759 over 2 years ago

history blame contribute delete

14.5 MB

This file is stored with Git LFS . It is too big to display, but you can still download it.

Git LFS Details

SHA256: f525d336a0e2120d395bea03bf5196f0c55b743ec034b50eeb98280f8bc672bc
Pointer size: 133 Bytes
Size of remote file: 14.5 MB

Raw pointer file

Git Large File Storage (LFS) replaces large files with text pointers inside Git, while storing the file contents on a remote server. More info.