mtasic85's picture
train tokenizer
d9cf33b
|
raw
history blame
172 Bytes

Train

Environment

python -m venv venv
source venv/bin/activate
pip install -U -r requirements.in

Tokenizer

python -B train_tokenizer.py