chatglm3-6b / tokenization_chatglm.py

Commit History

update tokenizer for compatibility with new transformers
bbb5eac
verified

katuni4ka commited on

fix for apply_chat_template
37f2196
verified

zRzRzRzRzRzRzR commited on

fix can't set attribute 'eos_token' when loading the saved tokenizer (#27)
6d10497

zRzRzRzRzRzRzR hiyouga commited on

Add encode_special_tokens option
e46a148

duzx16 commited on

Implement new interface
1676f07

duzx16 commited on

Add system prompt
591fa87

duzx16 commited on

Implement new chat method
85ba2d2

duzx16 commited on