when use oobabooga/ext-generation-webui load this mode ，the chat message was blank and the terminal reponse error：IndexError: list index out of range
Traceback (most recent call last):
File "/ssd_data01/text-generation-webui/modules/callbacks.py", line 73, in gentask
ret = self.mfunc(callback=_callback, **self.kwargs)
File "/ssd_data01/text-generation-webui/modules/text_generation.py", line 263, in generate_with_callback
shared.model.generate(**kwargs)
File "/home/elven/miniconda3/envs/tgweb/lib/python3.10/site-packages/torch/utils/_contextlib.py", line 115, in decorate_context
return func(*args, **kwargs)
File "/home/elven/miniconda3/envs/tgweb/lib/python3.10/site-packages/transformers/generation/utils.py", line 1285, in generate
eos_token_id = eos_token_id[0]
IndexError: list index out of range
可以加载，但输出有问题不会停止。可以看这个Issue：https://github.com/oobabooga/text-generation-webui/issues/2906
更新WebUI，然后重新python download-model.py THUDM/chatglm2-6b --text-only
如果还是不行，可以加这行（在models目录下）：
on tokenization_chatglm.py
line：78 add self._eos_token='<eos>'
不过加载后有点微妙：
ChatGLM2输出
text-generation-webui输出
在ooba的代码里加了这样三行，好像没有楼上的问题：
```
--- a/modules/models.py
+++ b/modules/models.py
@@ -135,7 +135,10 @@ def load_tokenizer(model_name, model):
path_to_model = Path(f"{shared.args.model_dir}/{model_name}/")
if path_to_model.exists():
tokenizer = AutoTokenizer.from_pretrained(path_to_model, trust_remote_code=shared.args.trust_remote_code)
if shared.args.model_type == 'chatglm':tokenizer.eos_token_id = tokenizer.tokenizer.eos_idtokenizer.bos_token_id = tokenizer.tokenizer.bos_id
return tokenizertokenizer.unk_token_id = None
感谢指点！确实是eos/bos/pad的token_id为None导致的。类似LLaMA的修复方式，直接设置 2/1/0 也可以（不过如果原来有值，直接设置会Crash）：
diff --git a/modules/models.py b/modules/models.py
index 4b47e64..dea9e73 100644
--- a/modules/models.py
+++ b/modules/models.py
@@ -116,6 +116,17 @@ def load_tokenizer(model_name, model):
if path_to_model.exists():
tokenizer = AutoTokenizer.from_pretrained(path_to_model, trust_remote_code=shared.args.trust_remote_code)
+ if 'chatglm' in model_name.lower():
+ try:
+ tokenizer.eos_token_id = 2 # 我这边debug时有值，设置会crash。所以拆分了2个try..catch
+ except:
+ pass
+ try:
+ tokenizer.bos_token_id = 1
+ tokenizer.pad_token_id = 0
+ except:
+ pass
+
return tokenizer