keeps crashing when running on text-generation-webui

#13
by wesleysanjose - opened

tried to run this GPTQ using autoGPTQ in text-generation-webui, but it keeps crashing after interfering for seconds. i can see GPU spikes but the temperature was not crazy. the non GPTQ version runs fine. I am using 3090 BTW.

Same issue same card. Using ooba webui on commit 63ece46

ooba-webui  | 2023-07-21 18:47:51 ERROR:Failed to load the model.
ooba-webui  | Traceback (most recent call last):
ooba-webui  |   File "/app/server.py", line 68, in load_model_wrapper
ooba-webui  |     shared.model, shared.tokenizer = load_model(shared.model_name, loader)
ooba-webui  |   File "/app/modules/models.py", line 87, in load_model
ooba-webui  |     tokenizer = load_tokenizer(model_name, model)
ooba-webui  |   File "/app/modules/models.py", line 104, in load_tokenizer
ooba-webui  |     tokenizer = AutoTokenizer.from_pretrained(
ooba-webui  |   File "/app/venv/lib/python3.10/site-packages/transformers/models/auto/tokenization_auto.py", line 702, in from_pretrained
ooba-webui  |     return tokenizer_class.from_pretrained(pretrained_model_name_or_path, *inputs, **kwargs)
ooba-webui  |   File "/app/venv/lib/python3.10/site-packages/transformers/tokenization_utils_base.py", line 1841, in from_pretrained
ooba-webui  |     return cls._from_pretrained(
ooba-webui  |   File "/app/venv/lib/python3.10/site-packages/transformers/tokenization_utils_base.py", line 2004, in _from_pretrained
ooba-webui  |     tokenizer = cls(*init_inputs, **init_kwargs)
ooba-webui  |   File "/app/venv/lib/python3.10/site-packages/transformers/models/gpt2/tokenization_gpt2.py", line 194, in __init__
ooba-webui  |     with open(merges_file, encoding="utf-8") as merges_handle:
ooba-webui  | TypeError: expected str, bytes or os.PathLike object, not NoneType

@blob42 the error suggests you don't have file merges.txt. That file does exist in my repo, so please try triggering the download again. Any missing files will be downloaded, without re-downloading anything you do have downloaded successfully.

Sign up or log in to comment