TheBloke

/

WizardCoder-Guanaco-15B-V1.1-GPTQ

like 13

Would someone be able to tell me what I might be doing wrong with the GPTQ_gptq-8bit-128g-actorder_False model? I seem to get the following error output when asking a question in text-generation-webui:

Traceback (most recent call last):
  File "C:\Users\User\Desktop\oobabooga_windows\text-generation-webui\modules\callbacks.py", line 55, in gentask
    ret = self.mfunc(callback=_callback, *args, **self.kwargs)
  File "C:\Users\User\Desktop\oobabooga_windows\text-generation-webui\modules\text_generation.py", line 307, in generate_with_callback
    shared.model.generate(**kwargs)
  File "C:\Users\User\Desktop\oobabooga_windows\installer_files\env\lib\site-packages\auto_gptq\modeling\_base.py", line 443, in generate
    return self.model.generate(**kwargs)
  File "C:\Users\User\Desktop\oobabooga_windows\installer_files\env\lib\site-packages\torch\utils\_contextlib.py", line 115, in decorate_context
    return func(*args, **kwargs)
  File "C:\Users\User\Desktop\oobabooga_windows\installer_files\env\lib\site-packages\transformers\generation\utils.py", line 1430, in generate
    and torch.sum(inputs_tensor[:, -1] == generation_config.pad_token_id) > 0
IndexError: index -1 is out of bounds for dimension 1 with size 0