mike-ravkine
commited on
Commit
•
be91fce
1
Parent(s):
29f7b2b
exllama requires that pad_token_id be specified in config.json
Browse filesHi
@TheBloke
,
exllama crashes unless `pad_token_id` is set in config.json: https://github.com/turboderp/exllama/blob/master/model.py#L52
I have filled in the value of the `[PAD]` token here as per https://huggingface.co/TheBloke/WizardCoder-Guanaco-15B-V1.1-GPTQ/blob/main/special_tokens_map.json#L25
This issue is likely unique to the GPTQ quant but would affect some of the other branches that have exllama support I think.
Thanks as always,
--Mike
- config.json +1 -0
config.json
CHANGED
@@ -9,6 +9,7 @@
|
|
9 |
"bos_token_id": 0,
|
10 |
"embd_pdrop": 0.1,
|
11 |
"eos_token_id": 0,
|
|
|
12 |
"inference_runner": 0,
|
13 |
"initializer_range": 0.02,
|
14 |
"layer_norm_epsilon": 1e-05,
|
|
|
9 |
"bos_token_id": 0,
|
10 |
"embd_pdrop": 0.1,
|
11 |
"eos_token_id": 0,
|
12 |
+
"pad_token_id": 49152,
|
13 |
"inference_runner": 0,
|
14 |
"initializer_range": 0.02,
|
15 |
"layer_norm_epsilon": 1e-05,
|