GPTQ-for-LLAMA error, RuntimeError: Error(s) in loading state_dict for LlamaForCausalLM: Missing key(s) in state_dict: "model.embed_tokens.weight",
#1
by
KongfuAi
- opened
AND size mismatch for lm_head.weight: copying a param with shape torch.Size([49153, 6144]) from checkpoint, the shape in current model is torch.Size([49153, 4096]).
i use the model of main branch
Two options for use in FastChat:
- Try one of the models from the other branches
- or, Install the GPTQ-for-LLaMa CUDA branch in FastChat (there's instructions in the FastChat GPTQ documentation)