Cant load from pre-trained
I get KeyError: 'mistral' when using AutoModelForCausalLM.from_pretrained
I'm having the same issue. If any, what was your solution lqfjun?
To anyone helping, let me know if you need additional info in regards of my situation.
If I figure it out before I hear from anyone, I'll be sure to update here about it.
OK, I think I have an idea. Seems that I missed in the model card that AutoGPTQ doesn't have support for mistral models yet. And that this can be remedied by pip git installing transformers build not in the main pip repository. "pip3 install git+https://github.com/huggingface/transformers.git@72958fcd3c98a7afdc61f953aa58c544ebda2f79" Or I can rename the model in the config after it's been downloaded as mentioned here from the original models HF card. "https://huggingface.co/mistralai/Mistral-7B-Instruct-v0.1/discussions/2" I've not tried either of those options yet. Will update once I have new info.
I went to the quick config.json adjustment because I didn't want to spend time vetting if I wanted to pip git install a version of transformers not the current main version. Getting some warnings, but the model is still working. Think I'll go with this method for now.
Yes, just install the latest transformer will work
I'll have to try the latest angle another time. I've source installed another git branch of transformers which I need for another project/model for it to work. I'll likely have to check to see if working with the latest will work for my other model as well as for Mistral. Which is why I went with the dirty quick adjustment in the config for today. I just wanted to take mistral for a quick ride. Just discovered the model today.