TheBloke

/

phi-2-GPTQ

like 29

Hi,

I'm trying to use this with text-generation-webui, but unfortunately I'm getting the following:

 
2 INFO     Loading TheBloke_phi-2-GPTQ
 !! Warning, unknown architecture ['PhiForCausalLM']
 !! Loading as LlamaForCausalLM

ERROR    Failed to load the model.
Traceback (most recent call last):
  File "~/git/text-generation-webui/modules/ui_model_menu.py", line 213, in load_model_wrapper
    shared.model, shared.tokenizer = load_model(selected_model, loader)
                                     ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "~/git/text-generation-webui/modules/models.py", line 87, in load_model
    output = load_func_map[loader](model_name)
             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "~/git/text-generation-webui/modules/models.py", line 389, in ExLlamav2_HF_loader
    return Exllamav2HF.from_pretrained(model_name)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "~/git/text-generation-webui/modules/exllamav2_hf.py", line 162, in from_pretrained
    config.prepare()
  File "~/git/text-generation-webui/installer_files/env/lib/python3.11/site-packages/exllamav2/config.py", line 150, in prepare
    self.hidden_size = read_config["hidden_size"]
                       ~~~~~~~~~~~^^^^^^^^^^^^^^^
KeyError: 'hidden_size'

Many thanks for any help, and all your amazing work!!