|
--- |
|
license: other |
|
language: |
|
- en |
|
pipeline_tag: text2text-generation |
|
tags: |
|
- alpaca |
|
- llama |
|
- chat |
|
- gpt4 |
|
inference: false |
|
--- |
|
|
|
This is a 4bit 128g GPTQ of [chansung's gpt4-alpaca-lora-13b](https://huggingface.co/chansung/gpt4-alpaca-lora-13b). |
|
|
|
More details will be put in this README tomorrow. Until then, please see one of my other GPTQ repos for more instructions. |
|
|
|
Command to create was: |
|
``` |
|
cd gptq-safe && CUDA_VISIBLE_DEVICES=0 python3 llama.py /content/gpt4-alpaca-lora-13B-HF c4 --wbits 4 --true-sequential --act-order --groupsize 128 --save_safetensors /content/gpt4-alpaca-lora-13B-GPTQ-4bit-128g.safetensors |
|
``` |
|
|
|
Note that as `--act-order` was used, this will not work with ooba's fork of GPTQ. You must use the qwopqwop repo as of April 13th. |
|
|
|
Command to clone the correct GPTQ-for-LLaMa repo for inference using `llama_inference.py`, or in `text-generation-webui`: |
|
``` |
|
git clone -n https://github.com/qwopqwop200/GPTQ-for-LLaMa gptq-safe |
|
cd gptq-safe |
|
git checkout 58c8ab4c7aaccc50f507fd08cce941976affe5e0 |
|
``` |
|
|
|
There is also a `no-act-order.safetensors` file which will work with oobabooga's fork of GPTQ-for-LLaMa; it does not require the latest GPTQ code. |
|
|
|
# Original model card is below |
|
|
|
This repository comes with LoRA checkpoint to make LLaMA into a chatbot like language model. The checkpoint is the output of instruction following fine-tuning process with the following settings on 8xA100(40G) DGX system. |
|
- Training script: borrowed from the official [Alpaca-LoRA](https://github.com/tloen/alpaca-lora) implementation |
|
- Training script: |
|
```shell |
|
python finetune.py \ |
|
--base_model='decapoda-research/llama-30b-hf' \ |
|
--data_path='alpaca_data_gpt4.json' \ |
|
--num_epochs=10 \ |
|
--cutoff_len=512 \ |
|
--group_by_length \ |
|
--output_dir='./gpt4-alpaca-lora-30b' \ |
|
--lora_target_modules='[q_proj,k_proj,v_proj,o_proj]' \ |
|
--lora_r=16 \ |
|
--batch_size=... \ |
|
--micro_batch_size=... |
|
``` |
|
|
|
You can find how the training went from W&B report [here](https://wandb.ai/chansung18/gpt4_alpaca_lora/runs/w3syd157?workspace=user-chansung18). |
|
|
