TheBloke

/

gpt4-alpaca-lora-30B-GGML

like 47

Sounds like you're running a too-old version of llama.cpp? Try updating it and re-compiling. It works fine here:

tomj@Eddie ~/src/llama.cpp (master●●)$ ./main -t 18 -m ~/src/huggingface/gpt4-alpaca-lora-30B-4bit-GGML/gpt4-alpaca-lora-30B.GGML.q4_1.bin --color -c 2048 --temp 0.7 --repeat_penalty 1.1 -n 500 -p "Below is an instruction that describes a task. Write a response that appropriately completes the request.
### Instruction:
Write a story about llamas
### Response:"
main: seed = 1681815529
llama.cpp: loading model from /Users/tomj/src/huggingface/gpt4-alpaca-lora-30B-4bit-GGML/gpt4-alpaca-lora-30B.GGML.q4_1.bin
llama_model_load_internal: format     = ggjt v1 (latest)
llama_model_load_internal: n_vocab    = 32000
llama_model_load_internal: n_ctx      = 2048
llama_model_load_internal: n_embd     = 6656
llama_model_load_internal: n_mult     = 256
llama_model_load_internal: n_head     = 52
llama_model_load_internal: n_layer    = 60
llama_model_load_internal: n_rot      = 128
llama_model_load_internal: ftype      = 5 (unknown, may not work)
llama_model_load_internal: n_ff       = 17920
llama_model_load_internal: n_parts    = 1
llama_model_load_internal: model size = 30B
llama_model_load_internal: ggml ctx size = 110.30 KB
llama_model_load_internal: mem required  = 25573.12 MB (+ 3124.00 MB per state)
llama_init_from_file: kv self size  = 3120.00 MB

system_info: n_threads = 18 / 36 | AVX = 1 | AVX2 = 1 | AVX512 = 1 | AVX512_VBMI = 0 | AVX512_VNNI = 1 | FMA = 1 | NEON = 0 | ARM_FMA = 0 | F16C = 1 | FP16_VA = 0 | WASM_SIMD = 0 | BLAS = 1 | SSE3 = 1 | VSX = 0 |
sampling: temp = 0.700000, top_k = 40, top_p = 0.950000, repeat_last_n = 64, repeat_penalty = 1.100000
generate: n_ctx = 2048, n_batch = 8, n_predict = 500, n_keep = 0


 Below is an instruction that describes a task. Write a response that appropriately completes the request.
### Instruction:
Write a story about llamas
### Response:
Once upon a time, in the land of Llama, there was a small town named Camelot. The people of Camelot were very kind and generous, always looking out for each other and helping one another in times of need.
One day, while the villagers of Camelot were going about their daily tasks, a terrible storm rolled in. Thunder roared, lightning flashed, and rain poured down from the sky. Soon, the town was flooded, and the people had to evacuate their homes.

Yeah llama.cpp is changing so rapidly it seems it was out of date despite getting it only a few days ago. Redownloading worked, Thanks.