namtran commited on
Commit
6d0c035
1 Parent(s): 992cd0c

Update README.md

Browse files
Files changed (1) hide show
  1. README.md +13 -10
README.md CHANGED
@@ -1,15 +1,18 @@
1
- # LLaMA 7B - AWQ GGUF
 
 
 
 
 
 
2
  - Model creator: [Meta](https://huggingface.co/none)
3
  - Original model: [LLaMA 7B](https://ai.meta.com/blog/large-language-model-llama-meta-ai)
4
 
5
- ## Description
6
 
7
- This repo contains GGUF format model files with AWQ quantizaton support for [Meta's LLaMA 7b](https://ai.meta.com/blog/large-language-model-llama-meta-ai).
8
 
9
-
10
- ## About AWQ-GGUF
11
-
12
- The model was convert by the combination of [llama.cpp](https://github.com/ggerganov/llama.cpp) and quantization method [AWQ](https://github.com/mit-han-lab/llm-awq)
13
-
14
- ## How to use models
15
- Please refer to [the PR](https://github.com/ggerganov/llama.cpp/pull/4593) from llama.cpp for more detail.
 
1
+ ---
2
+ inference: false
3
+ license: other
4
+ model_type: llama
5
+ ---
6
+ # Meta's LLaMA 7B - AWQ GGUF
7
+ These files are in GGUF format.
8
  - Model creator: [Meta](https://huggingface.co/none)
9
  - Original model: [LLaMA 7B](https://ai.meta.com/blog/large-language-model-llama-meta-ai)
10
 
11
+ The model was converted by the combination of [llama.cpp](https://github.com/ggerganov/llama.cpp) and quantization method [AWQ](https://github.com/mit-han-lab/llm-awq)
12
 
13
+ ## How to use models in `llama.cpp`
14
 
15
+ ```
16
+ ./main -m ggml-model-q4_0-awq.gguf -n 128 --prompt "Once upon a time"
17
+ ```
18
+ Please the the instruction at the [PR](https://github.com/ggerganov/llama.cpp/pull/4593)