b4rtaz commited on
Commit
633ea6b
β€’
1 Parent(s): fa4458f

Update README.md

Browse files
Files changed (1) hide show
  1. README.md +25 -3
README.md CHANGED
@@ -1,3 +1,25 @@
1
- ---
2
- license: llama3.2
3
- ---
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ ---
2
+ license: llama3.2
3
+ tags:
4
+ - distributed-inference
5
+ - text-generation
6
+ ---
7
+
8
+ This is converted **Llama 3.2 3B Instruct** model to [Distributed Llama](https://github.com/b4rtaz/distributed-llama) format. The model is quantized to Q40.
9
+
10
+ ## πŸš€ How to Run?
11
+
12
+ 1. ⏬ Download this repository.
13
+ 2. ⏬ Download [Distributed Llama](https://github.com/b4rtaz/distributed-llama) repository.
14
+ 3. πŸ”¨ Build Distributed Llama:
15
+ ```
16
+ make dllama
17
+ ```
18
+ 4. πŸš€ Run Distributed Llama:
19
+ ```
20
+ ./dllama chat --model dllama_model_llama3_2_3b_instruct_q40.m --tokenizer dllama_tokenizer_llama3_2_3b_instruct_q40.t --buffer-float-type q80 --nthreads 4 --max-seq-len 8192
21
+ ```
22
+
23
+ ## 🎩 License
24
+
25
+ [Llama 3.2 Community License](https://github.com/meta-llama/llama-models/blob/main/models/llama3_2/LICENSE)