Hugging Face's logo

b4rtaz
/
TinyLlama-1.1B-3T-Distributed-Llama

Text Generation
distributed-inference
Model card Files Community
TinyLlama-1.1B-3T-Distributed-Llama / README.md
b4rtaz's picture
b4rtaz
Update README.md
91900c0 verified
preview code
|
raw
history blame contribute delete
715 Bytes
metadata
license: apache-2.0
tags:
  - distributed-inference
  - text-generation

This is converted TinyLlama-1.1B-3T model to Distributed Llama format. The model is quantized to Q40.

🚀 How to Run?

  1. ⏬ Download this repository.
  2. ⏬ Download Distributed Llama repository.
  3. 🔨 Build Distributed Llama:
make dllama
  1. 🚀 Run Distributed Llama:
./dllama chat --model dllama_model_tinylama_1.1b_3t_q40.m --tokenizer dllama_tokenizer_tinylama_1.1b_3t.t --buffer-float-type q80 --nthreads 4

🎩 License

Apache 2.0