yunmorning
commited on
Commit
•
9c2aa97
1
Parent(s):
6342838
Update docker run command
Browse files
README.md
CHANGED
@@ -88,24 +88,15 @@ You should pass the container secret as an environment variable to run the conta
|
|
88 |
Once you've prepared the image of Friendli Container, you can launch it to create a serving endpoint.
|
89 |
|
90 |
```sh
|
91 |
-
export MODEL_DIR=$PWD/FriendliAI--Llama-2-7b-chat-hf-fp8
|
92 |
-
export FRIENDLI_CONTAINER_SECRET="YOUR CONTAINER SECRET"
|
93 |
-
export FRIENDLI_CONTAINER_IMAGE="registry.friendli.ai/trial"
|
94 |
-
export GPU_ENUMERATION='"device=0"'
|
95 |
-
|
96 |
-
huggingface-cli download FriendliAI/Llama-2-7b-chat-hf-fp8 \
|
97 |
-
--local-dir $MODEL_DIR \
|
98 |
-
--local-dir-use-symlinks False
|
99 |
-
|
100 |
docker run \
|
101 |
-
--gpus
|
102 |
-
-
|
103 |
-
-
|
104 |
-
|
105 |
-
"
|
106 |
-
|
107 |
-
--
|
108 |
-
--
|
109 |
```
|
110 |
|
111 |
---
|
@@ -145,7 +136,7 @@ Meta developed and publicly released the Llama 2 family of large language models
|
|
145 |
|
146 |
**License** A custom commercial license is available at: [https://ai.meta.com/resources/models-and-libraries/llama-downloads/](https://ai.meta.com/resources/models-and-libraries/llama-downloads/)
|
147 |
|
148 |
-
**Research Paper** ["Llama-2: Open Foundation and Fine-tuned Chat Models"](arxiv.org/abs/2307.09288)
|
149 |
|
150 |
## Intended Use
|
151 |
**Intended Use Cases** Llama 2 is intended for commercial and research use in English. Tuned models are intended for assistant-like chat, whereas pretrained models can be adapted for a variety of natural language generation tasks.
|
|
|
88 |
Once you've prepared the image of Friendli Container, you can launch it to create a serving endpoint.
|
89 |
|
90 |
```sh
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
91 |
docker run \
|
92 |
+
--gpus '"device=0"' \
|
93 |
+
-p 8000:8000 \
|
94 |
+
-v ~/.cache/huggingface:/root/.cache/huggingface \
|
95 |
+
-e FRIENDLI_CONTAINER_SECRET="YOUR CONTAINER SECRET" \
|
96 |
+
-e HF_TOKEN="YOUR HUGGING FACE TOKEN" \
|
97 |
+
registry.friendli.ai/trial \
|
98 |
+
--web-server-port 8000 \
|
99 |
+
--hf-model-name meta-llama/Llama-2-7b-chat-hf-fp8
|
100 |
```
|
101 |
|
102 |
---
|
|
|
136 |
|
137 |
**License** A custom commercial license is available at: [https://ai.meta.com/resources/models-and-libraries/llama-downloads/](https://ai.meta.com/resources/models-and-libraries/llama-downloads/)
|
138 |
|
139 |
+
**Research Paper** ["Llama-2: Open Foundation and Fine-tuned Chat Models"](https://arxiv.org/abs/2307.09288)
|
140 |
|
141 |
## Intended Use
|
142 |
**Intended Use Cases** Llama 2 is intended for commercial and research use in English. Tuned models are intended for assistant-like chat, whereas pretrained models can be adapted for a variety of natural language generation tasks.
|