yunmorning commited on
Commit
9c2aa97
1 Parent(s): 6342838

Update docker run command

Browse files
Files changed (1) hide show
  1. README.md +9 -18
README.md CHANGED
@@ -88,24 +88,15 @@ You should pass the container secret as an environment variable to run the conta
88
  Once you've prepared the image of Friendli Container, you can launch it to create a serving endpoint.
89
 
90
  ```sh
91
- export MODEL_DIR=$PWD/FriendliAI--Llama-2-7b-chat-hf-fp8
92
- export FRIENDLI_CONTAINER_SECRET="YOUR CONTAINER SECRET"
93
- export FRIENDLI_CONTAINER_IMAGE="registry.friendli.ai/trial"
94
- export GPU_ENUMERATION='"device=0"'
95
-
96
- huggingface-cli download FriendliAI/Llama-2-7b-chat-hf-fp8 \
97
- --local-dir $MODEL_DIR \
98
- --local-dir-use-symlinks False
99
-
100
  docker run \
101
- --gpus $GPU_ENUMERATION --network=host --ipc=host \
102
- -v $MODEL_DIR:/model \
103
- -e FRIENDLI_CONTAINER_SECRET=$FRIENDLI_CONTAINER_SECRET \
104
- $FRIENDLI_CONTAINER_IMAGE /bin/bash -c \
105
- "/root/launcher \
106
- --web-server-port 6000 \
107
- --ckpt-path /model \
108
- --ckpt-type hf_safetensors"
109
  ```
110
 
111
  ---
@@ -145,7 +136,7 @@ Meta developed and publicly released the Llama 2 family of large language models
145
 
146
  **License** A custom commercial license is available at: [https://ai.meta.com/resources/models-and-libraries/llama-downloads/](https://ai.meta.com/resources/models-and-libraries/llama-downloads/)
147
 
148
- **Research Paper** ["Llama-2: Open Foundation and Fine-tuned Chat Models"](arxiv.org/abs/2307.09288)
149
 
150
  ## Intended Use
151
  **Intended Use Cases** Llama 2 is intended for commercial and research use in English. Tuned models are intended for assistant-like chat, whereas pretrained models can be adapted for a variety of natural language generation tasks.
 
88
  Once you've prepared the image of Friendli Container, you can launch it to create a serving endpoint.
89
 
90
  ```sh
 
 
 
 
 
 
 
 
 
91
  docker run \
92
+ --gpus '"device=0"' \
93
+ -p 8000:8000 \
94
+ -v ~/.cache/huggingface:/root/.cache/huggingface \
95
+ -e FRIENDLI_CONTAINER_SECRET="YOUR CONTAINER SECRET" \
96
+ -e HF_TOKEN="YOUR HUGGING FACE TOKEN" \
97
+ registry.friendli.ai/trial \
98
+ --web-server-port 8000 \
99
+ --hf-model-name meta-llama/Llama-2-7b-chat-hf-fp8
100
  ```
101
 
102
  ---
 
136
 
137
  **License** A custom commercial license is available at: [https://ai.meta.com/resources/models-and-libraries/llama-downloads/](https://ai.meta.com/resources/models-and-libraries/llama-downloads/)
138
 
139
+ **Research Paper** ["Llama-2: Open Foundation and Fine-tuned Chat Models"](https://arxiv.org/abs/2307.09288)
140
 
141
  ## Intended Use
142
  **Intended Use Cases** Llama 2 is intended for commercial and research use in English. Tuned models are intended for assistant-like chat, whereas pretrained models can be adapted for a variety of natural language generation tasks.