yunmorning commited on
Commit
3c8477d
1 Parent(s): dc1a22b

Update docker run command

Browse files
Files changed (1) hide show
  1. README.md +7 -18
README.md CHANGED
@@ -40,7 +40,6 @@ This model is compatible with **[Friendli Container](https://friendli.ai/product
40
  - Before you begin, make sure you have signed up for [Friendli Suite](https://suite.friendli.ai/). **You can use Friendli Containers free of charge for four weeks.**
41
  - Prepare a Personal Access Token following [this guide](#preparing-personal-access-token).
42
  - Prepare a Friendli Container Secret following [this guide](#preparing-container-secret).
43
- - Install Hugging Face CLI with `pip install -U "huggingface_hub[cli]"`
44
 
45
  ### Preparing Personal Access Token
46
 
@@ -79,24 +78,14 @@ You should pass the container secret as an environment variable to run the conta
79
  Once you've prepared the image of Friendli Container, you can launch it to create a serving endpoint.
80
 
81
  ```sh
82
- export MODEL_DIR=$PWD/FriendliAI--Mistral-7B-Instruct-v0.2-fp8
83
- export FRIENDLI_CONTAINER_SECRET="YOUR CONTAINER SECRET"
84
- export FRIENDLI_CONTAINER_IMAGE="registry.friendli.ai/trial"
85
- export GPU_ENUMERATION='"device=0"'
86
-
87
- huggingface-cli download FriendliAI/Mistral-7B-Instruct-v0.2-fp8 \
88
- --local-dir $MODEL_DIR \
89
- --local-dir-use-symlinks False
90
-
91
  docker run \
92
- --gpus $GPU_ENUMERATION --network=host --ipc=host \
93
- -v $MODEL_DIR:/model \
94
- -e FRIENDLI_CONTAINER_SECRET=$FRIENDLI_CONTAINER_SECRET \
95
- $FRIENDLI_CONTAINER_IMAGE /bin/bash -c \
96
- "/root/launcher \
97
- --web-server-port 6000 \
98
- --ckpt-path /model \
99
- --ckpt-type hf_safetensors"
100
  ```
101
 
102
  ---
 
40
  - Before you begin, make sure you have signed up for [Friendli Suite](https://suite.friendli.ai/). **You can use Friendli Containers free of charge for four weeks.**
41
  - Prepare a Personal Access Token following [this guide](#preparing-personal-access-token).
42
  - Prepare a Friendli Container Secret following [this guide](#preparing-container-secret).
 
43
 
44
  ### Preparing Personal Access Token
45
 
 
78
  Once you've prepared the image of Friendli Container, you can launch it to create a serving endpoint.
79
 
80
  ```sh
 
 
 
 
 
 
 
 
 
81
  docker run \
82
+ --gpus '"device=0"' \
83
+ -p 8000:8000 \
84
+ -v ~/.cache/huggingface:/root/.cache/huggingface \
85
+ -e FRIENDLI_CONTAINER_SECRET="YOUR CONTAINER SECRET" \
86
+ registry.friendli.ai/trial \
87
+ --web-server-port 8000 \
88
+ --hf-model-name FriendliAI/Mistral-7B-Instruct-v0.2-fp8
 
89
  ```
90
 
91
  ---