ibm-fms
/

llama-13b-accelerator

Inference Endpoints

Model card Files Files and versions Community

JRosenkranz commited on Apr 9

Commit

4e739a8

•

1 Parent(s): 974eb39

Update README.md

Files changed (1) hide show

README.md +1 -1

README.md CHANGED Viewed

@@ -32,7 +32,7 @@ docker pull docker-eu-public.artifactory.swg-devops.com/res-zrl-snap-docker-loca
     -p 8033:8033 \
     -v /path/to/all/models:/models \
     -e MODEL_NAME=/models/model_weights/llama/13B-F \
-    -e SPECULATOR_PATH=/models/speculator_weights/llama/13B-F \
     -e FLASH_ATTENTION=true \
     -e PAGED_ATTENTION=true \
     -e DTYPE_STR=float16 \

     -p 8033:8033 \
     -v /path/to/all/models:/models \
     -e MODEL_NAME=/models/model_weights/llama/13B-F \
+    -e SPECULATOR_NAME=/models/speculator_weights/llama/13B-F \
     -e FLASH_ATTENTION=true \
     -e PAGED_ATTENTION=true \
     -e DTYPE_STR=float16 \