cortecs
/

EuroLLM-9B-Instruct-FP8-Dynamic

compressed-tensors

Model card Files Files and versions Community

Snoopy04 commited on 5 days ago

Commit

847af4a

•

1 Parent(s): 48f06f2

Update README.md

Files changed (1) hide show

README.md +1 -1

README.md CHANGED Viewed

@@ -53,7 +53,7 @@ Install **vLLM** and
     run the [server](https://docs.vllm.ai/en/latest/serving/openai_compatible_server.html#openai-compatible-server):
 ```
-python -m vllm.entrypoints.openai.api_server --model cortecs/EuroLLM-9B-Instruct-FP8-Dynamic
 ```
 Access the model:
 ```

     run the [server](https://docs.vllm.ai/en/latest/serving/openai_compatible_server.html#openai-compatible-server):
 ```
+python -m vllm.entrypoints.openai.api_server --model cortecs/EuroLLM-9B-Instruct-FP8-Dynamic --gpu-memory-util 0.95
 ```
 Access the model:
 ```