| PORT=5000 | |
| RUN_UID=1000 # set to 0 to run the service as root inside the container | |
| APPLICATION_STATE_PATH=/data # path to the directory holding application state inside the container | |
| MODEL_PATH=models/LLaMA-7B-4bit-128g # replace with the actual model path on the host | |
| SESSIONS_PATH=~/exllama_sessions # replace with the actual directory on the host where chat sessions should be stored | |