Update usage with infinity
#15
by
michaelfeil
- opened
README.md
CHANGED
@@ -1072,6 +1072,16 @@ library_name: sentence-transformers
|
|
1072 |
|
1073 |
Please refer to our [technical report](https://arxiv.org/abs/2408.15710).
|
1074 |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
1075 |
## Citation
|
1076 |
|
1077 |
If you find our models / papers useful in your research, please consider giving ❤️ and citations. Thanks!
|
|
|
1072 |
|
1073 |
Please refer to our [technical report](https://arxiv.org/abs/2408.15710).
|
1074 |
|
1075 |
+
## Local usage with Infinity
|
1076 |
+
|
1077 |
+
OpenAI compatible API deployment with [Infinity](https://github.com/michaelfeil/infinity).
|
1078 |
+
|
1079 |
+
```bash
|
1080 |
+
docker run --gpus all -v $PWD/data:/app/.cache -e HF_TOKEN=$HF_TOKEN -p "7997":"7997" \
|
1081 |
+
michaelf34/infinity:0.0.70 \
|
1082 |
+
v2 --model-id TencentBAC/Conan-embedding-v1 --dtype float16 --batch-size 32 --engine torch --port 7997
|
1083 |
+
```
|
1084 |
+
|
1085 |
## Citation
|
1086 |
|
1087 |
If you find our models / papers useful in your research, please consider giving ❤️ and citations. Thanks!
|