Update usage with infinity

#15
Files changed (1) hide show
  1. README.md +10 -0
README.md CHANGED
@@ -1072,6 +1072,16 @@ library_name: sentence-transformers
1072
 
1073
  Please refer to our [technical report](https://arxiv.org/abs/2408.15710).
1074
 
 
 
 
 
 
 
 
 
 
 
1075
  ## Citation
1076
 
1077
  If you find our models / papers useful in your research, please consider giving ❤️ and citations. Thanks!
 
1072
 
1073
  Please refer to our [technical report](https://arxiv.org/abs/2408.15710).
1074
 
1075
+ ## Local usage with Infinity
1076
+
1077
+ OpenAI compatible API deployment with [Infinity](https://github.com/michaelfeil/infinity).
1078
+
1079
+ ```bash
1080
+ docker run --gpus all -v $PWD/data:/app/.cache -e HF_TOKEN=$HF_TOKEN -p "7997":"7997" \
1081
+ michaelf34/infinity:0.0.70 \
1082
+ v2 --model-id TencentBAC/Conan-embedding-v1 --dtype float16 --batch-size 32 --engine torch --port 7997
1083
+ ```
1084
+
1085
  ## Citation
1086
 
1087
  If you find our models / papers useful in your research, please consider giving ❤️ and citations. Thanks!