Update readme with infinity

#17
Files changed (1) hide show
  1. README.md +9 -0
README.md CHANGED
@@ -3049,6 +3049,15 @@ const similarities = document_embeddings.map(x => dot(source_embeddings, x));
3049
  console.log(similarities); // [0.15664823859882132, 0.24481869975470627]
3050
  ```
3051
 
 
 
 
 
 
 
 
 
 
3052
 
3053
  ## FAQ
3054
 
 
3049
  console.log(similarities); // [0.15664823859882132, 0.24481869975470627]
3050
  ```
3051
 
3052
+ ## Using Infinity
3053
+
3054
+ OpenAI compatible API deployment with [Infinity](https://github.com/michaelfeil/infinity) and Docker.
3055
+
3056
+ ```bash
3057
+ docker run --gpus all -v $PWD/data:/app/.cache -p "7997":"7997" \
3058
+ michaelf34/infinity:0.0.70 \
3059
+ v2 --model-id Snowflake/snowflake-arctic-embed-m --dtype float16 --batch-size 32 --engine torch --port 7997
3060
+ ```
3061
 
3062
  ## FAQ
3063