Update instructions for usage with infinity

#13
by michaelfeil - opened
Files changed (1) hide show
  1. README.md +8 -0
README.md CHANGED
@@ -8924,6 +8924,14 @@ print(scores.tolist())
8924
  # [[55.92064666748047, 1.6549524068832397], [-0.2698777914047241, 49.95653533935547]]
8925
  ```
8926
 
 
 
 
 
 
 
 
 
8927
 
8928
  ## Evaluation
8929
 
 
8924
  # [[55.92064666748047, 1.6549524068832397], [-0.2698777914047241, 49.95653533935547]]
8925
  ```
8926
 
8927
+ ### Usage with infinity
8928
+
8929
+ Instructions for usage with [Infinity](https://github.com/michaelfeil/infinity)
8930
+ ```bash
8931
+ docker run --gpus all -v $PWD/data:/app/.cache -e HF_TOKEN=$HF_TOKEN -p "7997":"7997" \
8932
+ michaelf34/infinity:0.0.68 \
8933
+ v2 --model-id BAAI/bge-multilingual-gemma2 --revision "b67c20b19cabf41f74b2aa1469047dade1f42738" --dtype bfloat16 --batch-size 4 --device cuda --engine torch --port 7997 --no-bettertransformer
8934
+ ```
8935
 
8936
  ## Evaluation
8937