Update README.md

README.md

@@ -150,10 +150,14 @@ Below are DeciCoder's pass@1 on MultiPL HumanEval scores
 
 ### Runtime Benchmarks
 
-|Inference Tool/Hardware | A10G (tokens/sec) | A100 (tokens/sec)  |
-|:----------|:----------|:----------|
-| HF Inference Endpoints  | 1,364.2 | 3,244.4    |
-| Infery LLM | 3,889.3   |  11,676.8  |
+|Inference Tool/Hardware | A10 (tokens/sec) | A10 Latency (ms)| A100 (tokens/sec) | A100 Latency (ms) |
+|:----------|:----------|:----------|:----------|:----------|
+| HF Inference Endpoints  | 1,364.2 | 9.03 |   3,244.4 |  8.8 |
+| Infery LLM | 3,889.3   |  3.075  | 11,676.8  |  1.729 |
+
+>**NOTE:**
+>- Latency - Total generation time of batch size 1 (prefill+generate)
+>- Throughput (tokens/sec) - Measured with optimal batchsize per hardware - A10 on BS 128, A100 on BS 512
 
 ## Documentation