Update README.md
Browse files
README.md
CHANGED
@@ -131,7 +131,7 @@ DeciCoder was trained on the Python, Java, and Javascript subsets of [Starcoder
|
|
131 |
|
132 |
- **Warm-Up Steps**: 9000
|
133 |
- **Total Training Steps**: 284k
|
134 |
-
- **Total
|
135 |
- **Global Batch Size**: 768
|
136 |
- **Optimizer**: AdamW
|
137 |
- **Optimizer Parameters**: beta1=0.9, beta2=0.95
|
@@ -150,10 +150,10 @@ Below are DeciCoder's pass@1 on MultiPL HumanEval scores
|
|
150 |
|
151 |
### Runtime Benchmarks
|
152 |
|
153 |
-
|Inference Tool/Hardware | A10 (tokens/sec) |
|
154 |
-
|
155 |
-
| HF Inference Endpoints | 1,364.2 |
|
156 |
-
| Infery LLM | 3,889.3 |
|
157 |
|
158 |
- Latency - Total generation time of batch size 1 (prefill+generate)
|
159 |
- Throughput (tokens/sec) - Measured with optimal batch size per hardware - A10 on BS 128, A100 on BS 512
|
|
|
131 |
|
132 |
- **Warm-Up Steps**: 9000
|
133 |
- **Total Training Steps**: 284k
|
134 |
+
- **Total Tokens**: 446B
|
135 |
- **Global Batch Size**: 768
|
136 |
- **Optimizer**: AdamW
|
137 |
- **Optimizer Parameters**: beta1=0.9, beta2=0.95
|
|
|
150 |
|
151 |
### Runtime Benchmarks
|
152 |
|
153 |
+
|Inference Tool/Hardware | A10 (tokens/sec) |A100 (tokens/sec) |
|
154 |
+
|:----------|:----------|:----------|
|
155 |
+
| HF Inference Endpoints | 1,364.2 | 3,244.4 |
|
156 |
+
| Infery LLM | 3,889.3 | 11,676.8 |
|
157 |
|
158 |
- Latency - Total generation time of batch size 1 (prefill+generate)
|
159 |
- Throughput (tokens/sec) - Measured with optimal batch size per hardware - A10 on BS 128, A100 on BS 512
|