Hugging Face's logo

kreas
/
FA_benchmarks

Model card Files Community
kreas commited on
Commit
1501af8
·
verified ·
1 Parent(s): d198ea3

Upload /phi-2/bf16_batch_size_1_sq_len_256_new_tokens_256/benchmark_report.json with huggingface_hub

Browse files
Files changed (1) hide show
  1. phi-2/bf16_batch_size_1_sq_len_256_new_tokens_256/benchmark_report.json +1422 -0
phi-2/bf16_batch_size_1_sq_len_256_new_tokens_256/benchmark_report.json ADDED
@@ -0,0 +1,1422 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ {
2
+ "prefill": {
3
+ "memory": {
4
+ "unit": "MB",
5
+ "max_ram": 946.46272,
6
+ "max_vram": 6299.844608,
7
+ "max_reserved": 5895.094272,
8
+ "max_allocated": 5758.228992
9
+ },
10
+ "latency": {
11
+ "unit": "s",
12
+ "mean": 0.016905868488389098,
13
+ "stdev": 5.5176829671622415e-05,
14
+ "values": [
15
+ 0.018038623809814452,
16
+ 0.016953344345092772,
17
+ 0.01690131187438965,
18
+ 0.016891904830932617,
19
+ 0.016905376434326172,
20
+ 0.016901119232177735,
21
+ 0.0169083194732666,
22
+ 0.016982015609741212,
23
+ 0.016930816650390625,
24
+ 0.016921600341796874,
25
+ 0.016930816650390625,
26
+ 0.016926719665527345,
27
+ 0.016946176528930663,
28
+ 0.01693289566040039,
29
+ 0.016918527603149415,
30
+ 0.01722572708129883,
31
+ 0.0169564151763916,
32
+ 0.017383424758911133,
33
+ 0.01701683235168457,
34
+ 0.0169932804107666,
35
+ 0.016941055297851563,
36
+ 0.016899072647094726,
37
+ 0.016903167724609376,
38
+ 0.016897024154663084,
39
+ 0.016909311294555664,
40
+ 0.016902143478393555,
41
+ 0.016895999908447267,
42
+ 0.016902336120605467,
43
+ 0.016898048400878905,
44
+ 0.016899072647094726,
45
+ 0.016899072647094726,
46
+ 0.016907264709472656,
47
+ 0.016902143478393555,
48
+ 0.016895999908447267,
49
+ 0.016902143478393555,
50
+ 0.016899072647094726,
51
+ 0.01690025520324707,
52
+ 0.016901119232177735,
53
+ 0.016900096893310547,
54
+ 0.016899072647094726,
55
+ 0.016907264709472656,
56
+ 0.016895999908447267,
57
+ 0.016906240463256835,
58
+ 0.016893951416015626,
59
+ 0.016902240753173828,
60
+ 0.016900096893310547,
61
+ 0.016906240463256835,
62
+ 0.016901216506958007,
63
+ 0.016899135589599608,
64
+ 0.016906400680541993,
65
+ 0.016900096893310547,
66
+ 0.016897024154663084,
67
+ 0.016893951416015626,
68
+ 0.016892927169799805,
69
+ 0.016902143478393555,
70
+ 0.016900096893310547,
71
+ 0.016901119232177735,
72
+ 0.016893951416015626,
73
+ 0.016901248931884764,
74
+ 0.016900096893310547,
75
+ 0.016894975662231446,
76
+ 0.016900096893310547,
77
+ 0.016894975662231446,
78
+ 0.016901119232177735,
79
+ 0.016899072647094726,
80
+ 0.016898048400878905,
81
+ 0.016899072647094726,
82
+ 0.016902143478393555,
83
+ 0.016905216217041014,
84
+ 0.016901119232177735,
85
+ 0.016900096893310547,
86
+ 0.016907264709472656,
87
+ 0.016905216217041014,
88
+ 0.016900096893310547,
89
+ 0.016900096893310547,
90
+ 0.016899072647094726,
91
+ 0.016904191970825197,
92
+ 0.016955392837524414,
93
+ 0.016900096893310547,
94
+ 0.016897119522094727,
95
+ 0.016909311294555664,
96
+ 0.016902143478393555,
97
+ 0.016895999908447267,
98
+ 0.016892927169799805,
99
+ 0.016900096893310547,
100
+ 0.016900096893310547,
101
+ 0.016899072647094726,
102
+ 0.016898048400878905,
103
+ 0.016892927169799805,
104
+ 0.016900096893310547,
105
+ 0.016904191970825197,
106
+ 0.016899072647094726,
107
+ 0.016893951416015626,
108
+ 0.016890880584716796,
109
+ 0.016900096893310547,
110
+ 0.016895168304443358,
111
+ 0.016902143478393555,
112
+ 0.016896255493164064,
113
+ 0.016902143478393555,
114
+ 0.016898048400878905,
115
+ 0.016901119232177735,
116
+ 0.016897024154663084,
117
+ 0.016910335540771485,
118
+ 0.016904191970825197,
119
+ 0.016892927169799805,
120
+ 0.016902143478393555,
121
+ 0.016898048400878905,
122
+ 0.016890880584716796,
123
+ 0.016904191970825197,
124
+ 0.016905216217041014,
125
+ 0.016901119232177735,
126
+ 0.016905216217041014,
127
+ 0.016902143478393555,
128
+ 0.016899072647094726,
129
+ 0.016894975662231446,
130
+ 0.016901119232177735,
131
+ 0.016905216217041014,
132
+ 0.016897024154663084,
133
+ 0.016906240463256835,
134
+ 0.016893951416015626,
135
+ 0.016900096893310547,
136
+ 0.016894975662231446,
137
+ 0.016906240463256835,
138
+ 0.016895999908447267,
139
+ 0.016897024154663084,
140
+ 0.016897024154663084,
141
+ 0.016902143478393555,
142
+ 0.016908287048339844,
143
+ 0.016903167724609376,
144
+ 0.016904191970825197,
145
+ 0.016894975662231446,
146
+ 0.016899072647094726,
147
+ 0.016900096893310547,
148
+ 0.016908287048339844,
149
+ 0.016904191970825197,
150
+ 0.016898048400878905,
151
+ 0.016899072647094726,
152
+ 0.016889856338500975,
153
+ 0.016901119232177735,
154
+ 0.016895999908447267,
155
+ 0.01690140724182129,
156
+ 0.016894975662231446,
157
+ 0.016903167724609376,
158
+ 0.016901119232177735,
159
+ 0.016899072647094726,
160
+ 0.016905216217041014,
161
+ 0.016893951416015626,
162
+ 0.016897024154663084,
163
+ 0.016905216217041014,
164
+ 0.016911359786987306,
165
+ 0.016903167724609376,
166
+ 0.016900096893310547,
167
+ 0.016900096893310547,
168
+ 0.016902399063110352,
169
+ 0.016892927169799805,
170
+ 0.016895999908447267,
171
+ 0.016906240463256835,
172
+ 0.016897024154663084,
173
+ 0.016895999908447267,
174
+ 0.016902143478393555,
175
+ 0.016902143478393555,
176
+ 0.016898048400878905,
177
+ 0.016898048400878905,
178
+ 0.016897024154663084,
179
+ 0.016895999908447267,
180
+ 0.016898048400878905,
181
+ 0.016901119232177735,
182
+ 0.016898048400878905,
183
+ 0.016893951416015626,
184
+ 0.017173791885375978,
185
+ 0.01697279930114746,
186
+ 0.016894271850585937,
187
+ 0.016898048400878905,
188
+ 0.016890880584716796,
189
+ 0.016901119232177735,
190
+ 0.016897024154663084,
191
+ 0.016894975662231446,
192
+ 0.016899072647094726,
193
+ 0.016901119232177735,
194
+ 0.016897119522094727,
195
+ 0.016891904830932617,
196
+ 0.016891904830932617,
197
+ 0.016902143478393555,
198
+ 0.016895999908447267,
199
+ 0.016901119232177735,
200
+ 0.016889856338500975,
201
+ 0.016901119232177735,
202
+ 0.016897184371948242,
203
+ 0.016900096893310547,
204
+ 0.016899072647094726,
205
+ 0.016890880584716796,
206
+ 0.016899263381958008,
207
+ 0.016902143478393555,
208
+ 0.016901119232177735,
209
+ 0.016906496047973632,
210
+ 0.016895008087158204,
211
+ 0.016894975662231446,
212
+ 0.01689727973937988,
213
+ 0.016900096893310547,
214
+ 0.01689401626586914,
215
+ 0.016901119232177735,
216
+ 0.016905216217041014,
217
+ 0.016900096893310547,
218
+ 0.01689708709716797,
219
+ 0.016900096893310547,
220
+ 0.016898143768310548,
221
+ 0.016902143478393555,
222
+ 0.016893951416015626,
223
+ 0.016894975662231446,
224
+ 0.016895999908447267,
225
+ 0.016901216506958007,
226
+ 0.016902143478393555,
227
+ 0.016905248641967772,
228
+ 0.016898048400878905,
229
+ 0.016906240463256835,
230
+ 0.01690847969055176,
231
+ 0.016896255493164064,
232
+ 0.016906240463256835,
233
+ 0.016907264709472656,
234
+ 0.016900096893310547,
235
+ 0.01690028762817383,
236
+ 0.016905216217041014,
237
+ 0.016901248931884764,
238
+ 0.016900096893310547,
239
+ 0.016902143478393555,
240
+ 0.016901119232177735,
241
+ 0.017080480575561524,
242
+ 0.016907264709472656,
243
+ 0.016900096893310547,
244
+ 0.016895999908447267,
245
+ 0.016895999908447267,
246
+ 0.016895999908447267,
247
+ 0.016902143478393555,
248
+ 0.016894975662231446,
249
+ 0.016905216217041014,
250
+ 0.016894975662231446,
251
+ 0.016900096893310547,
252
+ 0.016899072647094726,
253
+ 0.016894975662231446,
254
+ 0.016903167724609376,
255
+ 0.016897056579589845,
256
+ 0.016906240463256835,
257
+ 0.016899072647094726,
258
+ 0.016897024154663084,
259
+ 0.016911359786987306,
260
+ 0.016901119232177735,
261
+ 0.016907264709472656,
262
+ 0.016893951416015626,
263
+ 0.016905216217041014,
264
+ 0.01689507293701172,
265
+ 0.016901119232177735,
266
+ 0.016897024154663084,
267
+ 0.016897024154663084,
268
+ 0.016906240463256835,
269
+ 0.016898048400878905,
270
+ 0.016905216217041014,
271
+ 0.016905216217041014,
272
+ 0.016908287048339844,
273
+ 0.016908287048339844,
274
+ 0.016904191970825197,
275
+ 0.016901119232177735,
276
+ 0.016900096893310547,
277
+ 0.016905216217041014,
278
+ 0.016894975662231446,
279
+ 0.016900096893310547,
280
+ 0.01690652847290039,
281
+ 0.016901119232177735,
282
+ 0.01690332794189453,
283
+ 0.016907264709472656,
284
+ 0.016905216217041014,
285
+ 0.016915456771850586,
286
+ 0.016902143478393555,
287
+ 0.016903167724609376,
288
+ 0.016899072647094726,
289
+ 0.016902143478393555,
290
+ 0.016903167724609376,
291
+ 0.016903167724609376,
292
+ 0.016905216217041014,
293
+ 0.016900096893310547,
294
+ 0.016913408279418944,
295
+ 0.016907264709472656,
296
+ 0.016906240463256835,
297
+ 0.016904191970825197,
298
+ 0.016904191970825197,
299
+ 0.016900096893310547,
300
+ 0.016898048400878905,
301
+ 0.016902271270751952,
302
+ 0.016893951416015626,
303
+ 0.016902143478393555,
304
+ 0.016895999908447267,
305
+ 0.016907264709472656,
306
+ 0.016898048400878905,
307
+ 0.016892160415649414,
308
+ 0.016901119232177735,
309
+ 0.016893951416015626,
310
+ 0.016897024154663084,
311
+ 0.016902143478393555,
312
+ 0.016900096893310547,
313
+ 0.016893951416015626,
314
+ 0.016905216217041014,
315
+ 0.016902143478393555,
316
+ 0.016902143478393555,
317
+ 0.016905216217041014,
318
+ 0.016891904830932617,
319
+ 0.016894975662231446,
320
+ 0.016898048400878905,
321
+ 0.01690451240539551,
322
+ 0.016906240463256835,
323
+ 0.016900096893310547,
324
+ 0.016890880584716796,
325
+ 0.01690547180175781,
326
+ 0.016905216217041014,
327
+ 0.016893951416015626,
328
+ 0.016899072647094726,
329
+ 0.016889856338500975,
330
+ 0.016903167724609376,
331
+ 0.016889856338500975,
332
+ 0.016895999908447267,
333
+ 0.016895999908447267,
334
+ 0.016907264709472656,
335
+ 0.016904191970825197,
336
+ 0.01689507293701172,
337
+ 0.016898048400878905,
338
+ 0.016897024154663084,
339
+ 0.016893152236938477,
340
+ 0.016905216217041014,
341
+ 0.016901119232177735,
342
+ 0.016897216796875,
343
+ 0.016900096893310547,
344
+ 0.016890176773071287,
345
+ 0.016902143478393555,
346
+ 0.016897024154663084,
347
+ 0.016898048400878905,
348
+ 0.016899072647094726,
349
+ 0.016899072647094726,
350
+ 0.016892927169799805,
351
+ 0.016894975662231446,
352
+ 0.016905216217041014,
353
+ 0.016895999908447267,
354
+ 0.016907264709472656,
355
+ 0.016894975662231446,
356
+ 0.016895999908447267,
357
+ 0.016903167724609376,
358
+ 0.016893951416015626,
359
+ 0.016902143478393555,
360
+ 0.016897024154663084,
361
+ 0.016897024154663084,
362
+ 0.016898048400878905,
363
+ 0.016894975662231446,
364
+ 0.016892927169799805,
365
+ 0.016911359786987306,
366
+ 0.016900096893310547,
367
+ 0.016903167724609376,
368
+ 0.016900096893310547,
369
+ 0.016899072647094726,
370
+ 0.016889055252075194,
371
+ 0.016909311294555664,
372
+ 0.016894975662231446,
373
+ 0.016900096893310547,
374
+ 0.016915456771850586,
375
+ 0.016903167724609376,
376
+ 0.016897024154663084,
377
+ 0.016905216217041014,
378
+ 0.016895999908447267,
379
+ 0.016899072647094726,
380
+ 0.016895200729370115,
381
+ 0.016895999908447267,
382
+ 0.016903167724609376,
383
+ 0.01690438461303711,
384
+ 0.016898048400878905,
385
+ 0.016899072647094726,
386
+ 0.016897024154663084,
387
+ 0.016905216217041014,
388
+ 0.016897024154663084,
389
+ 0.016899072647094726,
390
+ 0.016901119232177735,
391
+ 0.016901119232177735,
392
+ 0.016900096893310547,
393
+ 0.016961631774902345,
394
+ 0.016904191970825197,
395
+ 0.016893951416015626,
396
+ 0.016901119232177735,
397
+ 0.016903167724609376,
398
+ 0.016902143478393555,
399
+ 0.016900096893310547,
400
+ 0.016889856338500975,
401
+ 0.016898048400878905,
402
+ 0.016897056579589845,
403
+ 0.016901119232177735,
404
+ 0.016893951416015626,
405
+ 0.016910335540771485,
406
+ 0.016898048400878905,
407
+ 0.016902143478393555,
408
+ 0.016893951416015626,
409
+ 0.016902143478393555,
410
+ 0.016904191970825197,
411
+ 0.016906240463256835,
412
+ 0.016899072647094726,
413
+ 0.016889856338500975,
414
+ 0.016898048400878905,
415
+ 0.016909311294555664,
416
+ 0.016898048400878905,
417
+ 0.016902143478393555,
418
+ 0.016995328903198242,
419
+ 0.016894975662231446,
420
+ 0.016898048400878905,
421
+ 0.016892927169799805,
422
+ 0.016902336120605467,
423
+ 0.016900096893310547,
424
+ 0.016903167724609376,
425
+ 0.016894975662231446,
426
+ 0.016897024154663084,
427
+ 0.016903167724609376,
428
+ 0.016900320053100586,
429
+ 0.016903167724609376,
430
+ 0.016900096893310547,
431
+ 0.016897024154663084,
432
+ 0.016898048400878905,
433
+ 0.016904191970825197,
434
+ 0.016898048400878905,
435
+ 0.016898048400878905,
436
+ 0.016907264709472656,
437
+ 0.016902143478393555,
438
+ 0.016894975662231446,
439
+ 0.016890943527221678,
440
+ 0.016899072647094726,
441
+ 0.016892927169799805,
442
+ 0.016902143478393555,
443
+ 0.016902143478393555,
444
+ 0.016893951416015626,
445
+ 0.016912384033203123,
446
+ 0.016899072647094726,
447
+ 0.016901119232177735,
448
+ 0.016892927169799805,
449
+ 0.016901119232177735,
450
+ 0.016899072647094726,
451
+ 0.016905216217041014,
452
+ 0.016899072647094726,
453
+ 0.016902143478393555,
454
+ 0.016900096893310547,
455
+ 0.016905216217041014,
456
+ 0.016898048400878905,
457
+ 0.016894975662231446,
458
+ 0.016910335540771485,
459
+ 0.016907264709472656,
460
+ 0.016904191970825197,
461
+ 0.016901119232177735,
462
+ 0.016894975662231446,
463
+ 0.016901151657104492,
464
+ 0.016901119232177735,
465
+ 0.016903167724609376,
466
+ 0.01690025520324707,
467
+ 0.016917760848999024,
468
+ 0.016909311294555664,
469
+ 0.016897151947021485,
470
+ 0.016897024154663084,
471
+ 0.016899072647094726,
472
+ 0.016900096893310547,
473
+ 0.016907264709472656,
474
+ 0.016900096893310547,
475
+ 0.016905216217041014,
476
+ 0.016900096893310547,
477
+ 0.01690435218811035,
478
+ 0.016897024154663084,
479
+ 0.016902175903320313,
480
+ 0.016899072647094726,
481
+ 0.016907264709472656,
482
+ 0.016906240463256835,
483
+ 0.01689708709716797,
484
+ 0.016895999908447267,
485
+ 0.016904191970825197,
486
+ 0.016904191970825197,
487
+ 0.016903167724609376,
488
+ 0.016895999908447267,
489
+ 0.016900096893310547,
490
+ 0.016905216217041014,
491
+ 0.016903167724609376,
492
+ 0.016902143478393555,
493
+ 0.016892959594726562,
494
+ 0.016897024154663084,
495
+ 0.016906400680541993,
496
+ 0.016902143478393555,
497
+ 0.01689616012573242,
498
+ 0.016901119232177735,
499
+ 0.016899072647094726,
500
+ 0.016895999908447267,
501
+ 0.016899072647094726,
502
+ 0.016890880584716796,
503
+ 0.016899072647094726,
504
+ 0.016908287048339844,
505
+ 0.016902143478393555,
506
+ 0.016893951416015626,
507
+ 0.016904191970825197,
508
+ 0.016900096893310547,
509
+ 0.016902143478393555,
510
+ 0.016947200775146484,
511
+ 0.016904191970825197,
512
+ 0.016897024154663084,
513
+ 0.016900096893310547,
514
+ 0.016892927169799805,
515
+ 0.016907264709472656,
516
+ 0.016901119232177735,
517
+ 0.016899295806884766,
518
+ 0.016908287048339844,
519
+ 0.016893951416015626,
520
+ 0.016908287048339844,
521
+ 0.016894975662231446,
522
+ 0.016901248931884764,
523
+ 0.016906240463256835,
524
+ 0.016893951416015626,
525
+ 0.016906240463256835,
526
+ 0.016903167724609376,
527
+ 0.016900096893310547,
528
+ 0.016901119232177735,
529
+ 0.016907264709472656,
530
+ 0.016898048400878905,
531
+ 0.016903167724609376,
532
+ 0.016903167724609376,
533
+ 0.016902143478393555,
534
+ 0.01689811134338379,
535
+ 0.016902143478393555,
536
+ 0.016899072647094726,
537
+ 0.016898048400878905,
538
+ 0.016902143478393555,
539
+ 0.016901119232177735,
540
+ 0.016899072647094726,
541
+ 0.016899072647094726,
542
+ 0.016903167724609376,
543
+ 0.016899072647094726,
544
+ 0.016898048400878905,
545
+ 0.016899072647094726,
546
+ 0.016898048400878905,
547
+ 0.016898048400878905,
548
+ 0.016902143478393555,
549
+ 0.016907264709472656,
550
+ 0.016893951416015626,
551
+ 0.01690243148803711,
552
+ 0.016897024154663084,
553
+ 0.016904191970825197,
554
+ 0.016898048400878905,
555
+ 0.016898048400878905,
556
+ 0.016906240463256835,
557
+ 0.016906240463256835,
558
+ 0.016902143478393555,
559
+ 0.016898048400878905,
560
+ 0.016905216217041014,
561
+ 0.016895999908447267,
562
+ 0.016897024154663084,
563
+ 0.016900096893310547,
564
+ 0.016891904830932617,
565
+ 0.016900096893310547,
566
+ 0.016900096893310547,
567
+ 0.016899072647094726,
568
+ 0.016896095275878906,
569
+ 0.016897024154663084,
570
+ 0.016897024154663084,
571
+ 0.016898048400878905,
572
+ 0.016892927169799805,
573
+ 0.016899072647094726,
574
+ 0.01690012741088867,
575
+ 0.016906240463256835,
576
+ 0.016897024154663084,
577
+ 0.016907264709472656,
578
+ 0.016900096893310547,
579
+ 0.016901119232177735,
580
+ 0.016899072647094726,
581
+ 0.016891904830932617,
582
+ 0.016898048400878905,
583
+ 0.016898048400878905,
584
+ 0.016901119232177735,
585
+ 0.016892927169799805,
586
+ 0.016898048400878905,
587
+ 0.01690220832824707,
588
+ 0.016902463912963867,
589
+ 0.016898048400878905,
590
+ 0.016895999908447267,
591
+ 0.016900415420532226,
592
+ 0.016902143478393555,
593
+ 0.016902143478393555,
594
+ 0.016895999908447267,
595
+ 0.016897024154663084,
596
+ 0.016900096893310547,
597
+ 0.016901119232177735,
598
+ 0.016904191970825197,
599
+ 0.016903167724609376,
600
+ 0.016898048400878905,
601
+ 0.016906240463256835,
602
+ 0.016903167724609376,
603
+ 0.016893951416015626,
604
+ 0.016902143478393555,
605
+ 0.016905216217041014,
606
+ 0.016898048400878905
607
+ ]
608
+ },
609
+ "throughput": {
610
+ "unit": "tokens/s",
611
+ "value": 15142.67073447425
612
+ },
613
+ "energy": null,
614
+ "efficiency": null
615
+ },
616
+ "decode": {
617
+ "memory": {
618
+ "unit": "MB",
619
+ "max_ram": 946.46272,
620
+ "max_vram": 6683.623424,
621
+ "max_reserved": 6199.181312,
622
+ "max_allocated": 6066.59072
623
+ },
624
+ "latency": {
625
+ "unit": "s",
626
+ "mean": 3.9261285355885813,
627
+ "stdev": 0.0014551644341181862,
628
+ "values": [
629
+ 3.928184829711913,
630
+ 3.925171072006226,
631
+ 3.9250297050476046
632
+ ]
633
+ },
634
+ "throughput": {
635
+ "unit": "tokens/s",
636
+ "value": 64.94947826810565
637
+ },
638
+ "energy": null,
639
+ "efficiency": null
640
+ },
641
+ "per_token": {
642
+ "memory": null,
643
+ "latency": {
644
+ "unit": "s",
645
+ "mean": 0.015396582492504259,
646
+ "stdev": 7.388126008755291e-05,
647
+ "values": [
648
+ 0.015838208198547362,
649
+ 0.015426560401916504,
650
+ 0.015433728218078613,
651
+ 0.01539788818359375,
652
+ 0.01541222381591797,
653
+ 0.015426560401916504,
654
+ 0.015394047737121582,
655
+ 0.015382271766662597,
656
+ 0.015421440124511718,
657
+ 0.01542246437072754,
658
+ 0.015408127784729005,
659
+ 0.01540403175354004,
660
+ 0.015403008460998536,
661
+ 0.015409152030944824,
662
+ 0.015388671875,
663
+ 0.015409152030944824,
664
+ 0.015435775756835938,
665
+ 0.015450112342834473,
666
+ 0.01542246437072754,
667
+ 0.015393792152404785,
668
+ 0.015440896034240722,
669
+ 0.015435775756835938,
670
+ 0.015418368339538574,
671
+ 0.015450112342834473,
672
+ 0.015401984214782715,
673
+ 0.01582694435119629,
674
+ 0.015383551597595215,
675
+ 0.015450240135192871,
676
+ 0.015444864273071289,
677
+ 0.015408127784729005,
678
+ 0.015448063850402831,
679
+ 0.015430656433105469,
680
+ 0.015415295600891114,
681
+ 0.01539788818359375,
682
+ 0.015376383781433106,
683
+ 0.015405055999755859,
684
+ 0.015402175903320313,
685
+ 0.015389504432678223,
686
+ 0.015380640029907227,
687
+ 0.015428447723388672,
688
+ 0.015456255912780761,
689
+ 0.01546444797515869,
690
+ 0.015413248062133789,
691
+ 0.015431679725646973,
692
+ 0.015416319847106934,
693
+ 0.01541427230834961,
694
+ 0.01537945556640625,
695
+ 0.015407103538513184,
696
+ 0.01539583969116211,
697
+ 0.01558732795715332,
698
+ 0.015419391632080079,
699
+ 0.01589145565032959,
700
+ 0.015432703971862792,
701
+ 0.015433728218078613,
702
+ 0.015410176277160645,
703
+ 0.015437824249267578,
704
+ 0.015551487922668457,
705
+ 0.015446016311645508,
706
+ 0.015484928131103515,
707
+ 0.015433728218078613,
708
+ 0.015382528305053711,
709
+ 0.015409152030944824,
710
+ 0.01540608024597168,
711
+ 0.015458304405212403,
712
+ 0.015372287750244141,
713
+ 0.015430656433105469,
714
+ 0.01539788818359375,
715
+ 0.015442943572998047,
716
+ 0.015391743659973145,
717
+ 0.015405055999755859,
718
+ 0.015417344093322754,
719
+ 0.015420415878295898,
720
+ 0.015391743659973145,
721
+ 0.015458304405212403,
722
+ 0.015384575843811036,
723
+ 0.015421440124511718,
724
+ 0.01547980785369873,
725
+ 0.015440896034240722,
726
+ 0.015357952117919921,
727
+ 0.01548083209991455,
728
+ 0.01546444797515869,
729
+ 0.015462400436401368,
730
+ 0.015337471961975097,
731
+ 0.015418368339538574,
732
+ 0.015375359535217285,
733
+ 0.015429632186889648,
734
+ 0.01541222381591797,
735
+ 0.015432703971862792,
736
+ 0.01537331199645996,
737
+ 0.01537945556640625,
738
+ 0.015380479812622071,
739
+ 0.015386624336242676,
740
+ 0.015361023902893066,
741
+ 0.015408127784729005,
742
+ 0.015366144180297851,
743
+ 0.015419391632080079,
744
+ 0.01542246437072754,
745
+ 0.015429632186889648,
746
+ 0.015469568252563477,
747
+ 0.015435775756835938,
748
+ 0.015360159873962402,
749
+ 0.015358816146850586,
750
+ 0.015355903625488282,
751
+ 0.015484928131103515,
752
+ 0.015416319847106934,
753
+ 0.015444992065429687,
754
+ 0.01537331199645996,
755
+ 0.015380479812622071,
756
+ 0.015369215965270995,
757
+ 0.015366144180297851,
758
+ 0.015359999656677246,
759
+ 0.015384575843811036,
760
+ 0.015319040298461914,
761
+ 0.015409152030944824,
762
+ 0.015432703971862792,
763
+ 0.01539788818359375,
764
+ 0.015366144180297851,
765
+ 0.015561727523803711,
766
+ 0.015336447715759278,
767
+ 0.01536511993408203,
768
+ 0.015320063591003418,
769
+ 0.015411199569702149,
770
+ 0.015378432273864746,
771
+ 0.015433728218078613,
772
+ 0.015386624336242676,
773
+ 0.015762432098388672,
774
+ 0.015348735809326173,
775
+ 0.015315967559814453,
776
+ 0.015319040298461914,
777
+ 0.015375359535217285,
778
+ 0.015265791893005372,
779
+ 0.015401984214782715,
780
+ 0.015313920021057128,
781
+ 0.01538150405883789,
782
+ 0.01538764762878418,
783
+ 0.01541427230834961,
784
+ 0.015347711563110352,
785
+ 0.015323136329650879,
786
+ 0.015340543746948243,
787
+ 0.015393792152404785,
788
+ 0.015355903625488282,
789
+ 0.01539072036743164,
790
+ 0.015358976364135742,
791
+ 0.015385600090026855,
792
+ 0.015364095687866211,
793
+ 0.015633407592773436,
794
+ 0.015377408027648925,
795
+ 0.015408127784729005,
796
+ 0.015443967819213868,
797
+ 0.01538764762878418,
798
+ 0.01542246437072754,
799
+ 0.015416319847106934,
800
+ 0.015345664024353027,
801
+ 0.015411199569702149,
802
+ 0.01550438404083252,
803
+ 0.015425536155700683,
804
+ 0.015805439949035646,
805
+ 0.015378432273864746,
806
+ 0.015364224433898926,
807
+ 0.015387519836425782,
808
+ 0.015421440124511718,
809
+ 0.015450112342834473,
810
+ 0.015438847541809082,
811
+ 0.015458304405212403,
812
+ 0.015382528305053711,
813
+ 0.015485952377319336,
814
+ 0.015396863937377929,
815
+ 0.015410176277160645,
816
+ 0.015353856086730957,
817
+ 0.015409152030944824,
818
+ 0.015348735809326173,
819
+ 0.015430656433105469,
820
+ 0.015335424423217774,
821
+ 0.015451135635375977,
822
+ 0.015399935722351075,
823
+ 0.015447039604187012,
824
+ 0.015423487663269043,
825
+ 0.015408127784729005,
826
+ 0.015394816398620606,
827
+ 0.015428607940673827,
828
+ 0.015331328392028809,
829
+ 0.015388671875,
830
+ 0.015329279899597169,
831
+ 0.015424511909484863,
832
+ 0.015363072395324707,
833
+ 0.015410176277160645,
834
+ 0.015375359535217285,
835
+ 0.015678463935852052,
836
+ 0.015391743659973145,
837
+ 0.015423487663269043,
838
+ 0.01540608024597168,
839
+ 0.015418368339538574,
840
+ 0.015538175582885743,
841
+ 0.01537945556640625,
842
+ 0.015347711563110352,
843
+ 0.015420415878295898,
844
+ 0.01529958438873291,
845
+ 0.015321087837219239,
846
+ 0.015339520454406739,
847
+ 0.01540403175354004,
848
+ 0.015510527610778809,
849
+ 0.01540403175354004,
850
+ 0.015308799743652344,
851
+ 0.015416319847106934,
852
+ 0.015364095687866211,
853
+ 0.01529263973236084,
854
+ 0.015389472007751466,
855
+ 0.015315967559814453,
856
+ 0.01530470371246338,
857
+ 0.015301631927490235,
858
+ 0.015499263763427735,
859
+ 0.015341567993164062,
860
+ 0.015315967559814453,
861
+ 0.015384575843811036,
862
+ 0.01529651165008545,
863
+ 0.015380479812622071,
864
+ 0.015305727958679198,
865
+ 0.015361023902893066,
866
+ 0.015285247802734376,
867
+ 0.015352831840515137,
868
+ 0.015350784301757812,
869
+ 0.015396863937377929,
870
+ 0.015305727958679198,
871
+ 0.015489248275756836,
872
+ 0.015433504104614257,
873
+ 0.015325183868408204,
874
+ 0.015393792152404785,
875
+ 0.015357952117919921,
876
+ 0.015257599830627442,
877
+ 0.015334400177001953,
878
+ 0.015271936416625977,
879
+ 0.01529958438873291,
880
+ 0.015313920021057128,
881
+ 0.01540608024597168,
882
+ 0.01529958438873291,
883
+ 0.01541427230834961,
884
+ 0.01540403175354004,
885
+ 0.015328255653381348,
886
+ 0.01530675220489502,
887
+ 0.015431679725646973,
888
+ 0.015302656173706054,
889
+ 0.015392767906188964,
890
+ 0.01529958438873291,
891
+ 0.015392767906188964,
892
+ 0.015362048149108886,
893
+ 0.015448063850402831,
894
+ 0.015374336242675781,
895
+ 0.015339520454406739,
896
+ 0.01539583969116211,
897
+ 0.015320063591003418,
898
+ 0.015348735809326173,
899
+ 0.015357952117919921,
900
+ 0.015310848236083984,
901
+ 0.015352831840515137,
902
+ 0.015399935722351075,
903
+ 0.015844223976135254,
904
+ 0.015460351943969726,
905
+ 0.015351807594299317,
906
+ 0.01541427230834961,
907
+ 0.015439871788024903,
908
+ 0.01539788818359375,
909
+ 0.01538150405883789,
910
+ 0.015418368339538574,
911
+ 0.01538150405883789,
912
+ 0.015426560401916504,
913
+ 0.015353856086730957,
914
+ 0.015351807594299317,
915
+ 0.015350784301757812,
916
+ 0.015341567993164062,
917
+ 0.015324159622192383,
918
+ 0.01528115177154541,
919
+ 0.015322112083435058,
920
+ 0.015334400177001953,
921
+ 0.015366144180297851,
922
+ 0.015473664283752442,
923
+ 0.015393792152404785,
924
+ 0.01537945556640625,
925
+ 0.015380479812622071,
926
+ 0.015318016052246093,
927
+ 0.015315967559814453,
928
+ 0.01540608024597168,
929
+ 0.01529856014251709,
930
+ 0.015380479812622071,
931
+ 0.015368191719055176,
932
+ 0.015453184127807617,
933
+ 0.015316991806030274,
934
+ 0.015837183952331545,
935
+ 0.015351807594299317,
936
+ 0.015372287750244141,
937
+ 0.015333375930786132,
938
+ 0.015440896034240722,
939
+ 0.015334400177001953,
940
+ 0.015355903625488282,
941
+ 0.015334591865539551,
942
+ 0.015326016426086426,
943
+ 0.015335424423217774,
944
+ 0.015451135635375977,
945
+ 0.015370240211486816,
946
+ 0.015371264457702637,
947
+ 0.01536511993408203,
948
+ 0.015359999656677246,
949
+ 0.015410176277160645,
950
+ 0.01537945556640625,
951
+ 0.015320063591003418,
952
+ 0.01537331199645996,
953
+ 0.015372287750244141,
954
+ 0.015408127784729005,
955
+ 0.015385600090026855,
956
+ 0.015457280158996582,
957
+ 0.015405055999755859,
958
+ 0.015417344093322754,
959
+ 0.015392767906188964,
960
+ 0.015432703971862792,
961
+ 0.015368191719055176,
962
+ 0.015341567993164062,
963
+ 0.015400959968566894,
964
+ 0.015366144180297851,
965
+ 0.015386624336242676,
966
+ 0.015370240211486816,
967
+ 0.015411199569702149,
968
+ 0.01548799991607666,
969
+ 0.015424511909484863,
970
+ 0.015346688270568848,
971
+ 0.015447039604187012,
972
+ 0.015429632186889648,
973
+ 0.01539891242980957,
974
+ 0.015407103538513184,
975
+ 0.01539072036743164,
976
+ 0.015374336242675781,
977
+ 0.01539072036743164,
978
+ 0.015433728218078613,
979
+ 0.015428607940673827,
980
+ 0.015431679725646973,
981
+ 0.015419391632080079,
982
+ 0.015366144180297851,
983
+ 0.01537331199645996,
984
+ 0.015401984214782715,
985
+ 0.015347840309143067,
986
+ 0.015439743995666505,
987
+ 0.015476736068725586,
988
+ 0.015418368339538574,
989
+ 0.015361023902893066,
990
+ 0.015388671875,
991
+ 0.015375359535217285,
992
+ 0.015376383781433106,
993
+ 0.015370240211486816,
994
+ 0.015327232360839844,
995
+ 0.01539072036743164,
996
+ 0.015386624336242676,
997
+ 0.015351807594299317,
998
+ 0.015349760055541992,
999
+ 0.01538150405883789,
1000
+ 0.015354880332946777,
1001
+ 0.015358976364135742,
1002
+ 0.015378432273864746,
1003
+ 0.015358976364135742,
1004
+ 0.015408127784729005,
1005
+ 0.015326208114624023,
1006
+ 0.015393792152404785,
1007
+ 0.015384575843811036,
1008
+ 0.015380479812622071,
1009
+ 0.015338496208190918,
1010
+ 0.015376383781433106,
1011
+ 0.015386624336242676,
1012
+ 0.01541427230834961,
1013
+ 0.015370240211486816,
1014
+ 0.015321087837219239,
1015
+ 0.015424511909484863,
1016
+ 0.015362048149108886,
1017
+ 0.015331328392028809,
1018
+ 0.015401984214782715,
1019
+ 0.015413248062133789,
1020
+ 0.015331328392028809,
1021
+ 0.015367168426513672,
1022
+ 0.015372287750244141,
1023
+ 0.015411199569702149,
1024
+ 0.015380479812622071,
1025
+ 0.015321087837219239,
1026
+ 0.015457280158996582,
1027
+ 0.015327232360839844,
1028
+ 0.015411199569702149,
1029
+ 0.015389727592468263,
1030
+ 0.015325152397155761,
1031
+ 0.015382528305053711,
1032
+ 0.015417344093322754,
1033
+ 0.015342592239379883,
1034
+ 0.015936511993408203,
1035
+ 0.015446016311645508,
1036
+ 0.015419391632080079,
1037
+ 0.01541427230834961,
1038
+ 0.01540403175354004,
1039
+ 0.015376383781433106,
1040
+ 0.015441920280456543,
1041
+ 0.015375359535217285,
1042
+ 0.01548902416229248,
1043
+ 0.01539891242980957,
1044
+ 0.015393792152404785,
1045
+ 0.015386624336242676,
1046
+ 0.015405055999755859,
1047
+ 0.015344639778137208,
1048
+ 0.015441920280456543,
1049
+ 0.015411199569702149,
1050
+ 0.015432703971862792,
1051
+ 0.015375359535217285,
1052
+ 0.015429632186889648,
1053
+ 0.015319040298461914,
1054
+ 0.015393792152404785,
1055
+ 0.015408127784729005,
1056
+ 0.01539072036743164,
1057
+ 0.015396863937377929,
1058
+ 0.015384575843811036,
1059
+ 0.015366144180297851,
1060
+ 0.015481856346130371,
1061
+ 0.015491071701049805,
1062
+ 0.015524864196777344,
1063
+ 0.015455231666564942,
1064
+ 0.015454208374023438,
1065
+ 0.015820799827575685,
1066
+ 0.015369215965270995,
1067
+ 0.015369215965270995,
1068
+ 0.015473664283752442,
1069
+ 0.01538150405883789,
1070
+ 0.015417344093322754,
1071
+ 0.015382528305053711,
1072
+ 0.015424511909484863,
1073
+ 0.015378432273864746,
1074
+ 0.015427583694458008,
1075
+ 0.015347711563110352,
1076
+ 0.015383551597595215,
1077
+ 0.015359999656677246,
1078
+ 0.015440896034240722,
1079
+ 0.015427583694458008,
1080
+ 0.015419391632080079,
1081
+ 0.015396863937377929,
1082
+ 0.015388671875,
1083
+ 0.015364383697509766,
1084
+ 0.015505120277404786,
1085
+ 0.015410176277160645,
1086
+ 0.015423487663269043,
1087
+ 0.015420415878295898,
1088
+ 0.015415295600891114,
1089
+ 0.015318016052246093,
1090
+ 0.015450112342834473,
1091
+ 0.015427583694458008,
1092
+ 0.015435775756835938,
1093
+ 0.015374336242675781,
1094
+ 0.015433728218078613,
1095
+ 0.015419391632080079,
1096
+ 0.01541427230834961,
1097
+ 0.015372287750244141,
1098
+ 0.015355903625488282,
1099
+ 0.015386624336242676,
1100
+ 0.015363072395324707,
1101
+ 0.015411199569702149,
1102
+ 0.015375359535217285,
1103
+ 0.015376383781433106,
1104
+ 0.015349760055541992,
1105
+ 0.015368191719055176,
1106
+ 0.015303680419921875,
1107
+ 0.015364095687866211,
1108
+ 0.015343615531921387,
1109
+ 0.01542246437072754,
1110
+ 0.015428607940673827,
1111
+ 0.015348735809326173,
1112
+ 0.015409152030944824,
1113
+ 0.015341567993164062,
1114
+ 0.015370240211486816,
1115
+ 0.015319040298461914,
1116
+ 0.01537331199645996,
1117
+ 0.015438847541809082,
1118
+ 0.015378432273864746,
1119
+ 0.01528320026397705,
1120
+ 0.01538764762878418,
1121
+ 0.015385600090026855,
1122
+ 0.01541222381591797,
1123
+ 0.01533033561706543,
1124
+ 0.015327296257019042,
1125
+ 0.015324064254760742,
1126
+ 0.015357952117919921,
1127
+ 0.015302656173706054,
1128
+ 0.015400959968566894,
1129
+ 0.015310848236083984,
1130
+ 0.015416319847106934,
1131
+ 0.015263744354248047,
1132
+ 0.015369215965270995,
1133
+ 0.01542246437072754,
1134
+ 0.015428607940673827,
1135
+ 0.015375359535217285,
1136
+ 0.01540403175354004,
1137
+ 0.015315967559814453,
1138
+ 0.015346688270568848,
1139
+ 0.015318016052246093,
1140
+ 0.015405055999755859,
1141
+ 0.01532323169708252,
1142
+ 0.015432607650756837,
1143
+ 0.015356927871704102,
1144
+ 0.015371264457702637,
1145
+ 0.015423487663269043,
1146
+ 0.015453184127807617,
1147
+ 0.015415295600891114,
1148
+ 0.015405055999755859,
1149
+ 0.015349760055541992,
1150
+ 0.015356927871704102,
1151
+ 0.015313920021057128,
1152
+ 0.015444992065429687,
1153
+ 0.015301631927490235,
1154
+ 0.015421440124511718,
1155
+ 0.015359999656677246,
1156
+ 0.015388671875,
1157
+ 0.015371264457702637,
1158
+ 0.01553388786315918,
1159
+ 0.015427583694458008,
1160
+ 0.015382528305053711,
1161
+ 0.015417344093322754,
1162
+ 0.015361023902893066,
1163
+ 0.015567872047424316,
1164
+ 0.01561190414428711,
1165
+ 0.015433728218078613,
1166
+ 0.015349760055541992,
1167
+ 0.015453184127807617,
1168
+ 0.015341567993164062,
1169
+ 0.015326208114624023,
1170
+ 0.015393792152404785,
1171
+ 0.01527500820159912,
1172
+ 0.015456255912780761,
1173
+ 0.015372287750244141,
1174
+ 0.015343615531921387,
1175
+ 0.015356927871704102,
1176
+ 0.01528217601776123,
1177
+ 0.015459327697753907,
1178
+ 0.015366144180297851,
1179
+ 0.015399935722351075,
1180
+ 0.015318016052246093,
1181
+ 0.015408127784729005,
1182
+ 0.01530675220489502,
1183
+ 0.015343615531921387,
1184
+ 0.015343615531921387,
1185
+ 0.015401984214782715,
1186
+ 0.015432703971862792,
1187
+ 0.015408127784729005,
1188
+ 0.015410176277160645,
1189
+ 0.015408127784729005,
1190
+ 0.015386624336242676,
1191
+ 0.015393792152404785,
1192
+ 0.015384575843811036,
1193
+ 0.015369215965270995,
1194
+ 0.015364095687866211,
1195
+ 0.01571225643157959,
1196
+ 0.015323360443115235,
1197
+ 0.015349535942077637,
1198
+ 0.015353856086730957,
1199
+ 0.015363072395324707,
1200
+ 0.015370240211486816,
1201
+ 0.01537331199645996,
1202
+ 0.015330304145812988,
1203
+ 0.015378432273864746,
1204
+ 0.015372384071350097,
1205
+ 0.015324064254760742,
1206
+ 0.015339520454406739,
1207
+ 0.015337471961975097,
1208
+ 0.015341567993164062,
1209
+ 0.015407103538513184,
1210
+ 0.015377408027648925,
1211
+ 0.015453184127807617,
1212
+ 0.015367168426513672,
1213
+ 0.015396863937377929,
1214
+ 0.015341567993164062,
1215
+ 0.015392767906188964,
1216
+ 0.015349760055541992,
1217
+ 0.015361023902893066,
1218
+ 0.015352831840515137,
1219
+ 0.015337471961975097,
1220
+ 0.01530777645111084,
1221
+ 0.015321087837219239,
1222
+ 0.015419391632080079,
1223
+ 0.015344639778137208,
1224
+ 0.015353856086730957,
1225
+ 0.01530675220489502,
1226
+ 0.015411199569702149,
1227
+ 0.015380479812622071,
1228
+ 0.015363072395324707,
1229
+ 0.015396863937377929,
1230
+ 0.015388671875,
1231
+ 0.015362048149108886,
1232
+ 0.015361023902893066,
1233
+ 0.01539072036743164,
1234
+ 0.015396863937377929,
1235
+ 0.01538150405883789,
1236
+ 0.015354880332946777,
1237
+ 0.015295488357543945,
1238
+ 0.015335424423217774,
1239
+ 0.015396863937377929,
1240
+ 0.015331328392028809,
1241
+ 0.015353856086730957,
1242
+ 0.015380479812622071,
1243
+ 0.015378432273864746,
1244
+ 0.015362048149108886,
1245
+ 0.015355903625488282,
1246
+ 0.015420415878295898,
1247
+ 0.015396863937377929,
1248
+ 0.01540608024597168,
1249
+ 0.015340543746948243,
1250
+ 0.015319040298461914,
1251
+ 0.01539583969116211,
1252
+ 0.01529958438873291,
1253
+ 0.015428607940673827,
1254
+ 0.015431679725646973,
1255
+ 0.01568051242828369,
1256
+ 0.01537945556640625,
1257
+ 0.015348735809326173,
1258
+ 0.015363072395324707,
1259
+ 0.015424511909484863,
1260
+ 0.01537331199645996,
1261
+ 0.015415295600891114,
1262
+ 0.015418368339538574,
1263
+ 0.01541222381591797,
1264
+ 0.015407103538513184,
1265
+ 0.015355903625488282,
1266
+ 0.015354111671447753,
1267
+ 0.015386367797851562,
1268
+ 0.01538150405883789,
1269
+ 0.015417344093322754,
1270
+ 0.015426560401916504,
1271
+ 0.015425536155700683,
1272
+ 0.01539583969116211,
1273
+ 0.015419391632080079,
1274
+ 0.015454208374023438,
1275
+ 0.01539072036743164,
1276
+ 0.015372287750244141,
1277
+ 0.015392767906188964,
1278
+ 0.01539072036743164,
1279
+ 0.015388671875,
1280
+ 0.01530777645111084,
1281
+ 0.01542246437072754,
1282
+ 0.015423487663269043,
1283
+ 0.015405055999755859,
1284
+ 0.015407103538513184,
1285
+ 0.015403008460998536,
1286
+ 0.015493120193481445,
1287
+ 0.015407103538513184,
1288
+ 0.015434752464294434,
1289
+ 0.015394816398620606,
1290
+ 0.01547264003753662,
1291
+ 0.015453184127807617,
1292
+ 0.015428607940673827,
1293
+ 0.015392767906188964,
1294
+ 0.015501312255859375,
1295
+ 0.015721471786499023,
1296
+ 0.015392767906188964,
1297
+ 0.015416319847106934,
1298
+ 0.015403008460998536,
1299
+ 0.015357952117919921,
1300
+ 0.015355903625488282,
1301
+ 0.01540608024597168,
1302
+ 0.015368191719055176,
1303
+ 0.015467519760131837,
1304
+ 0.01539891242980957,
1305
+ 0.015369215965270995,
1306
+ 0.015316991806030274,
1307
+ 0.015366144180297851,
1308
+ 0.01536511993408203,
1309
+ 0.015366144180297851,
1310
+ 0.015429632186889648,
1311
+ 0.015458304405212403,
1312
+ 0.015367168426513672,
1313
+ 0.015394816398620606,
1314
+ 0.01529139232635498,
1315
+ 0.015429632186889648,
1316
+ 0.01539583969116211,
1317
+ 0.015418368339538574,
1318
+ 0.015371264457702637,
1319
+ 0.015394816398620606,
1320
+ 0.015394816398620606,
1321
+ 0.015471615791320802,
1322
+ 0.015442943572998047,
1323
+ 0.01548083209991455,
1324
+ 0.015363072395324707,
1325
+ 0.015423487663269043,
1326
+ 0.01580851173400879,
1327
+ 0.015469568252563477,
1328
+ 0.015376383781433106,
1329
+ 0.015401984214782715,
1330
+ 0.015322239875793457,
1331
+ 0.015380352020263673,
1332
+ 0.01531289577484131,
1333
+ 0.015420415878295898,
1334
+ 0.015408127784729005,
1335
+ 0.015403008460998536,
1336
+ 0.015353856086730957,
1337
+ 0.01540608024597168,
1338
+ 0.015354880332946777,
1339
+ 0.015441920280456543,
1340
+ 0.015466496467590332,
1341
+ 0.015438847541809082,
1342
+ 0.015374336242675781,
1343
+ 0.015413248062133789,
1344
+ 0.015336447715759278,
1345
+ 0.015405055999755859,
1346
+ 0.015433728218078613,
1347
+ 0.015436800003051757,
1348
+ 0.015359999656677246,
1349
+ 0.01542246437072754,
1350
+ 0.015442943572998047,
1351
+ 0.015363072395324707,
1352
+ 0.015374336242675781,
1353
+ 0.015410176277160645,
1354
+ 0.015383551597595215,
1355
+ 0.015439871788024903,
1356
+ 0.015391743659973145,
1357
+ 0.01538969612121582,
1358
+ 0.015370240211486816,
1359
+ 0.015337471961975097,
1360
+ 0.015356927871704102,
1361
+ 0.015403008460998536,
1362
+ 0.015428607940673827,
1363
+ 0.01538969612121582,
1364
+ 0.015366144180297851,
1365
+ 0.015446016311645508,
1366
+ 0.015380479812622071,
1367
+ 0.015378432273864746,
1368
+ 0.01538969612121582,
1369
+ 0.015413248062133789,
1370
+ 0.01530675220489502,
1371
+ 0.015346688270568848,
1372
+ 0.015295488357543945,
1373
+ 0.015410176277160645,
1374
+ 0.015341567993164062,
1375
+ 0.015377408027648925,
1376
+ 0.015500288009643554,
1377
+ 0.015458304405212403,
1378
+ 0.01538150405883789,
1379
+ 0.015429632186889648,
1380
+ 0.015332351684570313,
1381
+ 0.015384575843811036,
1382
+ 0.015335424423217774,
1383
+ 0.01541427230834961,
1384
+ 0.01530470371246338,
1385
+ 0.015482879638671876,
1386
+ 0.015359999656677246,
1387
+ 0.015394816398620606,
1388
+ 0.015416319847106934,
1389
+ 0.015468544006347656,
1390
+ 0.015344639778137208,
1391
+ 0.01541222381591797,
1392
+ 0.015333375930786132,
1393
+ 0.015391743659973145,
1394
+ 0.015314944267272949,
1395
+ 0.015401984214782715,
1396
+ 0.015299839973449707,
1397
+ 0.015401727676391601,
1398
+ 0.015330304145812988,
1399
+ 0.01530777645111084,
1400
+ 0.015411199569702149,
1401
+ 0.015440896034240722,
1402
+ 0.015355968475341797,
1403
+ 0.01537939167022705,
1404
+ 0.015386624336242676,
1405
+ 0.015394816398620606,
1406
+ 0.015352831840515137,
1407
+ 0.015434752464294434,
1408
+ 0.01528831958770752,
1409
+ 0.01540403175354004,
1410
+ 0.015314944267272949,
1411
+ 0.015418368339538574,
1412
+ 0.015375359535217285
1413
+ ]
1414
+ },
1415
+ "throughput": {
1416
+ "unit": "tokens/s",
1417
+ "value": 64.94947826810557
1418
+ },
1419
+ "energy": null,
1420
+ "efficiency": null
1421
+ }
1422
+ }