---
base_model: roneneldan/TinyStories-33M
library_name: Distily
tags:
- generated_from_trainer
model-index:
- name: distily_bench_obj_cross_v2.2
  results: []
---

# distily_bench_obj_cross_v2.2

This student model is distilled from the teacher model [roneneldan/TinyStories-33M](https://huggingface.co/roneneldan/TinyStories-33M) using the dataset (unspecified).

The [Distily](https://github.com/lapp0/distily) library was used for this distillation.

It achieves the following results on the evaluation set:
- eval_enwikippl: 211.4068
- eval_frwikippl: 98721.6094
- eval_zhwikippl: 3163838.5
- eval_tinystoriesppl: 10.2269
- eval_loss: 1.1274
- eval_runtime: 13.054
- eval_samples_per_second: 76.605
- eval_steps_per_second: 9.576

<!-- This model card has been generated automatically according to the information the Trainer had access to. You
should probably proofread and complete it, then remove this comment.

## Model description

More information needed

## Intended uses & limitations

More information needed

## Training and evaluation data

More information needed
-->

## Training procedure

### Training hyperparameters

The following hyperparameters were used during training:
- distillation_objective: DistillationObjective(logits_loss_component=LossComponent(label=logits, weight=1, loss_fn=kl, layer_mapper=None, projector=None), hs_loss_component=LossComponent(label=hs, weight=0, loss_fn=None, layer_mapper=None, projector=None), attn_loss_component=LossComponent(label=attn, weight=0, loss_fn=None, layer_mapper=None, projector=None))
- train_embeddings: True
- learning_rate: 0.04
- train_batch_size: 1
- eval_batch_size: 8
- seed: 42
- optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
- lr_scheduler_type: linear
- lr_scheduler_warmup_ratio: 0.5
- num_epochs: 1.0

### Resource Usage
Peak GPU Memory: 6.6047 GB

### Eval-Phase Metrics
| step | epoch | enwikippl | frwikippl | loss | runtime | samples_per_second | steps_per_second | tinystoriesppl | zhwikippl |
| --- | --- | --- | --- | --- | --- | --- | --- | --- | --- |
| **teacher eval** |  | 169.9865 | 47377.9414 |  |  |  |  | 3.9789 | 4998.1294 |
| 0 | 0 | 41695.9688 | 72257.4141 | 6.4315 | 13.1139 | 76.255 | 9.532 | 30243.7324 | 65683.7969 |
| 500 | 0.0051 | 9398.1553 | 40012.7852 | 5.1658 | 13.0965 | 76.356 | 9.545 | 4541.5229 | 57404.3438 |
| 1000 | 0.0101 | 1576.9297 | 23847.7656 | 3.8240 | 13.1411 | 76.097 | 9.512 | 417.6039 | 55078.9023 |
| 1500 | 0.0152 | 183.6923 | 30533.4805 | 2.7615 | 13.0992 | 76.341 | 9.543 | 14.4326 | 115298.7656 |
| 2000 | 0.0202 | 204.5679 | 21005.3770 | 2.4622 | 13.1229 | 76.203 | 9.525 | 19.6786 | 70364.2656 |
| 2500 | 0.0253 | 202.7026 | 126007.1484 | 2.9102 | 13.1226 | 76.205 | 9.526 | 7.6228 | 946041.0625 |
| 3000 | 0.0303 | 187.8947 | 38023.6719 | 1.6731 | 13.2404 | 75.527 | 9.441 | 11.9361 | 339606.3125 |
| 3500 | 0.0354 | 193.8230 | 55170.2188 | 1.4714 | 13.1096 | 76.28 | 9.535 | 11.2473 | 629307.9375 |
| 4000 | 0.0404 | 164.6342 | 44408.5859 | 1.3087 | 13.1496 | 76.048 | 9.506 | 10.7545 | 255472.375 |
| 4500 | 0.0455 | 157.0112 | 36630.5 | 1.2572 | 13.0578 | 76.583 | 9.573 | 10.3267 | 215947.0 |
| 5000 | 0.0505 | 281.8606 | 185762.6406 | 1.3042 | 13.185 | 75.844 | 9.48 | 11.0474 | 1601895.75 |
| 5500 | 0.0556 | 207.0872 | 94590.1641 | 1.1927 | 13.1207 | 76.215 | 9.527 | 10.3344 | 1017518.25 |
| 6000 | 0.0606 | 182.3242 | 66102.9375 | 1.1691 | 13.0788 | 76.46 | 9.557 | 10.1046 | 696083.6875 |
| 6500 | 0.0657 | 184.3266 | 63198.4570 | 1.1648 | 13.1174 | 76.235 | 9.529 | 10.3327 | 661148.25 |
| 7000 | 0.0707 | 182.6494 | 57815.5781 | 1.1628 | 13.0897 | 76.396 | 9.55 | 10.4258 | 566363.375 |
| 7500 | 0.0758 | 185.6090 | 56822.5273 | 1.1643 | 13.0989 | 76.343 | 9.543 | 10.8554 | 611107.75 |
| 8000 | 0.0808 | 188.3611 | 58598.5430 | 1.1638 | 13.176 | 75.896 | 9.487 | 10.9863 | 637418.625 |
| 8500 | 0.0859 | 179.5004 | 58784.5938 | 1.1624 | 13.1138 | 76.255 | 9.532 | 10.3263 | 592642.6875 |
| 9000 | 0.0909 | 192.1636 | 60255.8711 | 1.1641 | 13.0772 | 76.469 | 9.559 | 11.0086 | 697199.125 |
| 9500 | 0.0960 | 183.8703 | 58101.3398 | 1.1629 | 13.1072 | 76.294 | 9.537 | 10.7044 | 526016.625 |
| 10000 | 0.1010 | 223.6976 | 103754.3984 | 1.1601 | 13.1393 | 76.108 | 9.513 | 11.2120 | 2202829.5 |
| 10500 | 0.1061 | 209.6536 | 85775.0703 | 1.1473 | 13.0911 | 76.388 | 9.548 | 11.0163 | 1419534.0 |
| 11000 | 0.1111 | 199.6753 | 72527.6328 | 1.1460 | 13.1035 | 76.315 | 9.539 | 10.9478 | 952626.5625 |
| 11500 | 0.1162 | 195.6104 | 82812.9609 | 1.1439 | 13.0724 | 76.497 | 9.562 | 10.1503 | 1125486.875 |
| 12000 | 0.1212 | 176.4672 | 71826.1562 | 1.1464 | 13.2355 | 75.554 | 9.444 | 9.2515 | 966450.5625 |
| 12500 | 0.1263 | 174.9496 | 70681.9844 | 1.1453 | 13.0726 | 76.496 | 9.562 | 9.3222 | 933007.125 |
| 13000 | 0.1313 | 175.7033 | 74071.1172 | 1.1471 | 13.1948 | 75.787 | 9.473 | 9.1807 | 1092935.75 |
| 13500 | 0.1364 | 204.2988 | 78746.0156 | 1.1453 | 13.1815 | 75.864 | 9.483 | 11.2408 | 1120094.5 |
| 14000 | 0.1414 | 204.0140 | 81499.5234 | 1.1455 | 13.1274 | 76.177 | 9.522 | 10.8465 | 1272787.25 |
| 14500 | 0.1465 | 204.5838 | 74065.8906 | 1.1463 | 13.0803 | 76.451 | 9.556 | 11.4936 | 1088280.25 |
| 15000 | 0.1515 | 176.1736 | 78281.5625 | 1.1485 | 13.1271 | 76.178 | 9.522 | 8.9458 | 1139080.75 |
| 15500 | 0.1566 | 211.8905 | 76133.8125 | 1.1489 | 13.0891 | 76.399 | 9.55 | 11.8188 | 1199590.625 |
| 16000 | 0.1616 | 178.4536 | 78940.3906 | 1.1475 | 13.18 | 75.873 | 9.484 | 9.0822 | 1065867.0 |
| 16500 | 0.1667 | 207.0069 | 74563.125 | 1.1491 | 13.1876 | 75.829 | 9.479 | 11.6929 | 1061893.875 |
| 17000 | 0.1717 | 221.3361 | 75095.4141 | 1.1525 | 13.1785 | 75.881 | 9.485 | 12.6257 | 1094979.5 |
| 17500 | 0.1768 | 225.7255 | 80978.8906 | 1.1535 | 13.1832 | 75.854 | 9.482 | 12.6812 | 1185590.125 |
| 18000 | 0.1818 | 177.9153 | 73966.8594 | 1.1487 | 13.1725 | 75.916 | 9.489 | 9.2385 | 1019963.6875 |
| 18500 | 0.1869 | 174.8547 | 75978.5156 | 1.1517 | 13.0907 | 76.39 | 9.549 | 8.7911 | 1154993.375 |
| 19000 | 0.1919 | 228.3903 | 82312.8984 | 1.1527 | 13.1276 | 76.175 | 9.522 | 12.6576 | 1374806.25 |
| 19500 | 0.1970 | 180.7352 | 90230.4219 | 1.1446 | 13.1317 | 76.152 | 9.519 | 8.7570 | 2063998.0 |
| 20000 | 0.2020 | 219.0843 | 86722.625 | 1.1396 | 13.1423 | 76.09 | 9.511 | 11.5364 | 1853592.625 |
| 20500 | 0.2071 | 184.7913 | 91227.1875 | 1.1417 | 13.0914 | 76.386 | 9.548 | 8.8290 | 2247352.0 |
| 21000 | 0.2121 | 180.4415 | 91073.1562 | 1.1426 | 13.1748 | 75.902 | 9.488 | 8.7346 | 1853592.625 |
| 21500 | 0.2172 | 236.0438 | 85425.4297 | 1.1480 | 13.0492 | 76.633 | 9.579 | 13.1907 | 1922595.75 |
| 22000 | 0.2222 | 175.1326 | 81235.9219 | 1.1438 | 13.1679 | 75.942 | 9.493 | 8.6206 | 1660643.5 |
| 22500 | 0.2273 | 168.5344 | 81574.1719 | 1.1496 | 13.152 | 76.034 | 9.504 | 8.1520 | 1535772.375 |
| 23000 | 0.2323 | 171.4180 | 78962.6719 | 1.1458 | 13.1153 | 76.247 | 9.531 | 8.5203 | 1804792.625 |
| 23500 | 0.2374 | 234.5673 | 89452.1172 | 1.1476 | 13.1717 | 75.92 | 9.49 | 12.9438 | 1814933.0 |
| 24000 | 0.2424 | 233.2717 | 86223.2891 | 1.1468 | 13.0218 | 76.794 | 9.599 | 12.9776 | 1961978.25 |
| 24500 | 0.2475 | 228.7444 | 77039.9922 | 1.1510 | 13.1092 | 76.282 | 9.535 | 13.3746 | 1449771.25 |
| 25000 | 0.2525 | 178.7338 | 93484.1406 | 1.1490 | 13.1002 | 76.335 | 9.542 | 8.2698 | 1880489.125 |
| 25500 | 0.2576 | 170.9340 | 83987.7031 | 1.1507 | 13.0962 | 76.358 | 9.545 | 8.3274 | 1677565.625 |
| 26000 | 0.2626 | 256.8503 | 95158.1016 | 1.1600 | 13.0788 | 76.46 | 9.557 | 14.7243 | 2264807.5 |
| 26500 | 0.2677 | 178.9000 | 88911.9609 | 1.1467 | 13.1335 | 76.141 | 9.518 | 8.4408 | 1761972.875 |
| 27000 | 0.2727 | 171.4977 | 85371.2734 | 1.1546 | 13.1169 | 76.237 | 9.53 | 7.9949 | 1851615.625 |
| 27500 | 0.2778 | 177.9877 | 88949.5312 | 1.1462 | 13.0336 | 76.725 | 9.591 | 8.5073 | 1721086.125 |
| 28000 | 0.2828 | 170.3722 | 81161.5859 | 1.1520 | 13.1387 | 76.111 | 9.514 | 8.2340 | 1483420.0 |
| 28500 | 0.2879 | 253.1172 | 86393.5 | 1.1587 | 13.1803 | 75.871 | 9.484 | 14.4207 | 1973526.75 |
| 29000 | 0.2929 | 173.2066 | 91169.4375 | 1.1538 | 13.2095 | 75.703 | 9.463 | 8.1500 | 2046450.0 |
| 29500 | 0.2980 | 191.0282 | 85233.0547 | 1.1414 | 13.1479 | 76.058 | 9.507 | 9.6212 | 1736769.75 |
| 30000 | 0.3030 | 247.9741 | 82359.3047 | 1.1591 | 13.1167 | 76.239 | 9.53 | 14.4088 | 1651365.25 |
| 30500 | 0.3081 | 174.2091 | 83721.9609 | 1.1488 | 13.1075 | 76.292 | 9.537 | 8.4722 | 1796145.5 |
| 31000 | 0.3131 | 256.3036 | 82545.1094 | 1.1628 | 13.1148 | 76.25 | 9.531 | 14.6478 | 1717875.375 |
| 31500 | 0.3182 | 166.3327 | 82993.9375 | 1.1588 | 13.0891 | 76.399 | 9.55 | 7.8383 | 1914917.625 |
| 32000 | 0.3232 | 258.3770 | 79403.2266 | 1.1715 | 13.0466 | 76.648 | 9.581 | 15.7863 | 1586584.125 |
| 32500 | 0.3283 | 254.8189 | 82040.875 | 1.1619 | 13.1276 | 76.175 | 9.522 | 14.7914 | 1818810.75 |
| 33000 | 0.3333 | 168.8546 | 81173.0391 | 1.1567 | 13.1706 | 75.927 | 9.491 | 7.9622 | 1924136.5 |
| 33500 | 0.3384 | 273.0810 | 92867.2812 | 1.1720 | 13.198 | 75.769 | 9.471 | 15.8851 | 2072827.25 |
| 34000 | 0.3434 | 253.8929 | 89704.4688 | 1.1640 | 13.1463 | 76.067 | 9.508 | 14.8164 | 2015564.5 |
| 34500 | 0.3485 | 251.9336 | 80956.0312 | 1.1644 | 13.1328 | 76.145 | 9.518 | 14.6878 | 1758685.75 |
| 35000 | 0.3535 | 169.4311 | 85781.125 | 1.1570 | 13.1502 | 76.044 | 9.506 | 8.0027 | 1789450.25 |
| 35500 | 0.3586 | 157.4924 | 79487.0938 | 1.1681 | 13.2104 | 75.698 | 9.462 | 7.5228 | 1609178.0 |
| 36000 | 0.3636 | 270.5542 | 94370.5781 | 1.1723 | 13.2102 | 75.699 | 9.462 | 15.8759 | 2226463.0 |
| 36500 | 0.3687 | 272.3416 | 89200.4688 | 1.1731 | 13.1949 | 75.787 | 9.473 | 15.9173 | 1956227.375 |
| 37000 | 0.3737 | 171.2521 | 81781.2969 | 1.1552 | 13.2712 | 75.351 | 9.419 | 8.1403 | 1847667.875 |
| 37500 | 0.3788 | 287.3947 | 93438.0625 | 1.1844 | 13.0886 | 76.402 | 9.55 | 17.2819 | 1739088.5 |
| 38000 | 0.3838 | 276.8832 | 84100.1484 | 1.1787 | 13.1837 | 75.851 | 9.481 | 16.5952 | 1716043.125 |
| 38500 | 0.3889 | 169.9931 | 90880.8906 | 1.1645 | 13.0691 | 76.517 | 9.565 | 7.7999 | 1959362.25 |
| 39000 | 0.3939 | 200.5821 | 117644.8359 | 1.1479 | 13.1454 | 76.072 | 9.509 | 8.9082 | 4717038.0 |
| 39500 | 0.3990 | 273.4832 | 116827.4375 | 1.1610 | 13.1758 | 75.897 | 9.487 | 14.4750 | 4494663.5 |
| 40000 | 0.4040 | 164.3157 | 101922.0234 | 1.1859 | 13.1475 | 76.06 | 9.508 | 7.0509 | 3107785.25 |
| 40500 | 0.4091 | 295.5222 | 116926.1953 | 1.1712 | 13.1169 | 76.238 | 9.53 | 16.0581 | 3700232.25 |
| 41000 | 0.4141 | 172.2733 | 94417.125 | 1.1599 | 13.0682 | 76.522 | 9.565 | 7.7084 | 2915028.75 |
| 41500 | 0.4192 | 304.5276 | 117702.8594 | 1.1776 | 13.1146 | 76.251 | 9.531 | 16.9019 | 4569630.0 |
| 42000 | 0.4242 | 303.2448 | 107488.2031 | 1.1825 | 13.0434 | 76.667 | 9.583 | 17.1780 | 3488367.0 |
| 42500 | 0.4293 | 299.4282 | 112467.7344 | 1.1756 | 13.1071 | 76.295 | 9.537 | 16.4355 | 3977674.75 |
| 43000 | 0.4343 | 301.7801 | 102080.0 | 1.1815 | 13.1164 | 76.24 | 9.53 | 17.2584 | 3297416.5 |
| 43500 | 0.4394 | 307.5147 | 116876.8047 | 1.1816 | 13.1561 | 76.01 | 9.501 | 17.1922 | 4009640.75 |
| 44000 | 0.4444 | 308.2659 | 100659.3125 | 1.1890 | 13.0906 | 76.391 | 9.549 | 17.9555 | 2741531.0 |
| 44500 | 0.4495 | 319.4117 | 97498.5391 | 1.2073 | 13.0736 | 76.49 | 9.561 | 19.2204 | 3212319.75 |
| 45000 | 0.4545 | 308.3376 | 99342.2969 | 1.1906 | 13.0178 | 76.818 | 9.602 | 18.3554 | 3002646.75 |
| 45500 | 0.4596 | 174.9156 | 104878.4531 | 1.1724 | 13.1246 | 76.193 | 9.524 | 7.3603 | 3698260.25 |
| 46000 | 0.4646 | 332.8888 | 111442.7578 | 1.2029 | 13.1599 | 75.989 | 9.499 | 19.8723 | 3781065.5 |
| 46500 | 0.4697 | 305.6621 | 104583.4062 | 1.1797 | 13.0469 | 76.647 | 9.581 | 17.0281 | 3433885.25 |
| 47000 | 0.4747 | 322.9321 | 105478.3828 | 1.1987 | 13.0308 | 76.742 | 9.593 | 19.1791 | 3598015.25 |
| 47500 | 0.4798 | 149.7064 | 88157.4688 | 1.2194 | 13.0236 | 76.784 | 9.598 | 6.3502 | 3139454.0 |
| 48000 | 0.4848 | 169.4081 | 98819.0078 | 1.1749 | 13.1241 | 76.195 | 9.524 | 7.2793 | 3567425.25 |
| 48500 | 0.4899 | 166.3617 | 96562.3672 | 1.1708 | 13.0961 | 76.359 | 9.545 | 7.3647 | 2989058.5 |
| 49000 | 0.4949 | 300.0900 | 93175.2031 | 1.1878 | 13.0683 | 76.521 | 9.565 | 17.6384 | 3177372.75 |
| 49500 | 0.5 | 313.3461 | 96827.9453 | 1.1991 | 13.0645 | 76.543 | 9.568 | 19.0243 | 3347053.75 |
| 50000 | 0.5051 | 315.2939 | 108959.2422 | 1.1893 | 13.0564 | 76.591 | 9.574 | 17.7988 | 3361371.75 |
| 50500 | 0.5101 | 319.6468 | 105843.0547 | 1.1929 | 13.078 | 76.464 | 9.558 | 18.2292 | 3783085.25 |
| 51000 | 0.5152 | 178.0739 | 104819.3594 | 1.1697 | 13.0507 | 76.624 | 9.578 | 7.6160 | 3549388.5 |
| 51500 | 0.5202 | 174.6719 | 97635.9688 | 1.1697 | 13.0521 | 76.616 | 9.577 | 7.6083 | 3180765.25 |
| 52000 | 0.5253 | 160.6095 | 102570.0703 | 1.2036 | 13.0985 | 76.345 | 9.543 | 6.6913 | 3285125.0 |
| 52500 | 0.5303 | 305.5793 | 108805.8672 | 1.1877 | 13.0699 | 76.512 | 9.564 | 17.6976 | 3895754.25 |
| 53000 | 0.5354 | 278.3026 | 123494.4297 | 1.1579 | 13.0455 | 76.655 | 9.582 | 14.0152 | 4812372.0 |
| 53500 | 0.5404 | 178.8584 | 95729.4453 | 1.1594 | 13.0738 | 76.489 | 9.561 | 7.8583 | 3509837.25 |
| 54000 | 0.5455 | 167.4348 | 97190.0547 | 1.1758 | 13.039 | 76.693 | 9.587 | 7.2176 | 3420171.75 |
| 54500 | 0.5505 | 169.5919 | 104105.6562 | 1.1767 | 13.1092 | 76.282 | 9.535 | 7.2116 | 3695298.75 |
| 55000 | 0.5556 | 304.8109 | 86283.9922 | 1.1971 | 13.0954 | 76.363 | 9.545 | 18.7798 | 2656564.75 |
| 55500 | 0.5606 | 309.5341 | 99812.1719 | 1.1877 | 13.1094 | 76.281 | 9.535 | 17.9243 | 3105299.75 |
| 56000 | 0.5657 | 181.0470 | 109428.3672 | 1.1646 | 13.0706 | 76.508 | 9.563 | 7.7915 | 3606662.25 |
| 56500 | 0.5707 | 172.4268 | 119439.8281 | 1.1714 | 13.0907 | 76.39 | 9.549 | 7.3116 | 3859547.25 |
| 57000 | 0.5758 | 181.5280 | 112816.8594 | 1.1685 | 13.0909 | 76.389 | 9.549 | 7.4794 | 4470744.5 |
| 57500 | 0.5808 | 172.6440 | 108118.4219 | 1.1749 | 13.0779 | 76.465 | 9.558 | 7.2956 | 4247494.5 |
| 58000 | 0.5859 | 170.7784 | 96984.9688 | 1.1666 | 13.0674 | 76.526 | 9.566 | 7.4896 | 3497684.25 |
| 58500 | 0.5909 | 187.6657 | 108821.2266 | 1.1502 | 13.074 | 76.488 | 9.561 | 8.1907 | 4326414.0 |
| 59000 | 0.5960 | 181.9186 | 110170.6719 | 1.1612 | 13.0954 | 76.362 | 9.545 | 7.8024 | 3796229.75 |
| 59500 | 0.6010 | 169.7661 | 98367.5859 | 1.1752 | 13.085 | 76.424 | 9.553 | 7.1868 | 3917642.75 |
| 60000 | 0.6061 | 174.1653 | 109205.0625 | 1.1671 | 13.0729 | 76.494 | 9.562 | 7.5242 | 3867793.5 |
| 60500 | 0.6111 | 182.0173 | 110762.0 | 1.1582 | 13.0984 | 76.345 | 9.543 | 7.8018 | 4296505.5 |
| 61000 | 0.6162 | 292.0177 | 119036.7109 | 1.1671 | 13.0383 | 76.697 | 9.587 | 15.1496 | 4035396.75 |
| 61500 | 0.6212 | 180.6477 | 109636.6562 | 1.1586 | 13.1006 | 76.333 | 9.542 | 7.7460 | 4307981.5 |
| 62000 | 0.6263 | 290.2137 | 105218.7734 | 1.1723 | 13.0559 | 76.594 | 9.574 | 16.2907 | 3145321.75 |
| 62500 | 0.6313 | 291.2834 | 114901.7344 | 1.1686 | 13.0985 | 76.344 | 9.543 | 15.8347 | 4085227.5 |
| 63000 | 0.6364 | 278.7233 | 115006.9844 | 1.1620 | 13.1128 | 76.262 | 9.533 | 15.0069 | 4373999.0 |
| 63500 | 0.6414 | 289.1926 | 109976.8906 | 1.1687 | 12.9988 | 76.93 | 9.616 | 15.7257 | 4080870.25 |
| 64000 | 0.6465 | 290.6524 | 98895.5625 | 1.1732 | 13.0708 | 76.506 | 9.563 | 16.3562 | 3556972.0 |
| 64500 | 0.6515 | 297.2787 | 117289.1562 | 1.1680 | 13.0411 | 76.681 | 9.585 | 15.8733 | 4229402.0 |
| 65000 | 0.6566 | 282.6259 | 99503.3906 | 1.1691 | 13.102 | 76.324 | 9.541 | 15.8497 | 2820168.75 |
| 65500 | 0.6616 | 285.0887 | 103564.4922 | 1.1711 | 13.0433 | 76.668 | 9.583 | 15.8105 | 3205471.0 |
| 66000 | 0.6667 | 268.4873 | 110676.2578 | 1.1565 | 13.0615 | 76.561 | 9.57 | 14.4643 | 3839008.25 |
| 66500 | 0.6717 | 180.0331 | 96896.2109 | 1.1533 | 13.0839 | 76.43 | 9.554 | 7.9142 | 3579820.25 |
| 67000 | 0.6768 | 286.5390 | 114554.2422 | 1.1634 | 13.0931 | 76.376 | 9.547 | 15.3310 | 4395052.5 |
| 67500 | 0.6818 | 270.9948 | 102339.1875 | 1.1581 | 13.1022 | 76.323 | 9.54 | 14.7645 | 3014686.25 |
| 68000 | 0.6869 | 174.7666 | 99216.4609 | 1.1590 | 13.0235 | 76.784 | 9.598 | 7.6864 | 3225200.0 |
| 68500 | 0.6919 | 182.3842 | 106283.7656 | 1.1527 | 13.0238 | 76.782 | 9.598 | 7.9389 | 3881232.5 |
| 69000 | 0.6970 | 181.8341 | 101914.8359 | 1.1503 | 13.0809 | 76.447 | 9.556 | 8.0811 | 3175679.5 |
| 69500 | 0.7020 | 220.4548 | 93070.3203 | 1.1436 | 13.0061 | 76.887 | 9.611 | 10.9909 | 2977913.75 |
| 70000 | 0.7071 | 255.4018 | 100964.6562 | 1.1474 | 13.0343 | 76.721 | 9.59 | 13.2650 | 4088497.5 |
| 70500 | 0.7121 | 286.8609 | 120793.3594 | 1.1591 | 13.0079 | 76.876 | 9.61 | 15.0174 | 3774011.75 |
| 71000 | 0.7172 | 253.7847 | 106096.8125 | 1.1450 | 13.0882 | 76.404 | 9.551 | 13.0216 | 3865728.5 |
| 71500 | 0.7222 | 279.8482 | 112729.4219 | 1.1623 | 13.0771 | 76.469 | 9.559 | 15.0828 | 4050499.25 |
| 72000 | 0.7273 | 262.5128 | 99727.875 | 1.1512 | 13.0938 | 76.372 | 9.547 | 14.0373 | 2888705.25 |
| 72500 | 0.7323 | 184.2660 | 95614.9375 | 1.1445 | 13.0655 | 76.538 | 9.567 | 8.3834 | 3116921.5 |
| 73000 | 0.7374 | 266.8079 | 115803.4766 | 1.1492 | 13.0832 | 76.434 | 9.554 | 13.6403 | 4397400.0 |
| 73500 | 0.7424 | 230.1486 | 105463.5938 | 1.1373 | 13.0489 | 76.635 | 9.579 | 11.2846 | 4277062.0 |
| 74000 | 0.7475 | 264.5030 | 105805.8125 | 1.1503 | 13.1118 | 76.267 | 9.533 | 13.7059 | 3839008.25 |
| 74500 | 0.7525 | 258.7174 | 104915.3672 | 1.1492 | 13.06 | 76.57 | 9.571 | 13.8105 | 3102816.0 |
| 75000 | 0.7576 | 183.3511 | 99398.2969 | 1.1417 | 13.0784 | 76.462 | 9.558 | 8.5006 | 2929059.75 |
| 75500 | 0.7626 | 182.4444 | 94897.0938 | 1.1405 | 13.1237 | 76.198 | 9.525 | 8.5186 | 3265026.25 |
| 76000 | 0.7677 | 247.3889 | 104752.9062 | 1.1412 | 13.076 | 76.476 | 9.56 | 12.6346 | 3483716.0 |
| 76500 | 0.7727 | 253.8929 | 107148.1094 | 1.1430 | 13.1117 | 76.268 | 9.533 | 12.9910 | 3717053.5 |
| 77000 | 0.7778 | 188.3027 | 92560.3828 | 1.1375 | 13.0957 | 76.361 | 9.545 | 8.8484 | 3138615.75 |
| 77500 | 0.7828 | 240.6223 | 92972.0234 | 1.1425 | 13.0371 | 76.704 | 9.588 | 12.9289 | 3493024.25 |
| 78000 | 0.7879 | 178.5055 | 92919.6406 | 1.1440 | 13.087 | 76.412 | 9.551 | 8.3096 | 3285999.25 |
| 78500 | 0.7929 | 191.3466 | 106681.2266 | 1.1390 | 13.0407 | 76.683 | 9.585 | 8.6106 | 3740930.0 |
| 79000 | 0.7980 | 192.3125 | 114143.5547 | 1.1400 | 13.0628 | 76.553 | 9.569 | 8.5694 | 4021424.0 |
| 79500 | 0.8030 | 222.9796 | 88574.4453 | 1.1359 | 13.0912 | 76.387 | 9.548 | 11.6230 | 3083832.75 |
| 80000 | 0.8081 | 243.9444 | 104193.7578 | 1.1401 | 13.1212 | 76.213 | 9.527 | 12.3900 | 3348838.5 |
| 80500 | 0.8131 | 238.7563 | 97395.6641 | 1.1402 | 13.1003 | 76.334 | 9.542 | 12.6257 | 3038913.25 |
| 81000 | 0.8182 | 228.0278 | 103725.1094 | 1.1354 | 13.1397 | 76.105 | 9.513 | 11.2032 | 3587469.0 |
| 81500 | 0.8232 | 236.6571 | 107647.3125 | 1.1358 | 13.1 | 76.336 | 9.542 | 11.6837 | 3526734.5 |
| 82000 | 0.8283 | 227.7895 | 94743.5547 | 1.1349 | 13.0719 | 76.5 | 9.563 | 11.7360 | 2817161.0 |
| 82500 | 0.8333 | 215.8249 | 98965.1875 | 1.1316 | 13.0651 | 76.54 | 9.567 | 10.6053 | 3170598.5 |
| 83000 | 0.8384 | 234.6582 | 94883.7031 | 1.1384 | 13.0297 | 76.748 | 9.593 | 12.1979 | 3243321.0 |
| 83500 | 0.8434 | 189.5467 | 101377.8984 | 1.1374 | 13.0248 | 76.776 | 9.597 | 8.7173 | 3352414.25 |
| 84000 | 0.8485 | 193.5829 | 96691.6562 | 1.1354 | 13.0366 | 76.707 | 9.588 | 9.0339 | 2846625.25 |
| 84500 | 0.8535 | 196.7655 | 97601.6172 | 1.1351 | 13.1063 | 76.299 | 9.537 | 9.1239 | 3098679.0 |
| 85000 | 0.8586 | 223.5331 | 92183.0859 | 1.1319 | 13.0619 | 76.559 | 9.57 | 11.1778 | 3199491.0 |
| 85500 | 0.8636 | 213.2902 | 90484.9766 | 1.1311 | 13.0871 | 76.411 | 9.551 | 10.8684 | 2865675.75 |
| 86000 | 0.8687 | 192.1710 | 92092.2266 | 1.1325 | 13.0805 | 76.45 | 9.556 | 9.2500 | 2599077.25 |
| 86500 | 0.8737 | 226.9353 | 98111.6328 | 1.1334 | 13.1064 | 76.299 | 9.537 | 11.5479 | 3165525.75 |
| 87000 | 0.8788 | 204.7582 | 106343.6797 | 1.1310 | 13.0926 | 76.379 | 9.547 | 9.5713 | 3692343.25 |
| 87500 | 0.8838 | 217.5622 | 91716.8359 | 1.1318 | 13.1024 | 76.322 | 9.54 | 11.1243 | 2978709.0 |
| 88000 | 0.8889 | 208.4875 | 98457.6875 | 1.1303 | 13.0565 | 76.59 | 9.574 | 10.2574 | 3407419.25 |
| 88500 | 0.8939 | 234.9310 | 106613.6953 | 1.1328 | 13.0868 | 76.413 | 9.552 | 11.2762 | 3323915.5 |
| 89000 | 0.8990 | 193.3058 | 88968.3672 | 1.1323 | 13.0744 | 76.486 | 9.561 | 9.2780 | 3111103.75 |
| 89500 | 0.9040 | 204.5204 | 97952.7891 | 1.1323 | 13.0041 | 76.899 | 9.612 | 9.4639 | 3641472.5 |
| 90000 | 0.9091 | 204.2354 | 104649.6562 | 1.1301 | 13.052 | 76.617 | 9.577 | 9.6915 | 3580776.5 |
| 90500 | 0.9141 | 204.6234 | 93794.1094 | 1.1303 | 13.0339 | 76.723 | 9.59 | 9.8751 | 2991451.25 |
| 91000 | 0.9192 | 208.5037 | 94171.3516 | 1.1297 | 13.034 | 76.723 | 9.59 | 10.0488 | 3319483.75 |
| 91500 | 0.9242 | 213.5300 | 99188.5547 | 1.1291 | 13.0395 | 76.69 | 9.586 | 10.5598 | 3441222.25 |
| 92000 | 0.9293 | 207.6655 | 90542.2891 | 1.1295 | 13.0319 | 76.735 | 9.592 | 10.2286 | 2989058.5 |
| 92500 | 0.9343 | 215.0739 | 100256.0703 | 1.1295 | 13.0518 | 76.618 | 9.577 | 10.6255 | 3091247.0 |
| 93000 | 0.9394 | 208.0277 | 96290.7188 | 1.1287 | 13.0971 | 76.352 | 9.544 | 9.9776 | 3326576.25 |
| 93500 | 0.9444 | 207.6574 | 89616.0547 | 1.1288 | 13.0729 | 76.494 | 9.562 | 10.4967 | 2774643.5 |
| 94000 | 0.9495 | 215.1155 | 98846.8125 | 1.1283 | 13.0434 | 76.667 | 9.583 | 10.4396 | 3530500.0 |
| 94500 | 0.9545 | 208.6168 | 96902.9609 | 1.1282 | 13.1265 | 76.182 | 9.523 | 10.1000 | 3193519.0 |
| 95000 | 0.9596 | 211.5460 | 101485.0781 | 1.1286 | 13.0886 | 76.403 | 9.55 | 10.0650 | 3358680.0 |
| 95500 | 0.9646 | 211.6526 | 100730.2734 | 1.1280 | 13.0669 | 76.529 | 9.566 | 10.2629 | 3412879.75 |
| 96000 | 0.9697 | 210.3450 | 97340.7891 | 1.1287 | 13.0419 | 76.676 | 9.584 | 10.2332 | 3266770.25 |
| 96500 | 0.9747 | 213.0178 | 98652.0547 | 1.1278 | 13.1033 | 76.317 | 9.54 | 10.3259 | 3173983.75 |
| 97000 | 0.9798 | 210.8997 | 96412.8359 | 1.1280 | 13.0251 | 76.775 | 9.597 | 10.2531 | 3220043.0 |
| 97500 | 0.9848 | 211.1777 | 98499.3828 | 1.1278 | 13.0988 | 76.343 | 9.543 | 10.2239 | 3245920.25 |
| 98000 | 0.9899 | 211.2104 | 98721.6094 | 1.1274 | 13.0593 | 76.574 | 9.572 | 10.2252 | 3149521.0 |
| 98500 | 0.9949 | 211.2759 | 98499.3828 | 1.1275 | 13.0629 | 76.552 | 9.569 | 10.2273 | 3147842.5 |
| 99000 | 1.0 | 211.4068 | 98721.6094 | 1.1274 | 13.054 | 76.605 | 9.576 | 10.2269 | 3163838.5 |

### Framework versions
- Distily 0.2.0
- Transformers 4.44.0
- Pytorch 2.3.0
- Datasets 2.20.0