distily_bench_obj_cross_v2.2 / README.md

End of training

043a63d verified 3 months ago

22.6 kB

	---
	base_model: roneneldan/TinyStories-33M
	library_name: Distily
	tags:
	- generated_from_trainer
	model-index:
	- name: distily_bench_obj_cross_v2.2
	results: []
	---

	# distily_bench_obj_cross_v2.2

	This student model is distilled from the teacher model [roneneldan/TinyStories-33M](https://huggingface.co/roneneldan/TinyStories-33M) using the dataset (unspecified).

	The [Distily](https://github.com/lapp0/distily) library was used for this distillation.

	It achieves the following results on the evaluation set:
	- eval_enwikippl: 173.8350
	- eval_frwikippl: 60887.2422
	- eval_zhwikippl: 328379.5312
	- eval_tinystoriesppl: 10.2239
	- eval_loss: 1.1755
	- eval_runtime: 13.1746
	- eval_samples_per_second: 75.903
	- eval_steps_per_second: 9.488

	<!-- This model card has been generated automatically according to the information the Trainer had access to. You
	should probably proofread and complete it, then remove this comment.

	## Model description

	More information needed

	## Intended uses & limitations

	More information needed

	## Training and evaluation data

	More information needed
	-->

	## Training procedure

	### Training hyperparameters

	The following hyperparameters were used during training:
	- distillation_objective: DistillationObjective(logits_loss_component=LossComponent(label=logits, weight=1, loss_fn=kl, layer_mapper=None, projector=None), hs_loss_component=LossComponent(label=hs, weight=0, loss_fn=None, layer_mapper=None, projector=None), attn_loss_component=LossComponent(label=attn, weight=0, loss_fn=None, layer_mapper=None, projector=None))
	- train_embeddings: True
	- learning_rate: 0.004
	- train_batch_size: 1
	- eval_batch_size: 8
	- seed: 42
	- optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
	- lr_scheduler_type: linear
	- lr_scheduler_warmup_ratio: 0.5
	- num_epochs: 1.0

	### Resource Usage
	Peak GPU Memory: 6.6047 GB

	### Eval-Phase Metrics
	\| step \| epoch \| enwikippl \| frwikippl \| loss \| runtime \| samples_per_second \| steps_per_second \| tinystoriesppl \| zhwikippl \|
	\| --- \| --- \| --- \| --- \| --- \| --- \| --- \| --- \| --- \| --- \|
	\| teacher eval \| \| 169.9865 \| 47377.9414 \| \| \| \| \| 3.9789 \| 4998.1294 \|
	\| 0 \| 0 \| 52669.6523 \| 72993.9297 \| 6.5530 \| 13.1454 \| 76.072 \| 9.509 \| 39454.2930 \| 71157.1797 \|
	\| 500 \| 0.0051 \| 44956.75 \| 68626.6641 \| 6.4185 \| 13.0649 \| 76.541 \| 9.568 \| 32178.2012 \| 70101.9766 \|
	\| 1000 \| 0.0101 \| 33763.7852 \| 61252.8203 \| 6.1768 \| 13.1033 \| 76.317 \| 9.54 \| 22370.0684 \| 68183.5859 \|
	\| 1500 \| 0.0152 \| 33690.6094 \| 61252.8203 \| 6.1762 \| 13.0831 \| 76.434 \| 9.554 \| 22336.8125 \| 68147.2422 \|
	\| 2000 \| 0.0202 \| 21711.3809 \| 51603.3672 \| 5.8095 \| 13.0769 \| 76.47 \| 9.559 \| 12725.5420 \| 65753.9297 \|
	\| 2500 \| 0.0253 \| 21691.2031 \| 51559.7812 \| 5.8095 \| 13.0773 \| 76.468 \| 9.559 \| 12706.6240 \| 65613.7344 \|
	\| 3000 \| 0.0303 \| 21677.7617 \| 51559.7812 \| 5.8095 \| 13.0957 \| 76.361 \| 9.545 \| 12702.4199 \| 65648.7891 \|
	\| 3500 \| 0.0354 \| 9303.9834 \| 38552.1719 \| 5.1132 \| 13.1366 \| 76.123 \| 9.515 \| 4258.6343 \| 63461.6641 \|
	\| 4000 \| 0.0404 \| 9057.9209 \| 38432.9023 \| 5.1118 \| 13.086 \| 76.418 \| 9.552 \| 4096.3477 \| 62520.5195 \|
	\| 4500 \| 0.0455 \| 8782.9141 \| 38216.9688 \| 5.1102 \| 13.103 \| 76.318 \| 9.54 \| 3909.0979 \| 61741.4727 \|
	\| 5000 \| 0.0505 \| 8717.8486 \| 38206.1836 \| 5.1095 \| 13.1133 \| 76.259 \| 9.532 \| 3867.3118 \| 61708.5078 \|
	\| 5500 \| 0.0556 \| 8717.8486 \| 38206.1836 \| 5.1097 \| 13.0748 \| 76.483 \| 9.56 \| 3868.5918 \| 61675.6211 \|
	\| 6000 \| 0.0606 \| 8725.9502 \| 38206.1836 \| 5.1095 \| 13.1774 \| 75.888 \| 9.486 \| 3875.6340 \| 61675.6211 \|
	\| 6500 \| 0.0657 \| 1777.1456 \| 25311.8418 \| 3.8983 \| 13.1926 \| 75.8 \| 9.475 \| 462.9113 \| 63954.5195 \|
	\| 7000 \| 0.0707 \| 1634.0082 \| 25035.2793 \| 3.8856 \| 13.0968 \| 76.354 \| 9.544 \| 409.3667 \| 61675.6211 \|
	\| 7500 \| 0.0758 \| 1540.2275 \| 24880.5918 \| 3.8800 \| 13.0526 \| 76.613 \| 9.577 \| 375.2083 \| 59509.5859 \|
	\| 8000 \| 0.0808 \| 1508.3474 \| 24884.1035 \| 3.8793 \| 13.0768 \| 76.472 \| 9.559 \| 364.5079 \| 58925.0273 \|
	\| 8500 \| 0.0859 \| 1464.3663 \| 24786.1445 \| 3.8795 \| 13.0459 \| 76.653 \| 9.582 \| 347.5009 \| 58221.8125 \|
	\| 9000 \| 0.0909 \| 1450.2551 \| 24782.6699 \| 3.8793 \| 13.094 \| 76.371 \| 9.546 \| 341.6051 \| 58066.7188 \|
	\| 9500 \| 0.0960 \| 1456.5604 \| 24751.2559 \| 3.8791 \| 13.1265 \| 76.182 \| 9.523 \| 345.3246 \| 58221.8125 \|
	\| 10000 \| 0.1010 \| 1482.8590 \| 24814.0996 \| 3.8794 \| 13.095 \| 76.365 \| 9.546 \| 355.1973 \| 58424.1094 \|
	\| 10500 \| 0.1061 \| 1474.3835 \| 24800.1191 \| 3.8794 \| 13.0825 \| 76.438 \| 9.555 \| 351.5170 \| 58221.8125 \|
	\| 11000 \| 0.1111 \| 1489.4211 \| 24772.2012 \| 3.8791 \| 13.1427 \| 76.088 \| 9.511 \| 356.9635 \| 58486.4883 \|
	\| 11500 \| 0.1162 \| 1478.0425 \| 24779.1719 \| 3.8794 \| 13.0727 \| 76.495 \| 9.562 \| 354.0248 \| 58392.9141 \|
	\| 12000 \| 0.1212 \| 1481.9408 \| 24779.1719 \| 3.8796 \| 13.0577 \| 76.583 \| 9.573 \| 354.8745 \| 58392.9141 \|
	\| 12500 \| 0.1263 \| 157.4985 \| 35001.3008 \| 2.6554 \| 13.1308 \| 76.157 \| 9.52 \| 12.3119 \| 151077.2656 \|
	\| 13000 \| 0.1313 \| 169.6839 \| 26583.5117 \| 2.5062 \| 13.0753 \| 76.48 \| 9.56 \| 14.8852 \| 101955.1797 \|
	\| 13500 \| 0.1364 \| 196.3011 \| 21765.9062 \| 2.4544 \| 13.1825 \| 75.858 \| 9.482 \| 19.3727 \| 75017.1797 \|
	\| 14000 \| 0.1414 \| 190.3265 \| 21478.0918 \| 2.4500 \| 13.1157 \| 76.245 \| 9.531 \| 18.7379 \| 77210.1953 \|
	\| 14500 \| 0.1465 \| 188.2079 \| 21357.4160 \| 2.4495 \| 13.0923 \| 76.381 \| 9.548 \| 18.4810 \| 76187.0469 \|
	\| 15000 \| 0.1515 \| 189.2460 \| 21231.4473 \| 2.4492 \| 13.0975 \| 76.351 \| 9.544 \| 18.6923 \| 74538.3828 \|
	\| 15500 \| 0.1566 \| 188.9237 \| 21405.6016 \| 2.4492 \| 13.1439 \| 76.081 \| 9.51 \| 18.5468 \| 75358.2109 \|
	\| 16000 \| 0.1616 \| 188.8067 \| 21514.4375 \| 2.4490 \| 13.0852 \| 76.422 \| 9.553 \| 18.5162 \| 76349.8281 \|
	\| 16500 \| 0.1667 \| 190.2086 \| 21351.4102 \| 2.4489 \| 13.1595 \| 75.99 \| 9.499 \| 18.8210 \| 76146.375 \|
	\| 17000 \| 0.1717 \| 189.0994 \| 21333.375 \| 2.4491 \| 13.098 \| 76.348 \| 9.543 \| 18.6283 \| 75579.6641 \|
	\| 17500 \| 0.1768 \| 189.0262 \| 21484.1543 \| 2.4490 \| 13.1692 \| 75.935 \| 9.492 \| 18.6006 \| 76268.3984 \|
	\| 18000 \| 0.1818 \| 189.9141 \| 21599.4473 \| 2.4489 \| 13.1461 \| 76.068 \| 9.509 \| 18.7016 \| 76922.3125 \|
	\| 18500 \| 0.1869 \| 190.1939 \| 21351.4102 \| 2.4491 \| 13.1761 \| 75.895 \| 9.487 \| 18.8568 \| 75579.6641 \|
	\| 19000 \| 0.1919 \| 188.3976 \| 21315.3379 \| 2.4494 \| 13.1038 \| 76.314 \| 9.539 \| 18.4749 \| 75559.4922 \|
	\| 19500 \| 0.1970 \| 187.5166 \| 21393.5605 \| 2.4490 \| 13.1202 \| 76.218 \| 9.527 \| 18.3494 \| 75519.2188 \|
	\| 20000 \| 0.2020 \| 190.0024 \| 21520.4922 \| 2.4490 \| 13.1338 \| 76.139 \| 9.517 \| 18.7658 \| 75822.0234 \|
	\| 20500 \| 0.2071 \| 189.1580 \| 21441.8262 \| 2.4490 \| 13.0779 \| 76.465 \| 9.558 \| 18.5921 \| 76227.6719 \|
	\| 21000 \| 0.2121 \| 189.8847 \| 21496.2461 \| 2.4490 \| 13.1311 \| 76.155 \| 9.519 \| 18.7434 \| 76717.3984 \|
	\| 21500 \| 0.2172 \| 190.1055 \| 21429.7441 \| 2.4489 \| 13.1056 \| 76.304 \| 9.538 \| 18.7829 \| 76105.7891 \|
	\| 22000 \| 0.2222 \| 189.7671 \| 21575.1328 \| 2.4489 \| 13.0699 \| 76.512 \| 9.564 \| 18.6738 \| 76922.3125 \|
	\| 22500 \| 0.2273 \| 188.3319 \| 21363.4473 \| 2.4492 \| 13.1296 \| 76.164 \| 9.52 \| 18.4368 \| 75358.2109 \|
	\| 23000 \| 0.2323 \| 189.4659 \| 21393.5605 \| 2.4492 \| 13.1044 \| 76.31 \| 9.539 \| 18.6908 \| 75822.0234 \|
	\| 23500 \| 0.2374 \| 187.9821 \| 21345.3828 \| 2.4492 \| 13.1019 \| 76.325 \| 9.541 \| 18.3919 \| 75984.0234 \|
	\| 24000 \| 0.2424 \| 189.4146 \| 21532.625 \| 2.4489 \| 13.1461 \| 76.068 \| 9.509 \| 18.6892 \| 76512.9609 \|
	\| 24500 \| 0.2475 \| 258.9580 \| 127264.7109 \| 1.9007 \| 13.0286 \| 76.754 \| 9.594 \| 12.0471 \| 1619514.5 \|
	\| 25000 \| 0.2525 \| 311.6756 \| 205648.5469 \| 1.6701 \| 13.0855 \| 76.42 \| 9.553 \| 12.6346 \| 1182116.125 \|
	\| 25500 \| 0.2576 \| 182.4019 \| 50272.4961 \| 1.3738 \| 13.088 \| 76.406 \| 9.551 \| 10.9822 \| 252693.0781 \|
	\| 26000 \| 0.2626 \| 162.9404 \| 39467.0117 \| 1.2816 \| 13.1163 \| 76.241 \| 9.53 \| 10.5132 \| 214625.8125 \|
	\| 26500 \| 0.2677 \| 153.3747 \| 36651.1523 \| 1.2536 \| 13.0665 \| 76.532 \| 9.566 \| 10.3408 \| 150033.0 \|
	\| 27000 \| 0.2727 \| 149.3994 \| 32545.2773 \| 1.2488 \| 13.1249 \| 76.191 \| 9.524 \| 10.5267 \| 124175.5 \|
	\| 27500 \| 0.2778 \| 147.5134 \| 31601.1426 \| 1.2457 \| 13.1132 \| 76.259 \| 9.532 \| 10.4064 \| 133272.7188 \|
	\| 28000 \| 0.2828 \| 145.7415 \| 32339.6230 \| 1.2452 \| 13.0604 \| 76.567 \| 9.571 \| 10.1394 \| 135026.4219 \|
	\| 28500 \| 0.2879 \| 149.5152 \| 32673.8711 \| 1.2452 \| 13.0107 \| 76.86 \| 9.607 \| 10.3944 \| 130458.2578 \|
	\| 29000 \| 0.2929 \| 146.3581 \| 31681.3496 \| 1.2453 \| 13.0181 \| 76.816 \| 9.602 \| 10.3806 \| 125776.0312 \|
	\| 29500 \| 0.2980 \| 147.4791 \| 31717.0820 \| 1.2449 \| 13.0769 \| 76.471 \| 9.559 \| 10.3862 \| 133415.0156 \|
	\| 30000 \| 0.3030 \| 147.3706 \| 32067.4570 \| 1.2448 \| 13.1294 \| 76.165 \| 9.521 \| 10.3447 \| 132705.0312 \|
	\| 30500 \| 0.3081 \| 148.4822 \| 33536.5156 \| 1.2456 \| 13.1014 \| 76.328 \| 9.541 \| 10.3566 \| 133985.7344 \|
	\| 31000 \| 0.3131 \| 149.1219 \| 33244.8906 \| 1.2444 \| 13.0892 \| 76.399 \| 9.55 \| 10.3648 \| 139568.2031 \|
	\| 31500 \| 0.3182 \| 146.6759 \| 33109.375 \| 1.2451 \| 13.1438 \| 76.081 \| 9.51 \| 10.2519 \| 139419.3438 \|
	\| 32000 \| 0.3232 \| 148.4764 \| 32022.3203 \| 1.2451 \| 13.1327 \| 76.146 \| 9.518 \| 10.3978 \| 134307.8594 \|
	\| 32500 \| 0.3283 \| 146.0071 \| 32230.5059 \| 1.2448 \| 13.1347 \| 76.134 \| 9.517 \| 10.3272 \| 132917.5625 \|
	\| 33000 \| 0.3333 \| 147.7707 \| 34223.6719 \| 1.2454 \| 13.093 \| 76.377 \| 9.547 \| 10.2033 \| 142199.25 \|
	\| 33500 \| 0.3384 \| 150.1943 \| 34843.8672 \| 1.2452 \| 13.1431 \| 76.086 \| 9.511 \| 10.2392 \| 145460.7969 \|
	\| 34000 \| 0.3434 \| 147.6906 \| 32766.0488 \| 1.2446 \| 13.1388 \| 76.111 \| 9.514 \| 10.3571 \| 134200.3125 \|
	\| 34500 \| 0.3485 \| 147.1482 \| 31941.25 \| 1.2448 \| 13.0898 \| 76.395 \| 9.549 \| 10.3862 \| 130736.9922 \|
	\| 35000 \| 0.3535 \| 149.6890 \| 33423.3281 \| 1.2452 \| 13.0942 \| 76.37 \| 9.546 \| 10.3541 \| 139122.0938 \|
	\| 35500 \| 0.3586 \| 149.5325 \| 33160.7266 \| 1.2447 \| 13.1322 \| 76.148 \| 9.519 \| 10.4094 \| 144686.7188 \|
	\| 36000 \| 0.3636 \| 145.3750 \| 31777.4531 \| 1.2452 \| 13.0787 \| 76.46 \| 9.558 \| 10.2125 \| 133308.3125 \|
	\| 36500 \| 0.3687 \| 148.9487 \| 32298.6602 \| 1.2447 \| 13.0705 \| 76.508 \| 9.563 \| 10.4807 \| 133985.7344 \|
	\| 37000 \| 0.3737 \| 150.1129 \| 33116.3867 \| 1.2452 \| 13.1491 \| 76.051 \| 9.506 \| 10.5128 \| 142921.8594 \|
	\| 37500 \| 0.3788 \| 149.2490 \| 32731.4746 \| 1.2450 \| 13.0425 \| 76.672 \| 9.584 \| 10.4310 \| 138160.3438 \|
	\| 38000 \| 0.3838 \| 148.4189 \| 32858.4844 \| 1.2451 \| 13.1334 \| 76.142 \| 9.518 \| 10.3965 \| 137425.1094 \|
	\| 38500 \| 0.3889 \| 145.6230 \| 32011.0527 \| 1.2448 \| 13.1597 \| 75.989 \| 9.499 \| 10.2121 \| 131858.0469 \|
	\| 39000 \| 0.3939 \| 147.8051 \| 32844.6055 \| 1.2449 \| 13.2291 \| 75.591 \| 9.449 \| 10.2214 \| 133807.0938 \|
	\| 39500 \| 0.3990 \| 147.3307 \| 32692.2910 \| 1.2449 \| 13.1731 \| 75.912 \| 9.489 \| 10.4129 \| 139047.8125 \|
	\| 40000 \| 0.4040 \| 147.6105 \| 32385.2070 \| 1.2452 \| 13.0951 \| 76.364 \| 9.546 \| 10.2328 \| 134666.5938 \|
	\| 40500 \| 0.4091 \| 148.0744 \| 32637.0918 \| 1.2450 \| 13.0333 \| 76.727 \| 9.591 \| 10.4116 \| 138899.5156 \|
	\| 41000 \| 0.4141 \| 149.1219 \| 32618.7012 \| 1.2450 \| 13.1237 \| 76.198 \| 9.525 \| 10.4560 \| 136547.9844 \|
	\| 41500 \| 0.4192 \| 150.2699 \| 33852.1289 \| 1.2456 \| 13.0975 \| 76.35 \| 9.544 \| 10.4232 \| 143993.5 \|
	\| 42000 \| 0.4242 \| 148.0400 \| 31592.2227 \| 1.2450 \| 13.0907 \| 76.39 \| 9.549 \| 10.4318 \| 138603.375 \|
	\| 42500 \| 0.4293 \| 149.2722 \| 33263.6328 \| 1.2454 \| 13.1088 \| 76.284 \| 9.536 \| 10.3460 \| 130110.7344 \|
	\| 43000 \| 0.4343 \| 146.9943 \| 32357.8555 \| 1.2455 \| 13.1698 \| 75.931 \| 9.491 \| 10.2769 \| 135026.4219 \|
	\| 43500 \| 0.4394 \| 149.7238 \| 33069.7734 \| 1.2452 \| 13.0987 \| 76.343 \| 9.543 \| 10.4577 \| 135893.7656 \|
	\| 44000 \| 0.4444 \| 149.3011 \| 32772.9844 \| 1.2455 \| 13.0817 \| 76.443 \| 9.555 \| 10.5380 \| 136839.7344 \|
	\| 44500 \| 0.4495 \| 146.3694 \| 31726.0059 \| 1.2451 \| 13.0363 \| 76.709 \| 9.589 \| 10.3242 \| 129625.6094 \|
	\| 45000 \| 0.4545 \| 149.3531 \| 32362.4219 \| 1.2452 \| 13.0922 \| 76.382 \| 9.548 \| 10.4348 \| 130667.3203 \|
	\| 45500 \| 0.4596 \| 146.4262 \| 32339.6230 \| 1.2455 \| 13.0782 \| 76.463 \| 9.558 \| 10.2697 \| 133842.8281 \|
	\| 46000 \| 0.4646 \| 147.8222 \| 31521.1094 \| 1.2453 \| 13.2075 \| 75.714 \| 9.464 \| 10.4232 \| 131858.0469 \|
	\| 46500 \| 0.4697 \| 149.9443 \| 32858.4844 \| 1.2455 \| 13.0535 \| 76.608 \| 9.576 \| 10.4504 \| 138899.5156 \|
	\| 47000 \| 0.4747 \| 147.2907 \| 32312.3086 \| 1.2446 \| 13.1333 \| 76.143 \| 9.518 \| 10.2999 \| 129764.0156 \|
	\| 47500 \| 0.4798 \| 147.8280 \| 32031.3594 \| 1.2450 \| 13.1307 \| 76.157 \| 9.52 \| 10.2973 \| 135170.5938 \|
	\| 48000 \| 0.4848 \| 150.8706 \| 33652.4297 \| 1.2459 \| 13.1532 \| 76.027 \| 9.503 \| 10.5084 \| 143303.7344 \|
	\| 48500 \| 0.4899 \| 233.2175 \| 134016.5312 \| 1.2454 \| 13.0411 \| 76.681 \| 9.585 \| 10.7256 \| 484649.75 \|
	\| 49000 \| 0.4949 \| 192.8943 \| 80054.5391 \| 1.1914 \| 13.171 \| 75.925 \| 9.491 \| 10.3118 \| 439093.1875 \|
	\| 49500 \| 0.5 \| 181.6897 \| 66668.6797 \| 1.1807 \| 13.0366 \| 76.707 \| 9.588 \| 10.5398 \| 372647.1562 \|
	\| 50000 \| 0.5051 \| 178.5919 \| 64099.4805 \| 1.1779 \| 13.1197 \| 76.222 \| 9.528 \| 10.2413 \| 347117.8438 \|
	\| 50500 \| 0.5101 \| 170.0656 \| 56934.7031 \| 1.1777 \| 13.1278 \| 76.174 \| 9.522 \| 10.2163 \| 286528.3438 \|
	\| 51000 \| 0.5152 \| 176.2691 \| 60767.2773 \| 1.1768 \| 13.0822 \| 76.44 \| 9.555 \| 10.2807 \| 348044.9688 \|
	\| 51500 \| 0.5202 \| 175.1937 \| 59547.1094 \| 1.1764 \| 13.098 \| 76.348 \| 9.543 \| 10.2557 \| 329608.5 \|
	\| 52000 \| 0.5253 \| 178.4812 \| 65233.4531 \| 1.1766 \| 13.1056 \| 76.303 \| 9.538 \| 10.2637 \| 369380.6562 \|
	\| 52500 \| 0.5303 \| 176.5834 \| 62161.0664 \| 1.1768 \| 13.1118 \| 76.267 \| 9.533 \| 10.1969 \| 350841.9688 \|
	\| 53000 \| 0.5354 \| 174.2125 \| 60052.5039 \| 1.1765 \| 13.1029 \| 76.319 \| 9.54 \| 10.2527 \| 323423.5312 \|
	\| 53500 \| 0.5404 \| 176.9943 \| 63707.9219 \| 1.1763 \| 13.0066 \| 76.884 \| 9.611 \| 10.1889 \| 362352.625 \|
	\| 54000 \| 0.5455 \| 179.0075 \| 64104.0039 \| 1.1762 \| 13.2498 \| 75.473 \| 9.434 \| 10.3481 \| 362546.1875 \|
	\| 54500 \| 0.5505 \| 171.8867 \| 60762.9922 \| 1.1763 \| 13.1655 \| 75.956 \| 9.495 \| 10.0003 \| 306454.5938 \|
	\| 55000 \| 0.5556 \| 176.8846 \| 63207.3789 \| 1.1763 \| 13.1414 \| 76.095 \| 9.512 \| 10.2701 \| 337618.6875 \|
	\| 55500 \| 0.5606 \| 175.9485 \| 63649.6211 \| 1.1769 \| 13.0975 \| 76.35 \| 9.544 \| 10.1818 \| 371951.9688 \|
	\| 56000 \| 0.5657 \| 172.9887 \| 60955.8594 \| 1.1764 \| 13.1161 \| 76.242 \| 9.53 \| 10.0738 \| 341605.2812 \|
	\| 56500 \| 0.5707 \| 174.8141 \| 59287.6406 \| 1.1767 \| 13.0894 \| 76.398 \| 9.55 \| 10.1335 \| 333144.7812 \|
	\| 57000 \| 0.5758 \| 175.8667 \| 62415.5508 \| 1.1766 \| 13.0944 \| 76.369 \| 9.546 \| 10.1214 \| 344902.0312 \|
	\| 57500 \| 0.5808 \| 175.9553 \| 64830.4531 \| 1.1772 \| 13.1232 \| 76.201 \| 9.525 \| 10.1974 \| 364485.8125 \|
	\| 58000 \| 0.5859 \| 177.7224 \| 62728.4141 \| 1.1767 \| 13.1442 \| 76.079 \| 9.51 \| 10.2901 \| 352342.8125 \|
	\| 58500 \| 0.5909 \| 174.5569 \| 59045.9727 \| 1.1765 \| 13.1927 \| 75.799 \| 9.475 \| 10.2303 \| 330136.375 \|
	\| 59000 \| 0.5960 \| 176.9120 \| 62516.7031 \| 1.1762 \| 13.0404 \| 76.685 \| 9.586 \| 10.3225 \| 338159.4062 \|
	\| 59500 \| 0.6010 \| 176.6108 \| 62340.8320 \| 1.1763 \| 13.1224 \| 76.206 \| 9.526 \| 10.2863 \| 350280.6562 \|
	\| 60000 \| 0.6061 \| 172.9151 \| 62380.3789 \| 1.1763 \| 13.2135 \| 75.68 \| 9.46 \| 10.0223 \| 335106.0312 \|
	\| 60500 \| 0.6111 \| 172.8012 \| 58879.8633 \| 1.1761 \| 13.1236 \| 76.198 \| 9.525 \| 10.1251 \| 328905.75 \|
	\| 61000 \| 0.6162 \| 175.1665 \| 62723.9883 \| 1.1760 \| 13.0533 \| 76.609 \| 9.576 \| 10.1293 \| 365459.7188 \|
	\| 61500 \| 0.6212 \| 178.2050 \| 62994.0508 \| 1.1773 \| 13.1683 \| 75.94 \| 9.493 \| 10.3109 \| 339062.9688 \|
	\| 62000 \| 0.6263 \| 175.7850 \| 63131.7148 \| 1.1769 \| 13.1223 \| 76.206 \| 9.526 \| 10.1310 \| 348974.9375 \|
	\| 62500 \| 0.6313 \| 176.4672 \| 61663.9766 \| 1.1768 \| 13.0971 \| 76.353 \| 9.544 \| 10.2824 \| 356028.25 \|
	\| 63000 \| 0.6364 \| 176.9943 \| 63242.9531 \| 1.1762 \| 13.0327 \| 76.73 \| 9.591 \| 10.2671 \| 351779.0625 \|
	\| 63500 \| 0.6414 \| 173.9832 \| 60442.8633 \| 1.1762 \| 13.0653 \| 76.539 \| 9.567 \| 10.1339 \| 318202.5938 \|
	\| 64000 \| 0.6465 \| 175.6557 \| 59530.3594 \| 1.1761 \| 13.1488 \| 76.052 \| 9.507 \| 10.2680 \| 337258.5938 \|
	\| 64500 \| 0.6515 \| 174.9766 \| 62375.9766 \| 1.1759 \| 13.1342 \| 76.137 \| 9.517 \| 10.2570 \| 332079.9375 \|
	\| 65000 \| 0.6566 \| 172.8950 \| 59883.5625 \| 1.1763 \| 13.1758 \| 75.897 \| 9.487 \| 10.2223 \| 329256.9375 \|
	\| 65500 \| 0.6616 \| 177.9153 \| 63189.5391 \| 1.1765 \| 13.0493 \| 76.632 \| 9.579 \| 10.2252 \| 350093.9688 \|
	\| 66000 \| 0.6667 \| 175.5877 \| 60493.9570 \| 1.1759 \| 13.1408 \| 76.099 \| 9.512 \| 10.2960 \| 330841.75 \|
	\| 66500 \| 0.6717 \| 173.8889 \| 62152.3555 \| 1.1762 \| 13.1111 \| 76.271 \| 9.534 \| 10.1650 \| 350093.9688 \|
	\| 67000 \| 0.6768 \| 176.5698 \| 64266.7031 \| 1.1761 \| 13.0981 \| 76.347 \| 9.543 \| 10.1851 \| 362159.5 \|
	\| 67500 \| 0.6818 \| 175.3905 \| 62012.3906 \| 1.1764 \| 13.1151 \| 76.248 \| 9.531 \| 10.1159 \| 334034.9062 \|
	\| 68000 \| 0.6869 \| 173.2368 \| 58838.4375 \| 1.1761 \| 13.1682 \| 75.94 \| 9.493 \| 10.1725 \| 334927.125 \|
	\| 68500 \| 0.6919 \| 175.5400 \| 60357.7852 \| 1.1761 \| 13.1647 \| 75.961 \| 9.495 \| 10.2515 \| 349161.0312 \|
	\| 69000 \| 0.6970 \| 174.6111 \| 61942.5859 \| 1.1762 \| 13.139 \| 76.109 \| 9.514 \| 10.2498 \| 342518.0312 \|
	\| 69500 \| 0.7020 \| 172.1866 \| 59681.4531 \| 1.1761 \| 13.228 \| 75.597 \| 9.45 \| 10.1360 \| 319052.8125 \|
	\| 70000 \| 0.7071 \| 172.8950 \| 61750.8945 \| 1.1764 \| 13.2173 \| 75.658 \| 9.457 \| 10.1486 \| 327155.4375 \|
	\| 70500 \| 0.7121 \| 176.0235 \| 61559.8594 \| 1.1766 \| 13.1184 \| 76.229 \| 9.529 \| 10.2799 \| 360232.2188 \|
	\| 71000 \| 0.7172 \| 173.2704 \| 58788.6875 \| 1.1767 \| 13.1566 \| 76.007 \| 9.501 \| 10.1553 \| 317354.625 \|
	\| 71500 \| 0.7222 \| 174.7057 \| 61872.8008 \| 1.1759 \| 13.2179 \| 75.655 \| 9.457 \| 10.1285 \| 343432.9062 \|
	\| 72000 \| 0.7273 \| 176.7477 \| 62253.0820 \| 1.1765 \| 13.125 \| 76.19 \| 9.524 \| 10.3417 \| 350280.6562 \|
	\| 72500 \| 0.7323 \| 177.4336 \| 63815.6719 \| 1.1760 \| 13.1259 \| 76.185 \| 9.523 \| 10.2349 \| 366045.0 \|
	\| 73000 \| 0.7374 \| 176.1736 \| 60801.5391 \| 1.1760 \| 13.0704 \| 76.509 \| 9.564 \| 10.2510 \| 354322.4688 \|
	\| 73500 \| 0.7424 \| 175.7033 \| 63739.3398 \| 1.1767 \| 13.1804 \| 75.87 \| 9.484 \| 10.1076 \| 346747.5938 \|
	\| 74000 \| 0.7475 \| 175.8123 \| 61912.0508 \| 1.1763 \| 13.1949 \| 75.787 \| 9.473 \| 10.2718 \| 336898.875 \|
	\| 74500 \| 0.7525 \| 174.7870 \| 59934.1836 \| 1.1759 \| 13.1044 \| 76.31 \| 9.539 \| 10.2387 \| 331018.4688 \|
	\| 75000 \| 0.7576 \| 171.5209 \| 57795.2344 \| 1.1760 \| 13.1288 \| 76.168 \| 9.521 \| 10.2443 \| 294904.2812 \|
	\| 75500 \| 0.7626 \| 172.1866 \| 59698.3008 \| 1.1762 \| 13.1549 \| 76.018 \| 9.502 \| 10.1793 \| 329432.5 \|
	\| 76000 \| 0.7677 \| 175.2344 \| 61059.0117 \| 1.1759 \| 13.1045 \| 76.31 \| 9.539 \| 10.2671 \| 330665.4062 \|
	\| 76500 \| 0.7727 \| 173.0692 \| 60733.0391 \| 1.1755 \| 13.1001 \| 76.335 \| 9.542 \| 10.1364 \| 317693.5 \|
	\| 77000 \| 0.7778 \| 174.3678 \| 61872.8008 \| 1.1757 \| 13.1179 \| 76.232 \| 9.529 \| 10.2459 \| 329784.25 \|
	\| 77500 \| 0.7828 \| 172.8012 \| 60596.3281 \| 1.1760 \| 13.1925 \| 75.801 \| 9.475 \| 10.1671 \| 324287.6875 \|
	\| 78000 \| 0.7879 \| 174.2125 \| 60476.9375 \| 1.1759 \| 13.1527 \| 76.03 \| 9.504 \| 10.3020 \| 324633.9375 \|
	\| 78500 \| 0.7929 \| 173.1697 \| 60255.8711 \| 1.1757 \| 13.142 \| 76.092 \| 9.511 \| 10.2438 \| 328379.5312 \|
	\| 79000 \| 0.7980 \| 174.3070 \| 61248.4961 \| 1.1757 \| 13.1295 \| 76.164 \| 9.521 \| 10.2045 \| 332079.9375 \|
	\| 79500 \| 0.8030 \| 174.3542 \| 61352.0898 \| 1.1758 \| 13.1433 \| 76.084 \| 9.511 \| 10.2553 \| 323423.5312 \|
	\| 80000 \| 0.8081 \| 173.6668 \| 61265.7305 \| 1.1757 \| 13.1664 \| 75.951 \| 9.494 \| 10.2290 \| 334927.125 \|
	\| 80500 \| 0.8131 \| 174.2260 \| 61213.9844 \| 1.1757 \| 13.1433 \| 76.084 \| 9.511 \| 10.1881 \| 336719.0 \|
	\| 81000 \| 0.8182 \| 175.9280 \| 62958.5547 \| 1.1758 \| 13.1561 \| 76.011 \| 9.501 \| 10.2430 \| 347488.4688 \|
	\| 81500 \| 0.8232 \| 174.8141 \| 62516.7031 \| 1.1758 \| 13.0732 \| 76.492 \| 9.562 \| 10.2269 \| 329784.25 \|
	\| 82000 \| 0.8283 \| 174.4420 \| 61907.6836 \| 1.1757 \| 13.2691 \| 75.363 \| 9.42 \| 10.2218 \| 348974.9375 \|
	\| 82500 \| 0.8333 \| 172.8146 \| 58631.5781 \| 1.1761 \| 13.0985 \| 76.345 \| 9.543 \| 10.2506 \| 327155.4375 \|
	\| 83000 \| 0.8384 \| 172.7477 \| 59179.1836 \| 1.1757 \| 13.1399 \| 76.104 \| 9.513 \| 10.1923 \| 317185.5 \|
	\| 83500 \| 0.8434 \| 173.6601 \| 60425.8594 \| 1.1758 \| 13.1157 \| 76.244 \| 9.531 \| 10.2922 \| 318882.4688 \|
	\| 84000 \| 0.8485 \| 173.7408 \| 60835.7578 \| 1.1757 \| 13.1216 \| 76.21 \| 9.526 \| 10.2100 \| 320075.875 \|
	\| 84500 \| 0.8535 \| 173.5928 \| 61768.3281 \| 1.1760 \| 13.1128 \| 76.262 \| 9.533 \| 10.1826 \| 335463.8438 \|
	\| 85000 \| 0.8586 \| 172.5203 \| 60767.2773 \| 1.1758 \| 13.1532 \| 76.027 \| 9.503 \| 10.1818 \| 333678.6562 \|
	\| 85500 \| 0.8636 \| 174.4556 \| 60852.9336 \| 1.1758 \| 13.336 \| 74.985 \| 9.373 \| 10.2320 \| 339062.9688 \|
	\| 86000 \| 0.8687 \| 174.2193 \| 61334.8320 \| 1.1757 \| 13.2466 \| 75.491 \| 9.436 \| 10.2375 \| 324807.0625 \|
	\| 86500 \| 0.8737 \| 172.9083 \| 59379.5938 \| 1.1757 \| 13.1541 \| 76.022 \| 9.503 \| 10.2561 \| 328029.2812 \|
	\| 87000 \| 0.8788 \| 174.4623 \| 61872.8008 \| 1.1757 \| 13.1612 \| 75.981 \| 9.498 \| 10.2303 \| 336719.0 \|
	\| 87500 \| 0.8838 \| 174.3340 \| 61859.7617 \| 1.1757 \| 13.1513 \| 76.038 \| 9.505 \| 10.2654 \| 334927.125 \|
	\| 88000 \| 0.8889 \| 174.1922 \| 62174.2305 \| 1.1757 \| 13.3273 \| 75.034 \| 9.379 \| 10.2290 \| 335106.0312 \|
	\| 88500 \| 0.8939 \| 173.2838 \| 60425.8594 \| 1.1754 \| 13.1058 \| 76.302 \| 9.538 \| 10.1902 \| 329960.4062 \|
	\| 89000 \| 0.8990 \| 174.5435 \| 60801.5391 \| 1.1756 \| 13.0514 \| 76.62 \| 9.578 \| 10.2468 \| 326457.9375 \|
	\| 89500 \| 0.9040 \| 174.0236 \| 61231.2109 \| 1.1756 \| 13.0562 \| 76.592 \| 9.574 \| 10.1982 \| 326980.7812 \|
	\| 90000 \| 0.9091 \| 174.1046 \| 61369.4102 \| 1.1756 \| 13.1257 \| 76.187 \| 9.523 \| 10.2109 \| 329784.25 \|
	\| 90500 \| 0.9141 \| 173.1563 \| 60052.5039 \| 1.1754 \| 13.0825 \| 76.438 \| 9.555 \| 10.2185 \| 320759.75 \|
	\| 91000 \| 0.9192 \| 173.7139 \| 60733.0391 \| 1.1756 \| 13.1096 \| 76.28 \| 9.535 \| 10.2227 \| 324633.9375 \|
	\| 91500 \| 0.9242 \| 174.1315 \| 61872.8008 \| 1.1754 \| 13.2152 \| 75.67 \| 9.459 \| 10.2049 \| 336719.0 \|
	\| 92000 \| 0.9293 \| 174.0372 \| 61490.5039 \| 1.1754 \| 13.1908 \| 75.811 \| 9.476 \| 10.2134 \| 334927.125 \|
	\| 92500 \| 0.9343 \| 173.8215 \| 61386.6797 \| 1.1754 \| 13.1765 \| 75.893 \| 9.487 \| 10.2206 \| 330312.7188 \|
	\| 93000 \| 0.9394 \| 173.9159 \| 61041.7773 \| 1.1756 \| 13.158 \| 75.999 \| 9.5 \| 10.2273 \| 329081.125 \|
	\| 93500 \| 0.9444 \| 173.6062 \| 60870.0586 \| 1.1757 \| 13.2142 \| 75.676 \| 9.46 \| 10.2147 \| 327679.4062 \|
	\| 94000 \| 0.9495 \| 173.7812 \| 61162.2852 \| 1.1758 \| 13.1051 \| 76.306 \| 9.538 \| 10.2172 \| 327068.0938 \|
	\| 94500 \| 0.9545 \| 174.3273 \| 61507.8633 \| 1.1758 \| 13.1923 \| 75.802 \| 9.475 \| 10.2294 \| 329256.9375 \|
	\| 95000 \| 0.9596 \| 173.5390 \| 60664.6172 \| 1.1756 \| 13.0788 \| 76.46 \| 9.557 \| 10.2096 \| 328379.5312 \|
	\| 95500 \| 0.9646 \| 173.6870 \| 60664.6172 \| 1.1756 \| 13.2337 \| 75.565 \| 9.446 \| 10.2252 \| 329256.9375 \|
	\| 96000 \| 0.9697 \| 173.6735 \| 60698.8164 \| 1.1755 \| 13.1813 \| 75.865 \| 9.483 \| 10.2117 \| 328204.5 \|
	\| 96500 \| 0.9747 \| 173.9765 \| 60955.8594 \| 1.1755 \| 13.1412 \| 76.096 \| 9.512 \| 10.2307 \| 328905.75 \|
	\| 97000 \| 0.9798 \| 173.8215 \| 60835.7578 \| 1.1755 \| 13.2784 \| 75.31 \| 9.414 \| 10.2227 \| 327854.4375 \|
	\| 97500 \| 0.9848 \| 173.8215 \| 60887.2422 \| 1.1755 \| 13.0893 \| 76.398 \| 9.55 \| 10.2176 \| 327854.4375 \|
	\| 98000 \| 0.9899 \| 173.8215 \| 60852.9336 \| 1.1755 \| 13.1604 \| 75.986 \| 9.498 \| 10.2142 \| 328029.2812 \|
	\| 98500 \| 0.9949 \| 173.8350 \| 60887.2422 \| 1.1755 \| 13.2337 \| 75.565 \| 9.446 \| 10.2239 \| 328379.5312 \|
	\| 99000 \| 1.0 \| 173.8350 \| 60887.2422 \| 1.1755 \| 13.1746 \| 75.903 \| 9.488 \| 10.2239 \| 328379.5312 \|

	### Framework versions
	- Distily 0.2.0
	- Transformers 4.44.0
	- Pytorch 2.3.0
	- Datasets 2.20.0