End of training

8eac14b verified 3 months ago

7.61 kB

	---
	base_model: gpt2
	library_name: Distily
	license: mit
	tags:
	- generated_from_trainer
	model-index:
	- name: distily_bench_gpt2_attn_part_2
	results: []
	---

	# distily_bench_gpt2_attn_part_2

	This student model is distilled from the teacher model [gpt2](https://huggingface.co/gpt2) using the dataset (unspecified).

	The [Distily](https://github.com/lapp0/distily) library was used for this distillation.

	It achieves the following results on the evaluation set:
	- eval_enwikippl: 234.3043
	- eval_frwikippl: 1329.5667
	- eval_zhwikippl: 575.8531
	- eval_loss: 2.4344
	- eval_runtime: 87.576
	- eval_samples_per_second: 57.093
	- eval_steps_per_second: 7.137

	<!-- This model card has been generated automatically according to the information the Trainer had access to. You
	should probably proofread and complete it, then remove this comment.

	## Model description

	More information needed

	## Intended uses & limitations

	More information needed

	## Training and evaluation data

	More information needed
	-->

	## Training procedure

	### Training hyperparameters

	The following hyperparameters were used during training:
	- distillation_objective: DistillationObjective(logits_loss_component=LossComponent(label=logits, weight=1, loss_fn=kl, layer_mapper=None, projector=None), hs_loss_component=LossComponent(label=hs, weight=0, loss_fn=None, layer_mapper=None, projector=None), attn_loss_component=LossComponent(label=attn, weight=2.0, loss_fn=cos, layer_mapper=None, projector=None))
	- train_embeddings: True
	- learning_rate: 4e-05
	- train_batch_size: 8
	- eval_batch_size: 8
	- seed: 42
	- optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
	- lr_scheduler_type: constant
	- num_epochs: 1.0

	### Resource Usage
	Peak GPU Memory: 8.2206 GB

	### Eval-Phase Metrics
	\| step \| epoch \| enwikippl \| frwikippl \| loss \| runtime \| samples_per_second \| steps_per_second \| zhwikippl \|
	\| --- \| --- \| --- \| --- \| --- \| --- \| --- \| --- \| --- \|
	\| teacher eval \| \| 30.2086 \| 57.2728 \| \| \| \| \| 18.1784 \|
	\| 0 \| 0 \| 56314.7695 \| 59887.2773 \| 7.8201 \| 86.1883 \| 58.013 \| 7.252 \| 59033.8086 \|
	\| 1000 \| 0.0162 \| 792.2872 \| 4804.1196 \| 3.2178 \| 86.0729 \| 58.09 \| 7.261 \| 14971.4619 \|
	\| 2000 \| 0.0323 \| 563.5076 \| 3594.0178 \| 3.0017 \| 86.8465 \| 57.573 \| 7.197 \| 2436.5176 \|
	\| 3000 \| 0.0485 \| 462.8162 \| 3038.3840 \| 2.8886 \| 86.4026 \| 57.869 \| 7.234 \| 990.3070 \|
	\| 4000 \| 0.0646 \| 394.8591 \| 2549.8823 \| 2.7877 \| 86.3461 \| 57.907 \| 7.238 \| 1110.5270 \|
	\| 5000 \| 0.0808 \| 353.7159 \| 2113.5315 \| 2.6983 \| 86.5685 \| 57.758 \| 7.22 \| 845.9332 \|
	\| 6000 \| 0.0970 \| 305.3852 \| 1907.5966 \| 2.6161 \| 86.5704 \| 57.756 \| 7.22 \| 792.3533 \|
	\| 7000 \| 0.1131 \| 275.6120 \| 1710.8368 \| 2.5482 \| 89.5827 \| 55.814 \| 6.977 \| 704.9745 \|
	\| 8000 \| 0.1293 \| 248.5870 \| 1491.7041 \| 2.4852 \| 87.8996 \| 56.883 \| 7.11 \| 655.4906 \|
	\| 9000 \| 0.1455 \| 234.3043 \| 1329.5667 \| 2.4344 \| 87.576 \| 57.093 \| 7.137 \| 575.8531 \|
	\| 10000 \| 0.1616 \| 210.1025 \| 1200.8650 \| 2.3778 \| 88.2437 \| 56.661 \| 7.083 \| 680.8273 \|
	\| 11000 \| 0.1778 \| 195.9659 \| 1137.0879 \| 2.3352 \| 87.2617 \| 57.299 \| 7.162 \| 574.3172 \|
	\| 12000 \| 0.1939 \| 177.5484 \| 986.7017 \| 2.2840 \| 88.3708 \| 56.58 \| 7.072 \| 511.2562 \|
	\| 13000 \| 0.2101 \| 168.3902 \| 992.2828 \| 2.2496 \| 86.6049 \| 57.733 \| 7.217 \| 493.3487 \|
	\| 14000 \| 0.2263 \| 159.1225 \| 889.1183 \| 2.2152 \| 86.6651 \| 57.693 \| 7.212 \| 434.3372 \|
	\| 15000 \| 0.2424 \| 153.0509 \| 800.1130 \| 2.1876 \| 86.8087 \| 57.598 \| 7.2 \| 389.5484 \|
	\| 16000 \| 0.2586 \| 146.4697 \| 801.1292 \| 2.1678 \| 86.6505 \| 57.703 \| 7.213 \| 490.2618 \|
	\| 17000 \| 0.2747 \| 143.1525 \| 782.1525 \| 2.1519 \| 86.9013 \| 57.537 \| 7.192 \| 536.9359 \|
	\| 18000 \| 0.2909 \| 139.4116 \| 832.3362 \| 2.1366 \| 86.6158 \| 57.726 \| 7.216 \| 568.2902 \|
	\| 19000 \| 0.3071 \| 134.7601 \| 733.2869 \| 2.1223 \| 86.3964 \| 57.873 \| 7.234 \| 516.8853 \|
	\| 20000 \| 0.3232 \| 132.8793 \| 726.0842 \| 2.1108 \| 86.4694 \| 57.824 \| 7.228 \| 376.9092 \|
	\| 21000 \| 0.3394 \| 130.5677 \| 658.1619 \| 2.0982 \| 86.9266 \| 57.52 \| 7.19 \| 386.2850 \|
	\| 22000 \| 0.3556 \| 130.3043 \| 657.9764 \| 2.0894 \| 87.5122 \| 57.135 \| 7.142 \| 418.9560 \|
	\| 23000 \| 0.3717 \| 128.9555 \| 687.2317 \| 2.0831 \| 87.0863 \| 57.414 \| 7.177 \| 419.0120 \|
	\| 24000 \| 0.3879 \| 125.7514 \| 657.8835 \| 2.0732 \| 86.5642 \| 57.761 \| 7.22 \| 391.6348 \|
	\| 25000 \| 0.4040 \| 124.6431 \| 676.3678 \| 2.0703 \| 86.6818 \| 57.682 \| 7.21 \| 384.3811 \|
	\| 26000 \| 0.4202 \| 124.4787 \| 653.3537 \| 2.0597 \| 87.1964 \| 57.342 \| 7.168 \| 403.9578 \|
	\| 27000 \| 0.4364 \| 122.7223 \| 659.5090 \| 2.0546 \| 86.5911 \| 57.743 \| 7.218 \| 316.7159 \|
	\| 28000 \| 0.4525 \| 123.1997 \| 631.8794 \| 2.0501 \| 86.9662 \| 57.494 \| 7.187 \| 295.1540 \|
	\| 29000 \| 0.4687 \| 122.9990 \| 659.6486 \| 2.0423 \| 87.0187 \| 57.459 \| 7.182 \| 310.8498 \|
	\| 30000 \| 0.4848 \| 123.1041 \| 617.8698 \| 2.0464 \| 87.3158 \| 57.263 \| 7.158 \| 315.3653 \|
	\| 31000 \| 0.5010 \| 120.1201 \| 613.0962 \| 2.0351 \| 87.311 \| 57.267 \| 7.158 \| 308.4515 \|
	\| 32000 \| 0.5172 \| 119.4781 \| 630.6332 \| 2.0323 \| 87.0327 \| 57.45 \| 7.181 \| 274.0701 \|
	\| 33000 \| 0.5333 \| 118.0396 \| 624.0867 \| 2.0277 \| 86.8174 \| 57.592 \| 7.199 \| 285.6919 \|
	\| 34000 \| 0.5495 \| 119.1354 \| 588.2814 \| 2.0249 \| 86.7837 \| 57.614 \| 7.202 \| 316.9274 \|
	\| 35000 \| 0.5657 \| 117.2539 \| 567.9440 \| 2.0212 \| 87.7979 \| 56.949 \| 7.119 \| 303.2645 \|
	\| 36000 \| 0.5818 \| 117.5548 \| 608.1882 \| 2.0241 \| 86.4044 \| 57.867 \| 7.233 \| 256.7451 \|
	\| 37000 \| 0.5980 \| 116.6455 \| 600.1797 \| 2.0164 \| 87.1429 \| 57.377 \| 7.172 \| 345.4349 \|
	\| 38000 \| 0.6141 \| 116.6998 \| 564.5506 \| 2.0140 \| 87.5854 \| 57.087 \| 7.136 \| 283.1472 \|
	\| 39000 \| 0.6303 \| 113.8975 \| 538.0084 \| 2.0085 \| 86.8118 \| 57.596 \| 7.199 \| 285.9208 \|
	\| 40000 \| 0.6465 \| 115.6533 \| 579.6762 \| 2.0130 \| 87.6272 \| 57.06 \| 7.132 \| 268.8502 \|
	\| 41000 \| 0.6626 \| 114.0037 \| 569.7087 \| 2.0107 \| 86.8951 \| 57.541 \| 7.193 \| 298.6428 \|
	\| 42000 \| 0.6788 \| 114.4206 \| 558.2177 \| 2.0114 \| 86.4056 \| 57.867 \| 7.233 \| 325.7667 \|
	\| 43000 \| 0.6949 \| 114.1277 \| 570.9554 \| 2.0067 \| 86.7633 \| 57.628 \| 7.204 \| 297.6475 \|
	\| 44000 \| 0.7111 \| 112.6310 \| 603.2343 \| 2.0041 \| 87.2036 \| 57.337 \| 7.167 \| 265.9578 \|
	\| 45000 \| 0.7273 \| 112.3951 \| 582.9551 \| 1.9978 \| 86.1934 \| 58.009 \| 7.251 \| 276.3855 \|
	\| 46000 \| 0.7434 \| 112.9463 \| 591.3171 \| 1.9976 \| 86.5934 \| 57.741 \| 7.218 \| 270.1097 \|
	\| 47000 \| 0.7596 \| 112.1510 \| 564.6300 \| 1.9943 \| 86.827 \| 57.586 \| 7.198 \| 323.6853 \|
	\| 48000 \| 0.7758 \| 112.8236 \| 513.6188 \| 1.9968 \| 86.3974 \| 57.872 \| 7.234 \| 305.2553 \|
	\| 49000 \| 0.7919 \| 112.7886 \| 565.1480 \| 1.9948 \| 86.5571 \| 57.765 \| 7.221 \| 276.0167 \|
	\| 50000 \| 0.8081 \| 111.8900 \| 592.2350 \| 1.9932 \| 86.9533 \| 57.502 \| 7.188 \| 247.9840 \|
	\| 51000 \| 0.8242 \| 111.4391 \| 588.3229 \| 1.9920 \| 86.4566 \| 57.832 \| 7.229 \| 298.2842 \|
	\| 52000 \| 0.8404 \| 109.9350 \| 549.1997 \| 1.9904 \| 86.5867 \| 57.746 \| 7.218 \| 318.3695 \|
	\| 53000 \| 0.8566 \| 110.8264 \| 544.7263 \| 1.9856 \| 87.1758 \| 57.355 \| 7.169 \| 311.5147 \|
	\| 54000 \| 0.8727 \| 111.0849 \| 544.9952 \| 1.9857 \| 87.388 \| 57.216 \| 7.152 \| 334.9867 \|
	\| 55000 \| 0.8889 \| 111.1799 \| 602.7242 \| 1.9865 \| 86.8376 \| 57.579 \| 7.197 \| 265.1776 \|
	\| 56000 \| 0.9051 \| 111.2490 \| 553.5536 \| 1.9821 \| 87.5367 \| 57.119 \| 7.14 \| 326.5943 \|
	\| 57000 \| 0.9212 \| 110.1914 \| 582.8317 \| 1.9870 \| 87.0656 \| 57.428 \| 7.178 \| 1162.1104 \|
	\| 58000 \| 0.9374 \| 109.1016 \| 657.6982 \| 1.9860 \| 86.6253 \| 57.72 \| 7.215 \| 322.6926 \|
	\| 59000 \| 0.9535 \| 111.8119 \| 596.5937 \| 1.9831 \| 86.3097 \| 57.931 \| 7.241 \| 408.6782 \|
	\| 60000 \| 0.9697 \| 108.9746 \| 586.0871 \| 1.9748 \| 86.6963 \| 57.673 \| 7.209 \| 268.2405 \|
	\| 61000 \| 0.9859 \| 109.9862 \| 560.1890 \| 1.9777 \| 86.8734 \| 57.555 \| 7.194 \| 294.4846 \|
	\| 61875 \| 1.0 \| 110.1401 \| 573.6587 \| 1.9760 \| 86.766 \| 57.626 \| 7.203 \| 273.4851 \|

	### Framework versions
	- Distily 0.2.0
	- Transformers 4.44.0
	- Pytorch 2.3.0
	- Datasets 2.20.0

	---
	base_model: gpt2
	library_name: Distily
	license: mit
	tags:
	- generated_from_trainer
	model-index:
	- name: distily_bench_gpt2_attn_part_2
	results: []
	---

	# distily_bench_gpt2_attn_part_2

	This student model is distilled from the teacher model [gpt2](https://huggingface.co/gpt2) using the dataset (unspecified).

	The [Distily](https://github.com/lapp0/distily) library was used for this distillation.

	It achieves the following results on the evaluation set:
	- eval_enwikippl: 234.3043
	- eval_frwikippl: 1329.5667
	- eval_zhwikippl: 575.8531
	- eval_loss: 2.4344
	- eval_runtime: 87.576
	- eval_samples_per_second: 57.093
	- eval_steps_per_second: 7.137

	<!-- This model card has been generated automatically according to the information the Trainer had access to. You
	should probably proofread and complete it, then remove this comment.

	## Model description

	More information needed

	## Intended uses & limitations

	More information needed

	## Training and evaluation data

	More information needed
	-->

	## Training procedure

	### Training hyperparameters

	The following hyperparameters were used during training:
	- distillation_objective: DistillationObjective(logits_loss_component=LossComponent(label=logits, weight=1, loss_fn=kl, layer_mapper=None, projector=None), hs_loss_component=LossComponent(label=hs, weight=0, loss_fn=None, layer_mapper=None, projector=None), attn_loss_component=LossComponent(label=attn, weight=2.0, loss_fn=cos, layer_mapper=None, projector=None))
	- train_embeddings: True
	- learning_rate: 4e-05
	- train_batch_size: 8
	- eval_batch_size: 8
	- seed: 42
	- optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
	- lr_scheduler_type: constant
	- num_epochs: 1.0

	### Resource Usage
	Peak GPU Memory: 8.2206 GB

	### Eval-Phase Metrics
	\| step \| epoch \| enwikippl \| frwikippl \| loss \| runtime \| samples_per_second \| steps_per_second \| zhwikippl \|
	\| --- \| --- \| --- \| --- \| --- \| --- \| --- \| --- \| --- \|
	\| teacher eval \| \| 30.2086 \| 57.2728 \| \| \| \| \| 18.1784 \|
	\| 0 \| 0 \| 56314.7695 \| 59887.2773 \| 7.8201 \| 86.1883 \| 58.013 \| 7.252 \| 59033.8086 \|
	\| 1000 \| 0.0162 \| 792.2872 \| 4804.1196 \| 3.2178 \| 86.0729 \| 58.09 \| 7.261 \| 14971.4619 \|
	\| 2000 \| 0.0323 \| 563.5076 \| 3594.0178 \| 3.0017 \| 86.8465 \| 57.573 \| 7.197 \| 2436.5176 \|
	\| 3000 \| 0.0485 \| 462.8162 \| 3038.3840 \| 2.8886 \| 86.4026 \| 57.869 \| 7.234 \| 990.3070 \|
	\| 4000 \| 0.0646 \| 394.8591 \| 2549.8823 \| 2.7877 \| 86.3461 \| 57.907 \| 7.238 \| 1110.5270 \|
	\| 5000 \| 0.0808 \| 353.7159 \| 2113.5315 \| 2.6983 \| 86.5685 \| 57.758 \| 7.22 \| 845.9332 \|
	\| 6000 \| 0.0970 \| 305.3852 \| 1907.5966 \| 2.6161 \| 86.5704 \| 57.756 \| 7.22 \| 792.3533 \|
	\| 7000 \| 0.1131 \| 275.6120 \| 1710.8368 \| 2.5482 \| 89.5827 \| 55.814 \| 6.977 \| 704.9745 \|
	\| 8000 \| 0.1293 \| 248.5870 \| 1491.7041 \| 2.4852 \| 87.8996 \| 56.883 \| 7.11 \| 655.4906 \|
	\| 9000 \| 0.1455 \| 234.3043 \| 1329.5667 \| 2.4344 \| 87.576 \| 57.093 \| 7.137 \| 575.8531 \|
	\| 10000 \| 0.1616 \| 210.1025 \| 1200.8650 \| 2.3778 \| 88.2437 \| 56.661 \| 7.083 \| 680.8273 \|
	\| 11000 \| 0.1778 \| 195.9659 \| 1137.0879 \| 2.3352 \| 87.2617 \| 57.299 \| 7.162 \| 574.3172 \|
	\| 12000 \| 0.1939 \| 177.5484 \| 986.7017 \| 2.2840 \| 88.3708 \| 56.58 \| 7.072 \| 511.2562 \|
	\| 13000 \| 0.2101 \| 168.3902 \| 992.2828 \| 2.2496 \| 86.6049 \| 57.733 \| 7.217 \| 493.3487 \|
	\| 14000 \| 0.2263 \| 159.1225 \| 889.1183 \| 2.2152 \| 86.6651 \| 57.693 \| 7.212 \| 434.3372 \|
	\| 15000 \| 0.2424 \| 153.0509 \| 800.1130 \| 2.1876 \| 86.8087 \| 57.598 \| 7.2 \| 389.5484 \|
	\| 16000 \| 0.2586 \| 146.4697 \| 801.1292 \| 2.1678 \| 86.6505 \| 57.703 \| 7.213 \| 490.2618 \|
	\| 17000 \| 0.2747 \| 143.1525 \| 782.1525 \| 2.1519 \| 86.9013 \| 57.537 \| 7.192 \| 536.9359 \|
	\| 18000 \| 0.2909 \| 139.4116 \| 832.3362 \| 2.1366 \| 86.6158 \| 57.726 \| 7.216 \| 568.2902 \|
	\| 19000 \| 0.3071 \| 134.7601 \| 733.2869 \| 2.1223 \| 86.3964 \| 57.873 \| 7.234 \| 516.8853 \|
	\| 20000 \| 0.3232 \| 132.8793 \| 726.0842 \| 2.1108 \| 86.4694 \| 57.824 \| 7.228 \| 376.9092 \|
	\| 21000 \| 0.3394 \| 130.5677 \| 658.1619 \| 2.0982 \| 86.9266 \| 57.52 \| 7.19 \| 386.2850 \|
	\| 22000 \| 0.3556 \| 130.3043 \| 657.9764 \| 2.0894 \| 87.5122 \| 57.135 \| 7.142 \| 418.9560 \|
	\| 23000 \| 0.3717 \| 128.9555 \| 687.2317 \| 2.0831 \| 87.0863 \| 57.414 \| 7.177 \| 419.0120 \|
	\| 24000 \| 0.3879 \| 125.7514 \| 657.8835 \| 2.0732 \| 86.5642 \| 57.761 \| 7.22 \| 391.6348 \|
	\| 25000 \| 0.4040 \| 124.6431 \| 676.3678 \| 2.0703 \| 86.6818 \| 57.682 \| 7.21 \| 384.3811 \|
	\| 26000 \| 0.4202 \| 124.4787 \| 653.3537 \| 2.0597 \| 87.1964 \| 57.342 \| 7.168 \| 403.9578 \|
	\| 27000 \| 0.4364 \| 122.7223 \| 659.5090 \| 2.0546 \| 86.5911 \| 57.743 \| 7.218 \| 316.7159 \|
	\| 28000 \| 0.4525 \| 123.1997 \| 631.8794 \| 2.0501 \| 86.9662 \| 57.494 \| 7.187 \| 295.1540 \|
	\| 29000 \| 0.4687 \| 122.9990 \| 659.6486 \| 2.0423 \| 87.0187 \| 57.459 \| 7.182 \| 310.8498 \|
	\| 30000 \| 0.4848 \| 123.1041 \| 617.8698 \| 2.0464 \| 87.3158 \| 57.263 \| 7.158 \| 315.3653 \|
	\| 31000 \| 0.5010 \| 120.1201 \| 613.0962 \| 2.0351 \| 87.311 \| 57.267 \| 7.158 \| 308.4515 \|
	\| 32000 \| 0.5172 \| 119.4781 \| 630.6332 \| 2.0323 \| 87.0327 \| 57.45 \| 7.181 \| 274.0701 \|
	\| 33000 \| 0.5333 \| 118.0396 \| 624.0867 \| 2.0277 \| 86.8174 \| 57.592 \| 7.199 \| 285.6919 \|
	\| 34000 \| 0.5495 \| 119.1354 \| 588.2814 \| 2.0249 \| 86.7837 \| 57.614 \| 7.202 \| 316.9274 \|
	\| 35000 \| 0.5657 \| 117.2539 \| 567.9440 \| 2.0212 \| 87.7979 \| 56.949 \| 7.119 \| 303.2645 \|
	\| 36000 \| 0.5818 \| 117.5548 \| 608.1882 \| 2.0241 \| 86.4044 \| 57.867 \| 7.233 \| 256.7451 \|
	\| 37000 \| 0.5980 \| 116.6455 \| 600.1797 \| 2.0164 \| 87.1429 \| 57.377 \| 7.172 \| 345.4349 \|
	\| 38000 \| 0.6141 \| 116.6998 \| 564.5506 \| 2.0140 \| 87.5854 \| 57.087 \| 7.136 \| 283.1472 \|
	\| 39000 \| 0.6303 \| 113.8975 \| 538.0084 \| 2.0085 \| 86.8118 \| 57.596 \| 7.199 \| 285.9208 \|
	\| 40000 \| 0.6465 \| 115.6533 \| 579.6762 \| 2.0130 \| 87.6272 \| 57.06 \| 7.132 \| 268.8502 \|
	\| 41000 \| 0.6626 \| 114.0037 \| 569.7087 \| 2.0107 \| 86.8951 \| 57.541 \| 7.193 \| 298.6428 \|
	\| 42000 \| 0.6788 \| 114.4206 \| 558.2177 \| 2.0114 \| 86.4056 \| 57.867 \| 7.233 \| 325.7667 \|
	\| 43000 \| 0.6949 \| 114.1277 \| 570.9554 \| 2.0067 \| 86.7633 \| 57.628 \| 7.204 \| 297.6475 \|
	\| 44000 \| 0.7111 \| 112.6310 \| 603.2343 \| 2.0041 \| 87.2036 \| 57.337 \| 7.167 \| 265.9578 \|
	\| 45000 \| 0.7273 \| 112.3951 \| 582.9551 \| 1.9978 \| 86.1934 \| 58.009 \| 7.251 \| 276.3855 \|
	\| 46000 \| 0.7434 \| 112.9463 \| 591.3171 \| 1.9976 \| 86.5934 \| 57.741 \| 7.218 \| 270.1097 \|
	\| 47000 \| 0.7596 \| 112.1510 \| 564.6300 \| 1.9943 \| 86.827 \| 57.586 \| 7.198 \| 323.6853 \|
	\| 48000 \| 0.7758 \| 112.8236 \| 513.6188 \| 1.9968 \| 86.3974 \| 57.872 \| 7.234 \| 305.2553 \|
	\| 49000 \| 0.7919 \| 112.7886 \| 565.1480 \| 1.9948 \| 86.5571 \| 57.765 \| 7.221 \| 276.0167 \|
	\| 50000 \| 0.8081 \| 111.8900 \| 592.2350 \| 1.9932 \| 86.9533 \| 57.502 \| 7.188 \| 247.9840 \|
	\| 51000 \| 0.8242 \| 111.4391 \| 588.3229 \| 1.9920 \| 86.4566 \| 57.832 \| 7.229 \| 298.2842 \|
	\| 52000 \| 0.8404 \| 109.9350 \| 549.1997 \| 1.9904 \| 86.5867 \| 57.746 \| 7.218 \| 318.3695 \|
	\| 53000 \| 0.8566 \| 110.8264 \| 544.7263 \| 1.9856 \| 87.1758 \| 57.355 \| 7.169 \| 311.5147 \|
	\| 54000 \| 0.8727 \| 111.0849 \| 544.9952 \| 1.9857 \| 87.388 \| 57.216 \| 7.152 \| 334.9867 \|
	\| 55000 \| 0.8889 \| 111.1799 \| 602.7242 \| 1.9865 \| 86.8376 \| 57.579 \| 7.197 \| 265.1776 \|
	\| 56000 \| 0.9051 \| 111.2490 \| 553.5536 \| 1.9821 \| 87.5367 \| 57.119 \| 7.14 \| 326.5943 \|
	\| 57000 \| 0.9212 \| 110.1914 \| 582.8317 \| 1.9870 \| 87.0656 \| 57.428 \| 7.178 \| 1162.1104 \|
	\| 58000 \| 0.9374 \| 109.1016 \| 657.6982 \| 1.9860 \| 86.6253 \| 57.72 \| 7.215 \| 322.6926 \|
	\| 59000 \| 0.9535 \| 111.8119 \| 596.5937 \| 1.9831 \| 86.3097 \| 57.931 \| 7.241 \| 408.6782 \|
	\| 60000 \| 0.9697 \| 108.9746 \| 586.0871 \| 1.9748 \| 86.6963 \| 57.673 \| 7.209 \| 268.2405 \|
	\| 61000 \| 0.9859 \| 109.9862 \| 560.1890 \| 1.9777 \| 86.8734 \| 57.555 \| 7.194 \| 294.4846 \|
	\| 61875 \| 1.0 \| 110.1401 \| 573.6587 \| 1.9760 \| 86.766 \| 57.626 \| 7.203 \| 273.4851 \|

	### Framework versions
	- Distily 0.2.0
	- Transformers 4.44.0
	- Pytorch 2.3.0
	- Datasets 2.20.0