AlexWang99/4d-addition-3000

byt5_3k_4d

This model is a fine-tuned version of google/byt5-small on an unknown dataset. It achieves the following results on the evaluation set:

• Loss: 0.0348

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 5e-05
• train_batch_size: 800
• eval_batch_size: 800
• seed: 42
• optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
• lr_scheduler_type: linear
• num_epochs: 100

Training results

Training Loss	Epoch	Step	Validation Loss
No log	1.0	4	0.4702
No log	2.0	8	0.4366
0.8301	3.0	12	0.4202
0.8301	4.0	16	0.3843
0.7703	5.0	20	0.3898
0.7703	6.0	24	0.3573
0.7703	7.0	28	0.3422
0.7169	8.0	32	0.3304
0.7169	9.0	36	0.3049
0.6727	10.0	40	0.3067
0.6727	11.0	44	0.2965
0.6727	12.0	48	0.2693
0.6394	13.0	52	0.2711
0.6394	14.0	56	0.2561
0.6047	15.0	60	0.2454
0.6047	16.0	64	0.2368
0.6047	17.0	68	0.2250
0.565	18.0	72	0.2110
0.565	19.0	76	0.2109
0.5368	20.0	80	0.1950
0.5368	21.0	84	0.1974
0.5368	22.0	88	0.1819
0.518	23.0	92	0.1795
0.518	24.0	96	0.1648
0.4862	25.0	100	0.1675
0.4862	26.0	104	0.1550
0.4862	27.0	108	0.1530
0.4628	28.0	112	0.1447
0.4628	29.0	116	0.1442
0.4408	30.0	120	0.1310
0.4408	31.0	124	0.1336
0.4408	32.0	128	0.1235
0.4192	33.0	132	0.1191
0.4192	34.0	136	0.1193
0.4133	35.0	140	0.1123
0.4133	36.0	144	0.1156
0.4133	37.0	148	0.1051
0.3922	38.0	152	0.0999
0.3922	39.0	156	0.0991
0.3778	40.0	160	0.0995
0.3778	41.0	164	0.0912
0.3778	42.0	168	0.0903
0.3655	43.0	172	0.0841
0.3655	44.0	176	0.0790
0.3526	45.0	180	0.0827
0.3526	46.0	184	0.0756
0.3526	47.0	188	0.0747
0.3378	48.0	192	0.0737
0.3378	49.0	196	0.0747
0.3308	50.0	200	0.0729
0.3308	51.0	204	0.0665
0.3308	52.0	208	0.0663
0.321	53.0	212	0.0642
0.321	54.0	216	0.0640
0.3084	55.0	220	0.0632
0.3084	56.0	224	0.0599
0.3084	57.0	228	0.0580
0.2967	58.0	232	0.0567
0.2967	59.0	236	0.0525
0.2928	60.0	240	0.0522
0.2928	61.0	244	0.0536
0.2928	62.0	248	0.0524
0.2929	63.0	252	0.0568
0.2929	64.0	256	0.0530
0.283	65.0	260	0.0476
0.283	66.0	264	0.0479
0.283	67.0	268	0.0507
0.2766	68.0	272	0.0461
0.2766	69.0	276	0.0444
0.2677	70.0	280	0.0456
0.2677	71.0	284	0.0437
0.2677	72.0	288	0.0428
0.2614	73.0	292	0.0419
0.2614	74.0	296	0.0414
0.2595	75.0	300	0.0418
0.2595	76.0	304	0.0412
0.2595	77.0	308	0.0396
0.2582	78.0	312	0.0382
0.2582	79.0	316	0.0381
0.2511	80.0	320	0.0387
0.2511	81.0	324	0.0388
0.2511	82.0	328	0.0372
0.2481	83.0	332	0.0360
0.2481	84.0	336	0.0366
0.2474	85.0	340	0.0365
0.2474	86.0	344	0.0357
0.2474	87.0	348	0.0355
0.2537	88.0	352	0.0360
0.2537	89.0	356	0.0359
0.2438	90.0	360	0.0355
0.2438	91.0	364	0.0353
0.2438	92.0	368	0.0349
0.2461	93.0	372	0.0343
0.2461	94.0	376	0.0342
0.2395	95.0	380	0.0344
0.2395	96.0	384	0.0347
0.2395	97.0	388	0.0350
0.2439	98.0	392	0.0349
0.2439	99.0	396	0.0348
0.2432	100.0	400	0.0348

Framework versions

• Transformers 4.35.2
• Pytorch 2.2.1+cu121
• Datasets 2.18.0
• Tokenizers 0.15.2