README.md

metadata

license: bsd-3-clause
tags:
  - generated_from_trainer
metrics:
  - rouge
  - bleu
model-index:
  - name: CommitPredictorT5
    results: []

CommitPredictorT5

This model is a fine-tuned version of Salesforce/codet5-base-multi-sum on the None dataset. It achieves the following results on the evaluation set:

• Loss: 2.7376
• Rouge1: 0.3735
• Rouge2: 0.1504
• Rougel: 0.3701
• Rougelsum: 0.3697
• Gen Len: 19.1365
• Bleu: 0.1453

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 2e-05
• train_batch_size: 32
• eval_batch_size: 32
• seed: 42
• optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
• lr_scheduler_type: linear
• num_epochs: 100

Training results

Training Loss	Epoch	Step	Validation Loss	Rouge1	Rouge2	Rougel	Rougelsum	Gen Len	Bleu
3.2138	1.0	687	2.6919	0.3246	0.0964	0.3216	0.3218	9.125	0.0937
2.7517	2.0	1374	2.5546	0.3394	0.1042	0.3364	0.3362	8.9803	0.1066
2.4305	3.0	2061	2.4836	0.3523	0.1136	0.3496	0.3496	9.2049	0.1123
2.2956	4.0	2748	2.4483	0.3658	0.126	0.3633	0.3633	9.525	0.1146
2.1888	5.0	3435	2.4312	0.3665	0.1332	0.3636	0.3634	10.0631	0.1253
2.0056	6.0	4122	2.4251	0.3674	0.1352	0.3646	0.3644	9.8365	0.1240
1.9128	7.0	4809	2.4289	0.3725	0.1431	0.3694	0.3694	9.8713	0.1295
1.8487	8.0	5496	2.4625	0.3683	0.1377	0.3657	0.3659	10.0947	0.1291
1.6726	9.0	6183	2.4643	0.3725	0.1449	0.3702	0.3697	13.1967	0.1325
1.6292	10.0	6870	2.4716	0.3688	0.1438	0.3664	0.3661	13.7656	0.1307
1.5025	11.0	7557	2.4974	0.3762	0.1494	0.3732	0.3732	14.0988	0.1391
1.4224	12.0	8244	2.5273	0.3723	0.1489	0.3692	0.3688	12.084	0.1388
1.3912	13.0	8931	2.5460	0.375	0.1499	0.3722	0.3723	13.9529	0.1399
1.2766	14.0	9618	2.5771	0.3698	0.1453	0.3668	0.3667	12.5102	0.1386
1.2188	15.0	10305	2.6005	0.3789	0.1493	0.3763	0.376	16.0545	0.1423
1.1779	16.0	10992	2.6296	0.3757	0.1497	0.3729	0.3723	14.7201	0.1423
1.0739	17.0	11679	2.6512	0.3749	0.1522	0.3717	0.3715	16.0008	0.1468
1.0408	18.0	12366	2.6792	0.3758	0.1494	0.3735	0.3733	18.4971	0.1465
0.9567	19.0	13053	2.7153	0.3695	0.144	0.3669	0.3667	16.2357	0.1435
0.9072	20.0	13740	2.7376	0.3735	0.1504	0.3701	0.3697	19.1365	0.1453

Framework versions

• Transformers 4.25.1
• Pytorch 1.13.0+cu117
• Datasets 2.7.1
• Tokenizers 0.13.2