CommitPredictorT5

This model is a fine-tuned version of Salesforce/codet5-base-multi-sum on the None dataset. It achieves the following results on the evaluation set:

• Loss: 2.4669
• Bleu: 0.0002
• Precisions: [0.003189792663476874, 0.00016826518593303046, 0.000321853878339234, 0.0036900369003690036]
• Brevity Penalty: 0.2394
• Length Ratio: 0.4116
• Translation Length: 10658
• Reference Length: 25896

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 2e-05
• train_batch_size: 42
• eval_batch_size: 42
• seed: 42
• gradient_accumulation_steps: 3
• total_train_batch_size: 126
• optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
• lr_scheduler_type: linear
• num_epochs: 100

Training results

Training Loss	Epoch	Step	Validation Loss	Bleu	Precisions	Brevity Penalty	Length Ratio	Translation Length	Reference Length
No log	1.0	299	2.8109	0.0002	[0.003640040444893832, 0.00019327406262079628, 0.0003745318352059925, 0.006024096385542169]	0.1982	0.3819	9889	25896
3.1102	2.0	598	2.6662	0.0002	[0.004371150407311742, 0.00018691588785046728, 0.00036114120621162876, 0.005319148936170213]	0.2074	0.3887	10065	25896
3.1102	3.0	897	2.5869	0.0002	[0.0033418517790446234, 0.00018321729571271528, 0.0003546099290780142, 0.005494505494505495]	0.2132	0.3928	10173	25896
2.6696	4.0	1196	2.5371	0.0002	[0.0033398821218074658, 0.00018301610541727673, 0.0003522367030644593, 0.004672897196261682]	0.2135	0.3931	10179	25896
2.6696	5.0	1495	2.5077	0.0002	[0.003243655790879603, 0.0001734304543877905, 0.0003356831151393085, 0.005208333333333333]	0.2298	0.4047	10481	25896
2.4738	6.0	1794	2.4810	0.0002	[0.0029016345874842827, 0.00017784101013693757, 0.00034234851078397807, 0.0045662100456621]	0.2220	0.3992	10338	25896
2.3139	7.0	2093	2.4625	0.0002	[0.002756130013305455, 0.0001722356183258698, 0.00033101621979476995, 0.00423728813559322]	0.2319	0.4063	10521	25896
2.3139	8.0	2392	2.4556	0.0002	[0.0027348170501697473, 0.00016983695652173913, 0.0003266906239790918, 0.004273504273504274]	0.2364	0.4094	10603	25896
2.1842	9.0	2691	2.4470	0.0002	[0.003198193961057285, 0.000169061707523246, 0.00032658393207054214, 0.004784688995215311]	0.2378	0.4105	10630	25896
2.1842	10.0	2990	2.4439	0.0002	[0.0033203680865193054, 0.00017167381974248928, 0.000328515111695138, 0.0038022813688212928]	0.2330	0.4070	10540	25896
2.0831	11.0	3289	2.4435	0.0002	[0.0032796101949025486, 0.000167897918065816, 0.000321853878339234, 0.003875968992248062]	0.2401	0.4121	10671	25896
1.9685	12.0	3588	2.4483	0.0002	[0.0037652056381540836, 0.0001772421127259837, 0.0003397893306150187, 0.004098360655737705]	0.2231	0.3999	10357	25896
1.9685	13.0	3887	2.4557	0.0002	[0.0033178500331785005, 0.00017143836790673754, 0.000327653997378768, 0.0036900369003690036]	0.2334	0.4073	10548	25896
1.8816	14.0	4186	2.4669	0.0002	[0.003189792663476874, 0.00016826518593303046, 0.000321853878339234, 0.0036900369003690036]	0.2394	0.4116	10658	25896

Framework versions

• Transformers 4.25.1
• Pytorch 1.13.0+cu117
• Datasets 2.7.1
• Tokenizers 0.13.2