metadata
license: bsd-3-clause
tags:
- generated_from_trainer
metrics:
- bleu
model-index:
- name: CommitPredictorT5
results: []
CommitPredictorT5
This model is a fine-tuned version of Salesforce/codet5-base-multi-sum on the None dataset. It achieves the following results on the evaluation set:
- Loss: 2.4669
- Bleu: 0.0002
- Precisions: [0.003189792663476874, 0.00016826518593303046, 0.000321853878339234, 0.0036900369003690036]
- Brevity Penalty: 0.2394
- Length Ratio: 0.4116
- Translation Length: 10658
- Reference Length: 25896
Model description
More information needed
Intended uses & limitations
More information needed
Training and evaluation data
More information needed
Training procedure
Training hyperparameters
The following hyperparameters were used during training:
- learning_rate: 2e-05
- train_batch_size: 42
- eval_batch_size: 42
- seed: 42
- gradient_accumulation_steps: 3
- total_train_batch_size: 126
- optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
- lr_scheduler_type: linear
- num_epochs: 100
Training results
| Training Loss | Epoch | Step | Validation Loss | Bleu | Precisions | Brevity Penalty | Length Ratio | Translation Length | Reference Length |
|---|---|---|---|---|---|---|---|---|---|
| No log | 1.0 | 299 | 2.8109 | 0.0002 | [0.003640040444893832, 0.00019327406262079628, 0.0003745318352059925, 0.006024096385542169] | 0.1982 | 0.3819 | 9889 | 25896 |
| 3.1102 | 2.0 | 598 | 2.6662 | 0.0002 | [0.004371150407311742, 0.00018691588785046728, 0.00036114120621162876, 0.005319148936170213] | 0.2074 | 0.3887 | 10065 | 25896 |
| 3.1102 | 3.0 | 897 | 2.5869 | 0.0002 | [0.0033418517790446234, 0.00018321729571271528, 0.0003546099290780142, 0.005494505494505495] | 0.2132 | 0.3928 | 10173 | 25896 |
| 2.6696 | 4.0 | 1196 | 2.5371 | 0.0002 | [0.0033398821218074658, 0.00018301610541727673, 0.0003522367030644593, 0.004672897196261682] | 0.2135 | 0.3931 | 10179 | 25896 |
| 2.6696 | 5.0 | 1495 | 2.5077 | 0.0002 | [0.003243655790879603, 0.0001734304543877905, 0.0003356831151393085, 0.005208333333333333] | 0.2298 | 0.4047 | 10481 | 25896 |
| 2.4738 | 6.0 | 1794 | 2.4810 | 0.0002 | [0.0029016345874842827, 0.00017784101013693757, 0.00034234851078397807, 0.0045662100456621] | 0.2220 | 0.3992 | 10338 | 25896 |
| 2.3139 | 7.0 | 2093 | 2.4625 | 0.0002 | [0.002756130013305455, 0.0001722356183258698, 0.00033101621979476995, 0.00423728813559322] | 0.2319 | 0.4063 | 10521 | 25896 |
| 2.3139 | 8.0 | 2392 | 2.4556 | 0.0002 | [0.0027348170501697473, 0.00016983695652173913, 0.0003266906239790918, 0.004273504273504274] | 0.2364 | 0.4094 | 10603 | 25896 |
| 2.1842 | 9.0 | 2691 | 2.4470 | 0.0002 | [0.003198193961057285, 0.000169061707523246, 0.00032658393207054214, 0.004784688995215311] | 0.2378 | 0.4105 | 10630 | 25896 |
| 2.1842 | 10.0 | 2990 | 2.4439 | 0.0002 | [0.0033203680865193054, 0.00017167381974248928, 0.000328515111695138, 0.0038022813688212928] | 0.2330 | 0.4070 | 10540 | 25896 |
| 2.0831 | 11.0 | 3289 | 2.4435 | 0.0002 | [0.0032796101949025486, 0.000167897918065816, 0.000321853878339234, 0.003875968992248062] | 0.2401 | 0.4121 | 10671 | 25896 |
| 1.9685 | 12.0 | 3588 | 2.4483 | 0.0002 | [0.0037652056381540836, 0.0001772421127259837, 0.0003397893306150187, 0.004098360655737705] | 0.2231 | 0.3999 | 10357 | 25896 |
| 1.9685 | 13.0 | 3887 | 2.4557 | 0.0002 | [0.0033178500331785005, 0.00017143836790673754, 0.000327653997378768, 0.0036900369003690036] | 0.2334 | 0.4073 | 10548 | 25896 |
| 1.8816 | 14.0 | 4186 | 2.4669 | 0.0002 | [0.003189792663476874, 0.00016826518593303046, 0.000321853878339234, 0.0036900369003690036] | 0.2394 | 0.4116 | 10658 | 25896 |
Framework versions
- Transformers 4.25.1
- Pytorch 1.13.0+cu117
- Datasets 2.7.1
- Tokenizers 0.13.2