yesj1234/mbart-mmt_mid2_ko-ja

tst-translation-output2

This model is a fine-tuned version of facebook/mbart-large-50-many-to-many-mmt on an unknown dataset. It achieves the following results on the evaluation set:

• Loss: 0.9049
• Bleu: 10.3643
• Gen Len: 17.4046

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 5e-05
• train_batch_size: 4
• eval_batch_size: 4
• seed: 42
• distributed_type: multi-GPU
• num_devices: 4
• total_train_batch_size: 16
• total_eval_batch_size: 16
• optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
• lr_scheduler_type: linear
• lr_scheduler_warmup_steps: 500
• num_epochs: 35

Training results

Training Loss	Epoch	Step	Validation Loss	Bleu	Gen Len
1.2499	0.23	1500	1.1806	6.1112	18.0495
1.1007	0.46	3000	1.0686	7.4845	17.6068
1.0334	0.68	4500	1.0013	9.0076	17.6214
0.992	0.91	6000	0.9599	8.6786	17.868
0.7881	1.14	7500	0.9644	9.2343	17.2061
0.7675	1.37	9000	0.9427	10.0578	17.6006
0.7665	1.59	10500	0.9238	10.436	17.2095
0.7707	1.82	12000	0.9049	10.5971	17.2971
0.6119	2.05	13500	0.9392	10.8369	17.3201
0.5579	2.28	15000	0.9429	10.3486	17.3221
0.5633	2.5	16500	0.9310	10.6114	17.3679
0.5764	2.73	18000	0.9265	9.9612	17.1339

Framework versions

• Transformers 4.34.0
• Pytorch 2.1.0+cu121
• Datasets 2.14.5
• Tokenizers 0.14.1