indosum-base-3

This model is a fine-tuned version of LazarusNLP/IndoNanoT5-base on an unknown dataset. It achieves the following results on the evaluation set:

• Loss: 0.7690
• Rouge1: 71.9504
• Rouge2: 64.7658
• Rougel: 68.7524
• Rougelsum: 70.9978
• Gen Len: 99.2

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 0.001
• train_batch_size: 16
• eval_batch_size: 32
• seed: 42
• optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
• lr_scheduler_type: linear
• num_epochs: 5.0

Training results

Training Loss	Epoch	Step	Validation Loss	Rouge1	Rouge2	Rougel	Rougelsum	Gen Len
1.1937	1.0	892	0.8370	64.7751	56.7545	61.4956	63.81	90.984
0.6828	2.0	1784	0.6911	69.9628	62.6338	66.8253	69.0763	101.1173
0.4847	3.0	2676	0.6692	69.9807	62.5614	66.7619	69.0683	95.6133
0.3348	4.0	3568	0.7029	70.9247	63.6191	67.7749	70.0079	100.1547
0.1988	5.0	4460	0.7690	71.4437	64.1873	68.2379	70.5264	98.6667

Framework versions

• Transformers 4.40.2
• Pytorch 2.3.1+cu121
• Datasets 2.20.0
• Tokenizers 0.19.1