indosum-lora-3

This model is a fine-tuned version of LazarusNLP/IndoNanoT5-base on an unknown dataset. It achieves the following results on the evaluation set:

• Loss: 0.5148
• Rouge1: 72.9552
• Rouge2: 66.0943
• Rougel: 69.9638
• Rougelsum: 72.0769
• Gen Len: 102.96

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 0.001
• train_batch_size: 16
• eval_batch_size: 32
• seed: 42
• optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
• lr_scheduler_type: linear
• num_epochs: 5.0

Training results

Training Loss	Epoch	Step	Validation Loss	Rouge1	Rouge2	Rougel	Rougelsum	Gen Len
0.7809	1.0	892	0.5511	70.7387	63.5003	67.571	69.7824	100.4227
0.5981	2.0	1784	0.5352	71.407	64.2122	68.3064	70.4861	104.784
0.5542	3.0	2676	0.5363	72.3351	65.2788	69.2683	71.4248	102.4427
0.5238	4.0	3568	0.5148	72.8512	65.9167	69.8532	71.9591	102.4173
0.5034	5.0	4460	0.5105	72.6979	65.6731	69.6184	71.7604	102.2533

Framework versions

• Transformers 4.40.2
• Pytorch 2.3.1+cu121
• Datasets 2.20.0
• Tokenizers 0.19.1