metadata

license: apache-2.0
base_model: allenai/led-base-16384
tags:
  - generated_from_trainer
datasets:
  - billsum
metrics:
  - rouge
model-index:
  - name: LED_billsum_model
    results:
      - task:
          name: Sequence-to-sequence Language Modeling
          type: text2text-generation
        dataset:
          name: billsum
          type: billsum
          config: default
          split: ca_test
          args: default
        metrics:
          - name: Rouge1
            type: rouge
            value: 0.1447

LED_billsum_model

This model is a fine-tuned version of allenai/led-base-16384 on the billsum dataset. It achieves the following results on the evaluation set:

Loss: 1.6576
Rouge1: 0.1447
Rouge2: 0.0854
Rougel: 0.1292
Rougelsum: 0.1339
Gen Len: 20.0

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

learning_rate: 2e-05
train_batch_size: 3
eval_batch_size: 3
seed: 42
optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
lr_scheduler_type: linear
num_epochs: 10

Training results

Training Loss	Epoch	Step	Validation Loss	Rouge1	Rouge2	Rougel	Rougelsum	Gen Len
1.4849	1.0	330	1.6511	0.1463	0.0827	0.1276	0.1337	20.0
1.3361	2.0	660	1.6056	0.148	0.0799	0.1268	0.1336	20.0
1.1727	3.0	990	1.5833	0.1459	0.0827	0.1289	0.1341	20.0
1.0601	4.0	1320	1.5987	0.1462	0.0859	0.1299	0.1344	20.0
0.9789	5.0	1650	1.6030	0.1414	0.0794	0.125	0.1302	20.0
0.8724	6.0	1980	1.6060	0.1476	0.0868	0.1298	0.1356	20.0
0.7994	7.0	2310	1.6295	0.1348	0.0758	0.1198	0.1253	20.0
0.7762	8.0	2640	1.6317	0.1422	0.0831	0.1261	0.1312	20.0
0.7087	9.0	2970	1.6501	0.1421	0.0825	0.1264	0.1311	20.0
0.7014	10.0	3300	1.6576	0.1447	0.0854	0.1292	0.1339	20.0

Framework versions

Transformers 4.34.1
Pytorch 2.1.0+cu118
Datasets 2.14.6
Tokenizers 0.14.1