Safetensors
English
t5
Generated from Trainer
Edit model card

You need to agree to share your contact information to access this model

This repository is publicly accessible, but you have to accept the conditions to access its files and content.

Log in or Sign Up to review the conditions and access this model content.

senate_bills_summary_model

This model is a fine-tuned version of google-t5/t5-small on the None dataset. It achieves the following results on the evaluation set:

  • Loss: 1.9099
  • Rouge1: 0.2477
  • Rouge2: 0.1963
  • Rougel: 0.2407
  • Rougelsum: 0.2406
  • Gen Len: 18.9992

Model description

This model is a fine-tuned Google T5-Small model that is fine-tuned to summarize United States Senate Bills.

Intended uses & limitations

Summarize United States Federal Legislation.

Training and evaluation data

Trained on ~13.1k bills and summaries.

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

  • learning_rate: 2e-05
  • train_batch_size: 14
  • eval_batch_size: 14
  • seed: 42
  • optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
  • lr_scheduler_type: linear
  • num_epochs: 4
  • mixed_precision_training: Native AMP

Training results

Training Loss Epoch Step Validation Loss Rouge1 Rouge2 Rougel Rougelsum Gen Len
2.2318 1.0 749 1.9710 0.2475 0.1952 0.2405 0.2402 18.9985
2.1782 2.0 1498 1.9331 0.2478 0.1959 0.2408 0.2406 18.9992
2.1355 3.0 2247 1.9141 0.2479 0.1961 0.2409 0.2407 18.9992
2.1079 4.0 2996 1.9099 0.2477 0.1963 0.2407 0.2406 18.9992

Framework versions

  • Transformers 4.40.1
  • Pytorch 2.2.1+cu121
  • Datasets 2.19.1
  • Tokenizers 0.19.1
Downloads last month
0
Safetensors
Model size
60.5M params
Tensor type
F32
·
Inference API
Unable to determine this model's library. Check the docs .

Model tree for MTSUFall2024SoftwareEngineering/UnitedStatesSenateBillsSummary

Base model

google-t5/t5-small
Finetuned
(1527)
this model

Dataset used to train MTSUFall2024SoftwareEngineering/UnitedStatesSenateBillsSummary