metadata

license: apache-2.0
base_model: google/flan-t5-large
tags:
  - generated_from_trainer
datasets:
  - adithya7/background-summaries
metrics:
  - rouge
model-index:
  - name: '2023_12_18_08_41_35'
    results:
      - task:
          name: Sequence-to-sequence Language Modeling
          type: text2text-generation
        dataset:
          name: background_summ
          type: background_summ
          config: background-summ
          split: validation
          args: background-summ
        metrics:
          - name: Rouge1
            type: rouge
            value: 39.8
widget:
  - text: >-
      Is this review positive or negative? Review: Best cast iron skillet you
      will ever buy.
    example_title: Sentiment analysis
  - text: >-
      Barack Obama nominated Hilary Clinton as his secretary of state on Monday.
      He chose her because she had ...
    example_title: Coreference resolution
  - text: >-
      On a shelf, there are five books: a gray book, a red book, a purple book,
      a blue book, and a black book ...
    example_title: Logic puzzles
  - text: >-
      The two men running to become New York City's next mayor will face off in
      their first debate Wednesday night ...
    example_title: Reading comprehension

2023_12_18_08_41_35

This model is a fine-tuned version of google/flan-t5-large on the background_summ dataset. It achieves the following results on the evaluation set:

Loss: 2.3928
Rouge1: 39.8
Rouge2: 18.8
Rougel: 26.7
Rougelsum: 36.1
Bertscore Precision: 88.4
Bertscore Recall: 86.8
Bertscore F1: 87.5

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

learning_rate: 1e-05
train_batch_size: 1
eval_batch_size: 1
seed: 42
optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
lr_scheduler_type: linear
num_epochs: 10

Training results

Training Loss	Epoch	Step	Validation Loss	Rouge1	Rouge2	Rougel	Rougelsum	Bertscore Precision	Bertscore Recall	Bertscore F1
1.6858	1.0	714	2.0262	41.1	19.3	27.1	37.3	87.9	87.1	87.5
1.1309	2.0	1428	2.0889	40.8	19.6	27.3	37.1	87.8	87.1	87.4
0.7568	3.0	2142	2.1569	40.8	19.1	27.3	37.0	87.8	87.0	87.4
0.6779	4.0	2856	2.1800	39.5	18.4	26.4	35.9	87.8	86.7	87.2
0.5567	5.0	3570	2.2454	40.1	19.0	26.8	36.6	88.2	86.8	87.4
0.5264	6.0	4284	2.3172	38.8	18.1	26.1	35.2	88.0	86.6	87.3
0.5046	7.0	4998	2.3409	40.1	19.0	27.0	36.4	88.4	86.8	87.6
0.4465	8.0	5712	2.3751	39.8	18.7	26.7	36.1	88.4	86.7	87.6
0.4524	9.0	6426	2.3824	40.0	19.0	27.1	36.4	88.5	86.8	87.6
0.4308	10.0	7140	2.3928	39.8	18.8	26.7	36.1	88.4	86.8	87.5

Framework versions

Transformers 4.33.1
Pytorch 1.13.1
Datasets 2.14.5
Tokenizers 0.13.3