---
license: apache-2.0
base_model: google/mt5-large
tags:
- generated_from_trainer
metrics:
- rouge
model-index:
- name: MT5-large_NO-idun-20epoch-earlystopping
  results: []
---

<!-- This model card has been generated automatically according to the information the Trainer had access to. You
should probably proofread and complete it, then remove this comment. -->

# MT5-large_NO-idun-20epoch-earlystopping

This model is a fine-tuned version of [google/mt5-large](https://huggingface.co/google/mt5-large) on an unknown dataset.
It achieves the following results on the evaluation set:
- Loss: 1.6061
- Rouge1: 41.2146
- Rouge2: 18.153
- Rougel: 28.4036
- Rougelsum: 36.8514
- Gen Len: 111.1064

## Model description

More information needed

## Intended uses & limitations

More information needed

## Training and evaluation data

More information needed

## Training procedure

### Training hyperparameters

The following hyperparameters were used during training:
- learning_rate: 5e-05
- train_batch_size: 2
- eval_batch_size: 2
- seed: 42
- gradient_accumulation_steps: 4
- total_train_batch_size: 8
- optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
- lr_scheduler_type: linear
- num_epochs: 20

### Training results

| Training Loss | Epoch | Step | Validation Loss | Rouge1  | Rouge2  | Rougel  | Rougelsum | Gen Len  |
|:-------------:|:-----:|:----:|:---------------:|:-------:|:-------:|:-------:|:---------:|:--------:|
| No log        | 0.98  | 46   | 13.1717         | 15.636  | 4.987   | 10.443  | 14.3841   | 119.9149 |
| No log        | 1.99  | 93   | 7.7651          | 10.907  | 1.4566  | 6.9291  | 10.152    | 127.0    |
| No log        | 2.99  | 140  | 6.4230          | 18.8954 | 0.4957  | 13.2179 | 17.1171   | 127.0    |
| No log        | 4.0   | 187  | 2.1315          | 37.1306 | 13.4104 | 21.2261 | 33.0925   | 127.0    |
| No log        | 4.98  | 233  | 1.7761          | 37.6703 | 14.7962 | 22.835  | 34.0213   | 113.5638 |
| No log        | 5.99  | 280  | 1.6807          | 38.6245 | 15.7401 | 24.5743 | 34.5933   | 113.2340 |
| No log        | 6.99  | 327  | 1.6484          | 38.7899 | 15.737  | 24.9265 | 34.6166   | 114.3404 |
| No log        | 8.0   | 374  | 1.6156          | 39.3812 | 15.7133 | 24.979  | 35.2788   | 120.0319 |
| No log        | 8.98  | 420  | 1.6138          | 40.0966 | 17.4991 | 26.5925 | 36.5511   | 117.2234 |
| No log        | 9.99  | 467  | 1.6152          | 40.3623 | 17.7244 | 27.0847 | 36.2108   | 113.2128 |
| 6.3618        | 10.99 | 514  | 1.6102          | 41.2763 | 18.0108 | 27.6185 | 37.1836   | 113.1064 |
| 6.3618        | 12.0  | 561  | 1.6070          | 41.2369 | 17.8711 | 27.3781 | 36.9853   | 115.7766 |
| 6.3618        | 12.98 | 607  | 1.6087          | 42.0737 | 18.414  | 27.8849 | 38.1238   | 113.3404 |
| 6.3618        | 13.99 | 654  | 1.6038          | 41.4279 | 17.8899 | 27.79   | 36.929    | 115.1383 |
| 6.3618        | 14.99 | 701  | 1.6061          | 40.8051 | 17.4437 | 27.1414 | 36.494    | 113.8936 |
| 6.3618        | 16.0  | 748  | 1.6074          | 41.8104 | 18.0504 | 27.934  | 37.3843   | 114.8511 |
| 6.3618        | 16.98 | 794  | 1.6053          | 41.4314 | 17.955  | 27.7884 | 36.9083   | 114.3830 |
| 6.3618        | 17.99 | 841  | 1.6057          | 41.8533 | 18.0219 | 27.7616 | 37.4008   | 113.2128 |
| 6.3618        | 18.99 | 888  | 1.6060          | 41.5846 | 18.3563 | 28.4177 | 37.1366   | 112.1915 |
| 6.3618        | 19.68 | 920  | 1.6061          | 41.2146 | 18.153  | 28.4036 | 36.8514   | 111.1064 |


### Framework versions

- Transformers 4.32.1
- Pytorch 2.3.0+cu121
- Datasets 2.12.0
- Tokenizers 0.13.2