Hugging Face's logo

devagonal
/
mt5-semantic

Text2Text Generation
Transformers
TensorBoard
Safetensors
t5
Generated from Trainer
text-generation-inference
Inference Endpoints
Model card Files Metrics Community
mt5-semantic / README.md
devagonal's picture
devagonal
End of training
8662a8c
preview code
|
raw
history blame
4.08 kB
metadata
license: apache-2.0
base_model: google/mt5-base
tags:
  - generated_from_trainer
metrics:
  - rouge
model-index:
  - name: mt5-semantic
    results: []

mt5-semantic

This model is a fine-tuned version of google/mt5-base on an unknown dataset. It achieves the following results on the evaluation set:

  • Loss: 5.5169
  • Rouge1: 0.1333
  • Rouge2: 0.0
  • Rougel: 0.1333
  • Rougelsum: 0.1333

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

  • learning_rate: 0.0003
  • train_batch_size: 8
  • eval_batch_size: 4
  • seed: 42
  • optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
  • lr_scheduler_type: linear
  • num_epochs: 30

Training results

Training Loss Epoch Step Validation Loss Rouge1 Rouge2 Rougel Rougelsum
No log 1.0 1 13.3180 0.0 0.0 0.0 0.0
No log 2.0 2 10.2648 0.0 0.0 0.0 0.0
No log 3.0 3 14.4358 0.2222 0.125 0.2222 0.2222
No log 4.0 4 12.2551 0.2222 0.125 0.2222 0.2222
No log 5.0 5 8.3587 0.2222 0.125 0.2222 0.2222
No log 6.0 6 9.0556 0.2222 0.125 0.2222 0.2222
No log 7.0 7 9.8159 0.25 0.1429 0.25 0.25
No log 8.0 8 8.7513 0.1333 0.0 0.1333 0.1333
No log 9.0 9 9.2218 0.2 0.0 0.2 0.2
No log 10.0 10 8.9089 0.1333 0.0 0.1333 0.1333
No log 11.0 11 10.3286 0.1667 0.0 0.1667 0.1667
No log 12.0 12 9.6879 0.1667 0.0 0.1667 0.1667
No log 13.0 13 8.9536 0.1333 0.0 0.1333 0.1333
No log 14.0 14 8.3151 0.1333 0.0 0.1333 0.1333
No log 15.0 15 7.2542 0.1333 0.0 0.1333 0.1333
No log 16.0 16 7.4371 0.1333 0.0 0.1333 0.1333
No log 17.0 17 7.8925 0.1333 0.0 0.1333 0.1333
No log 18.0 18 7.5644 0.1333 0.0 0.1333 0.1333
No log 19.0 19 8.1827 0.1333 0.0 0.1333 0.1333
No log 20.0 20 6.8002 0.1333 0.0 0.1333 0.1333
No log 21.0 21 7.1925 0.1333 0.0 0.1333 0.1333
No log 22.0 22 7.9999 0.1333 0.0 0.1333 0.1333
No log 23.0 23 7.4629 0.1333 0.0 0.1333 0.1333
No log 24.0 24 8.1180 0.1333 0.0 0.1333 0.1333
No log 25.0 25 7.0687 0.1333 0.0 0.1333 0.1333
No log 26.0 26 8.3404 0.1333 0.0 0.1333 0.1333
No log 27.0 27 6.2122 0.1333 0.0 0.1333 0.1333
No log 28.0 28 5.5703 0.1333 0.0 0.1333 0.1333
No log 29.0 29 5.5613 0.1333 0.0 0.1333 0.1333
No log 30.0 30 5.5169 0.1333 0.0 0.1333 0.1333

Framework versions

  • Transformers 4.35.2
  • Pytorch 2.1.0+cu118
  • Datasets 2.15.0
  • Tokenizers 0.15.0