license: mit
tags:
- kobart-summarization-diary
- generated_from_trainer
base_model: gogamza/kobart-summarization
model-index:
- name: summary
results: []
summary
This model is a fine-tuned version of gogamza/kobart-summarization on an unknown dataset. It achieves the following results on the evaluation set:
- Loss: 0.4011
Model description
This model summarizes the diary.
Training and evaluation data
This model was trained by the self-instruction process. All data used for fine-tuning this model were generated by chatGPT 3.5.
Training procedure
Training hyperparameters
The following hyperparameters were used during training:
- learning_rate: 5.6e-05
- train_batch_size: 8
- eval_batch_size: 8
- seed: 42
- optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
- lr_scheduler_type: linear
- lr_scheduler_warmup_steps: 300
- num_epochs: 50
Training results
| Training Loss | Epoch | Step | Validation Loss |
|---|---|---|---|
| 1.4804 | 1.47 | 500 | 0.4035 |
| 0.2475 | 2.93 | 1000 | 0.4011 |
| 0.1249 | 4.4 | 1500 | 0.4591 |
| 0.072 | 5.87 | 2000 | 0.4671 |
| 0.039 | 7.33 | 2500 | 0.5022 |
Framework versions
- Transformers 4.37.2
- Pytorch 2.1.2+cu118
- Datasets 2.16.1
- Tokenizers 0.15.0
How to Get Started with the Model
Use the code below to get started with the model. You can adjust hyperparameters to fit on your data. ''' def diary_summary(text): input_ids = tokenizer.encode(text, return_tensors = 'pt').to(device) summary_text_ids = model.generate(input_ids = input_ids, bos_token_id = model.config.bos_token_id, eos_token_id = model.config.eos_token_id, length_penalty = 2.0, max_length = 150, num_beams = 2) return tokenizer.decode(summary_text_ids[0], skip_special_tokens = True) '''