gpt2_custom_c_q_and_a_v3

This model is a fine-tuned version of lighteternal/gpt2-finetuned-greek on an unknown dataset. It achieves the following results on the evaluation set:

• Loss: 0.0066

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 5e-05
• train_batch_size: 16
• eval_batch_size: 8
• seed: 42
• optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
• lr_scheduler_type: linear
• lr_scheduler_warmup_steps: 100
• num_epochs: 200

Training results

Training Loss	Epoch	Step	Validation Loss
2.3925	8.33	100	0.7608
0.4388	16.67	200	0.1300
0.1291	25.0	300	0.0496
0.0634	33.33	400	0.0244
0.0402	41.67	500	0.0164
0.0281	50.0	600	0.0115
0.0214	58.33	700	0.0091
0.0178	66.67	800	0.0086
0.0151	75.0	900	0.0079
0.0137	83.33	1000	0.0076
0.0122	91.67	1100	0.0075
0.0122	100.0	1200	0.0073
0.0111	108.33	1300	0.0072
0.011	116.67	1400	0.0070
0.0101	125.0	1500	0.0069
0.0107	133.33	1600	0.0072
0.0097	141.67	1700	0.0069
0.0094	150.0	1800	0.0069
0.0095	158.33	1900	0.0068
0.0094	166.67	2000	0.0067
0.0088	175.0	2100	0.0066
0.0088	183.33	2200	0.0067
0.0087	191.67	2300	0.0066
0.0088	200.0	2400	0.0066

Framework versions

• Transformers 4.32.1
• Pytorch 2.0.1+cu118
• Tokenizers 0.13.3