pretrained-asd_wo-cot_w-asd

This model is a fine-tuned version of slplab/polyglot-ko-1.3b_pretrained-asd on an unknown dataset. It achieves the following results on the evaluation set:

• Loss: 10.9440

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 3e-05
• train_batch_size: 2
• eval_batch_size: 2
• seed: 42
• gradient_accumulation_steps: 4
• total_train_batch_size: 8
• optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
• lr_scheduler_type: linear
• num_epochs: 3
• mixed_precision_training: Native AMP

Training results

Training Loss	Epoch	Step	Validation Loss
13.2673	0.1290	1	13.2282
13.2184	0.2581	2	13.1124
13.0976	0.3871	3	12.9704
12.9545	0.5161	4	12.8240
12.8232	0.6452	5	12.6843
12.6522	0.7742	6	12.5389
12.5444	0.9032	7	12.3854
12.3496	1.0323	8	12.2316
12.2126	1.1613	9	12.0854
12.0724	1.2903	10	11.9376
11.9188	1.4194	11	11.7927
11.7615	1.5484	12	11.6542
11.6456	1.6774	13	11.5224
11.5276	1.8065	14	11.4017
11.3736	1.9355	15	11.2929
11.281	2.0645	16	11.1996
11.15	2.1935	17	11.1171
11.0963	2.3226	18	11.0492
11.0646	2.4516	19	10.9964
10.9801	2.5806	20	10.9614
10.9325	2.7097	21	10.9440

Framework versions

• PEFT 0.11.1
• Transformers 4.41.1
• Pytorch 2.3.0+cu121
• Datasets 2.19.1
• Tokenizers 0.19.1