query

This model is a fine-tuned version of meta-llama/Meta-Llama-3-8B on the None dataset. It achieves the following results on the evaluation set:

• Loss: 0.3182

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 2e-05
• train_batch_size: 4
• eval_batch_size: 4
• seed: 42
• optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
• lr_scheduler_type: linear
• num_epochs: 32
• mixed_precision_training: Native AMP

Training results

Training Loss	Epoch	Step	Validation Loss
No log	1.0	4	14.3920
No log	2.0	8	13.2479
No log	3.0	12	12.0852
No log	4.0	16	10.4945
No log	5.0	20	8.7942
No log	6.0	24	6.9466
No log	7.0	28	4.9444
No log	8.0	32	2.8411
No log	9.0	36	1.2720
No log	10.0	40	0.7259
No log	11.0	44	0.5651
No log	12.0	48	0.5014
No log	13.0	52	0.4775
No log	14.0	56	0.4608
No log	15.0	60	0.4456
No log	16.0	64	0.4314
No log	17.0	68	0.4171
No log	18.0	72	0.4032
No log	19.0	76	0.3910
No log	20.0	80	0.3808
No log	21.0	84	0.3719
No log	22.0	88	0.3638
No log	23.0	92	0.3561
No log	24.0	96	0.3491
No log	25.0	100	0.3426
No log	26.0	104	0.3370
No log	27.0	108	0.3320
No log	28.0	112	0.3278
No log	29.0	116	0.3241
No log	30.0	120	0.3216
No log	31.0	124	0.3195
No log	32.0	128	0.3182

Framework versions

• PEFT 0.10.0
• Transformers 4.41.1
• Pytorch 2.3.0+cu121
• Datasets 2.19.1
• Tokenizers 0.19.1