lc_reddit

This model is a fine-tuned version of mistralai/Mistral-7B-v0.1 on an unknown dataset. It achieves the following results on the evaluation set:

• Loss: 2.4421

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 2e-05
• train_batch_size: 1
• eval_batch_size: 1
• seed: 42
• optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
• lr_scheduler_type: cosine
• num_epochs: 50

Training results

Training Loss	Epoch	Step	Validation Loss
1.893	1.0	306	1.8913
2.0254	2.0	612	1.8820
1.7988	3.0	918	1.8821
1.6315	4.0	1224	1.8907
1.6124	5.0	1530	1.9035
1.7146	6.0	1836	1.9222
1.5807	7.0	2142	1.9383
1.4356	8.0	2448	1.9796
1.4935	9.0	2754	1.9838
1.3923	10.0	3060	2.0112
1.5712	11.0	3366	2.0217
1.5216	12.0	3672	2.0493
1.2513	13.0	3978	2.0731
1.2956	14.0	4284	2.1151
1.5635	15.0	4590	2.1298
1.4956	16.0	4896	2.1525
1.3592	17.0	5202	2.1751
1.1305	18.0	5508	2.1893
1.1396	19.0	5814	2.2454
1.3858	20.0	6120	2.2797
1.3174	21.0	6426	2.2689
1.5609	22.0	6732	2.3098
1.3431	23.0	7038	2.3238
1.3111	24.0	7344	2.3742
1.1365	25.0	7650	2.3727
1.3318	26.0	7956	2.3978
1.3297	27.0	8262	2.3647
1.2178	28.0	8568	2.3971
1.2757	29.0	8874	2.4292
1.236	30.0	9180	2.4170
1.1888	31.0	9486	2.4439
1.0917	32.0	9792	2.4225
1.1148	33.0	10098	2.4166
1.1907	34.0	10404	2.4318
1.1906	35.0	10710	2.4352
1.2238	36.0	11016	2.4471
1.1596	37.0	11322	2.4382
1.2184	38.0	11628	2.4343
1.2428	39.0	11934	2.4422
1.3111	40.0	12240	2.4397
1.2845	41.0	12546	2.4460
1.3173	42.0	12852	2.4428
1.193	43.0	13158	2.4430
1.1774	44.0	13464	2.4425
1.1868	45.0	13770	2.4396
1.2042	46.0	14076	2.4430
1.2833	47.0	14382	2.4398
1.2766	48.0	14688	2.4410
1.4958	49.0	14994	2.4412
1.1868	50.0	15300	2.4421

Framework versions

• PEFT 0.11.1
• Transformers 4.41.2
• Pytorch 2.1.0+cu118
• Datasets 2.19.2
• Tokenizers 0.19.1