easylm-sft-gemma-2-9b

This model is a fine-tuned version of google/gemma-2-9b on the alpaca_farm dataset. It achieves the following results on the evaluation set:

• Loss: 0.7115

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 3e-06
• train_batch_size: 2
• eval_batch_size: 2
• seed: 42
• distributed_type: multi-GPU
• num_devices: 8
• total_train_batch_size: 16
• total_eval_batch_size: 16
• optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
• lr_scheduler_type: cosine
• num_epochs: 2

Training results

Training Loss	Epoch	Step	Validation Loss
0.6921	0.016	10	0.6721
0.6141	0.032	20	0.6662
0.6665	0.048	30	0.6603
0.6116	0.064	40	0.6611
0.6102	0.08	50	0.6580
0.6886	0.096	60	0.6593
0.6415	0.112	70	0.6596
0.6214	0.128	80	0.6595
0.6816	0.144	90	0.6584
0.6481	0.16	100	0.6597
0.6022	0.176	110	0.6590
0.6703	0.192	120	0.6607
0.6742	0.208	130	0.6615
0.6369	0.224	140	0.6615
0.7142	0.24	150	0.6602
0.6707	0.256	160	0.6611
0.6629	0.272	170	0.6609
0.6299	0.288	180	0.6610
0.6351	0.304	190	0.6607
0.5885	0.32	200	0.6610
0.6613	0.336	210	0.6619
0.6151	0.352	220	0.6602
0.6342	0.368	230	0.6609
0.6376	0.384	240	0.6601
0.679	0.4	250	0.6601
0.6911	0.416	260	0.6593
0.6717	0.432	270	0.6592
0.6758	0.448	280	0.6603
0.6243	0.464	290	0.6603
0.643	0.48	300	0.6586
0.603	0.496	310	0.6573
0.6336	0.512	320	0.6568
0.6198	0.528	330	0.6569
0.6989	0.544	340	0.6578
0.6353	0.56	350	0.6570
0.6746	0.576	360	0.6568
0.6883	0.592	370	0.6571
0.6772	0.608	380	0.6566
0.6563	0.624	390	0.6564
0.6077	0.64	400	0.6554
0.6291	0.656	410	0.6552
0.6073	0.672	420	0.6547
0.6598	0.688	430	0.6551
0.593	0.704	440	0.6547
0.6352	0.72	450	0.6547
0.6216	0.736	460	0.6540
0.6937	0.752	470	0.6535
0.669	0.768	480	0.6530
0.6052	0.784	490	0.6525
0.6218	0.8	500	0.6525
0.6341	0.816	510	0.6526
0.6681	0.832	520	0.6522
0.6203	0.848	530	0.6516
0.6682	0.864	540	0.6506
0.6212	0.88	550	0.6501
0.6887	0.896	560	0.6502
0.64	0.912	570	0.6504
0.6176	0.928	580	0.6500
0.6285	0.944	590	0.6500
0.6661	0.96	600	0.6489
0.6537	0.976	610	0.6488
0.657	0.992	620	0.6482
0.4004	1.008	630	0.6503
0.4014	1.024	640	0.7170
0.4179	1.04	650	0.6923
0.3998	1.056	660	0.6921
0.3705	1.072	670	0.7054
0.3513	1.088	680	0.7036
0.3815	1.104	690	0.7025
0.3684	1.12	700	0.7049
0.3914	1.1360	710	0.7069
0.4082	1.152	720	0.7018
0.3494	1.168	730	0.7042
0.3715	1.184	740	0.7071
0.3675	1.2	750	0.7085
0.3319	1.216	760	0.7112
0.3823	1.232	770	0.7141
0.3571	1.248	780	0.7113
0.3503	1.264	790	0.7127
0.3742	1.28	800	0.7159
0.4087	1.296	810	0.7139
0.3781	1.312	820	0.7073
0.3475	1.328	830	0.7129
0.3724	1.3440	840	0.7113
0.3612	1.3600	850	0.7130
0.3254	1.376	860	0.7139
0.3626	1.392	870	0.7145
0.351	1.408	880	0.7147
0.3357	1.424	890	0.7105
0.371	1.44	900	0.7079
0.3566	1.456	910	0.7070
0.3762	1.472	920	0.7118
0.3755	1.488	930	0.7126
0.3595	1.504	940	0.7107
0.3828	1.52	950	0.7118
0.3793	1.536	960	0.7173
0.3446	1.552	970	0.7150
0.3707	1.568	980	0.7135
0.3604	1.584	990	0.7141
0.3441	1.6	1000	0.7137
0.3705	1.616	1010	0.7154
0.3857	1.6320	1020	0.7189
0.3952	1.6480	1030	0.7148
0.3815	1.6640	1040	0.7116
0.3507	1.6800	1050	0.7108
0.3662	1.696	1060	0.7124
0.3581	1.712	1070	0.7136
0.3867	1.728	1080	0.7132
0.3707	1.744	1090	0.7127
0.4078	1.76	1100	0.7122
0.3713	1.776	1110	0.7111
0.3525	1.792	1120	0.7110
0.3873	1.808	1130	0.7115
0.4008	1.8240	1140	0.7119
0.3889	1.8400	1150	0.7119
0.3591	1.8560	1160	0.7116
0.3843	1.8720	1170	0.7116
0.3713	1.888	1180	0.7115
0.3659	1.904	1190	0.7115
0.3588	1.92	1200	0.7115
0.3556	1.936	1210	0.7115
0.3278	1.952	1220	0.7116
0.3642	1.968	1230	0.7115
0.3718	1.984	1240	0.7115
0.3611	2.0	1250	0.7115

Framework versions

• Transformers 4.43.3
• Pytorch 2.3.0+cu121
• Datasets 2.20.0
• Tokenizers 0.19.1