results_1026_medium

This model is a fine-tuned version of google/gemma-2b-it on the None dataset. It achieves the following results on the evaluation set:

• Loss: 1.6380

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 0.0002
• train_batch_size: 3
• eval_batch_size: 2
• seed: 42
• gradient_accumulation_steps: 8
• total_train_batch_size: 24
• optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
• lr_scheduler_type: cosine
• lr_scheduler_warmup_ratio: 0.03
• num_epochs: 20
• mixed_precision_training: Native AMP

Training results

Training Loss	Epoch	Step	Validation Loss
3.7375	0.0390	10	4.2741
3.911	0.0780	20	4.2339
3.7483	0.1170	30	4.1269
4.0206	0.1559	40	3.8986
4.6902	0.1949	50	3.5532
3.2196	0.2339	60	3.2401
3.0539	0.2729	70	3.0130
2.4944	0.3119	80	2.8331
2.4502	0.3509	90	2.6492
2.5653	0.3899	100	2.4577
2.7832	0.4288	110	2.3803
2.4319	0.4678	120	2.3086
1.8581	0.5068	130	2.2263
1.588	0.5458	140	2.1658
1.9006	0.5848	150	2.1379
2.7059	0.6238	160	2.1338
2.2761	0.6628	170	2.0950
1.6663	0.7018	180	2.0510
1.3498	0.7407	190	2.0417
1.6824	0.7797	200	2.0340
2.6165	0.8187	210	2.0028
2.1451	0.8577	220	1.9910
1.3869	0.8967	230	1.9698
1.3288	0.9357	240	1.9555
1.5853	0.9747	250	1.9570
2.0552	1.0136	260	1.9191
2.3622	1.0526	270	1.9206
2.0127	1.0916	280	1.9111
1.2681	1.1306	290	1.8985
1.303	1.1696	300	1.8957
1.9404	1.2086	310	1.8921
2.3832	1.2476	320	1.8766
1.7822	1.2865	330	1.8680
1.128	1.3255	340	1.8651
1.3576	1.3645	350	1.8616
1.8378	1.4035	360	1.8536
2.281	1.4425	370	1.8409
1.7978	1.4815	380	1.8333
1.1124	1.5205	390	1.8319
1.2702	1.5595	400	1.8315
1.785	1.5984	410	1.8288
2.2559	1.6374	420	1.8210
1.7997	1.6764	430	1.8109
1.0795	1.7154	440	1.8121
1.197	1.7544	450	1.8074
1.8304	1.7934	460	1.8126
2.2655	1.8324	470	1.8006
1.7715	1.8713	480	1.8003
1.0938	1.9103	490	1.7942
1.3049	1.9493	500	1.7928
1.5426	1.9883	510	1.7930
2.1529	2.0273	520	1.7900
2.0421	2.0663	530	1.7771
1.4795	2.1053	540	1.7755
0.9904	2.1442	550	1.7820
1.2937	2.1832	560	1.7812
2.0973	2.2222	570	1.7786
2.0218	2.2612	580	1.7715
1.4054	2.3002	590	1.7682
0.9818	2.3392	600	1.7749
1.307	2.3782	610	1.7700
2.1172	2.4172	620	1.7693
2.0437	2.4561	630	1.7652
1.5118	2.4951	640	1.7557
1.0348	2.5341	650	1.7611
1.3371	2.5731	660	1.7579
2.1021	2.6121	670	1.7543
1.9754	2.6511	680	1.7487
1.4214	2.6901	690	1.7471
0.9251	2.7290	700	1.7550
1.3379	2.7680	710	1.7502
2.1229	2.8070	720	1.7485
1.9831	2.8460	730	1.7448
1.4134	2.8850	740	1.7348
1.0077	2.9240	750	1.7426
1.2997	2.9630	760	1.7426
1.5318	3.0019	770	1.7349
2.3008	3.0409	780	1.7255
1.8745	3.0799	790	1.7276
1.146	3.1189	800	1.7298
1.0261	3.1579	810	1.7374
1.3219	3.1969	820	1.7431
2.2955	3.2359	830	1.7253
1.8123	3.2749	840	1.7233
1.1207	3.3138	850	1.7222
0.9991	3.3528	860	1.7291
1.2965	3.3918	870	1.7280
2.2965	3.4308	880	1.7243
1.8012	3.4698	890	1.7186
1.0526	3.5088	900	1.7187
0.9943	3.5478	910	1.7296
1.318	3.5867	920	1.7271
2.2739	3.6257	930	1.7169
1.8798	3.6647	940	1.7152
1.1567	3.7037	950	1.7070
0.9919	3.7427	960	1.7166
1.3261	3.7817	970	1.7130
2.3114	3.8207	980	1.7050
1.8273	3.8596	990	1.7034
1.1196	3.8986	1000	1.7026
1.0395	3.9376	1010	1.7127
1.3201	3.9766	1020	1.7129
1.8356	4.0156	1030	1.6966
2.0688	4.0546	1040	1.6940
1.6132	4.0936	1050	1.6956
0.8688	4.1326	1060	1.7055
1.1553	4.1715	1070	1.7119
1.6403	4.2105	1080	1.7046
2.1075	4.2495	1090	1.6948
1.6034	4.2885	1100	1.6959
0.9718	4.3275	1110	1.6976
1.0579	4.3665	1120	1.7145
1.665	4.4055	1130	1.6964
2.0631	4.4444	1140	1.6893
1.4689	4.4834	1150	1.6904
0.912	4.5224	1160	1.6923
1.0805	4.5614	1170	1.7014
1.6814	4.6004	1180	1.6933
2.0787	4.6394	1190	1.6858
1.5954	4.6784	1200	1.6862
0.9737	4.7173	1210	1.6861
1.0679	4.7563	1220	1.6953
1.6742	4.7953	1230	1.6895
2.0564	4.8343	1240	1.6798
1.5753	4.8733	1250	1.6808
0.9045	4.9123	1260	1.6801
1.0897	4.9513	1270	1.6883
1.3817	4.9903	1280	1.6865
2.0454	5.0292	1290	1.6746
1.8581	5.0682	1300	1.6706
1.2457	5.1072	1310	1.6780
0.8817	5.1462	1320	1.6873
1.1588	5.1852	1330	1.6930
2.0329	5.2242	1340	1.6813
1.8986	5.2632	1350	1.6762
1.2858	5.3021	1360	1.6741
0.8416	5.3411	1370	1.6825
1.1407	5.3801	1380	1.6903
1.9853	5.4191	1390	1.6737
1.8268	5.4581	1400	1.6726
1.2513	5.4971	1410	1.6732
0.867	5.5361	1420	1.6779
1.1384	5.5750	1430	1.6867
2.0149	5.6140	1440	1.6721
1.8491	5.6530	1450	1.6690
1.3532	5.6920	1460	1.6700
0.8445	5.7310	1470	1.6740
1.1203	5.7700	1480	1.6833
2.0431	5.8090	1490	1.6672
1.7919	5.8480	1500	1.6647
1.1206	5.8869	1510	1.6644
0.8747	5.9259	1520	1.6688
1.1668	5.9649	1530	1.6802
1.3985	6.0039	1540	1.6616
2.1421	6.0429	1550	1.6563
1.6559	6.0819	1560	1.6583
0.9856	6.1209	1570	1.6690
0.9008	6.1598	1580	1.6789
1.2231	6.1988	1590	1.6901
2.131	6.2378	1600	1.6656
1.6599	6.2768	1610	1.6677
0.9681	6.3158	1620	1.6656
0.8858	6.3548	1630	1.6737
1.2407	6.3938	1640	1.6805
2.166	6.4327	1650	1.6594
1.6903	6.4717	1660	1.6588
0.945	6.5107	1670	1.6599
0.9337	6.5497	1680	1.6679
1.2023	6.5887	1690	1.6814
2.1931	6.6277	1700	1.6595
1.6423	6.6667	1710	1.6596
0.9213	6.7057	1720	1.6609
0.9269	6.7446	1730	1.6659
1.2439	6.7836	1740	1.6691
2.1721	6.8226	1750	1.6527
1.7012	6.8616	1760	1.6516
0.9872	6.9006	1770	1.6563
0.8626	6.9396	1780	1.6643
1.2399	6.9786	1790	1.6661
1.747	7.0175	1800	1.6468
1.8333	7.0565	1810	1.6455
1.3556	7.0955	1820	1.6538
0.8188	7.1345	1830	1.6637
0.9805	7.1735	1840	1.6687
1.6384	7.2125	1850	1.6642
1.9626	7.2515	1860	1.6491
1.4326	7.2904	1870	1.6582
0.8209	7.3294	1880	1.6608
0.9862	7.3684	1890	1.6661
1.6188	7.4074	1900	1.6634
1.9313	7.4464	1910	1.6480
1.3892	7.4854	1920	1.6551
0.79	7.5244	1930	1.6603
0.9712	7.5634	1940	1.6679
1.5826	7.6023	1950	1.6606
1.9704	7.6413	1960	1.6463
1.4192	7.6803	1970	1.6506
0.7849	7.7193	1980	1.6553
0.9904	7.7583	1990	1.6606
1.6352	7.7973	2000	1.6519
2.0166	7.8363	2010	1.6425
1.4613	7.8752	2020	1.6448
0.822	7.9142	2030	1.6523
0.9734	7.9532	2040	1.6570
1.2641	7.9922	2050	1.6503
2.0093	8.0312	2060	1.6390
1.7434	8.0702	2070	1.6389
1.1463	8.1092	2080	1.6490
0.7531	8.1481	2090	1.6601
1.0364	8.1871	2100	1.6617
1.9569	8.2261	2110	1.6436
1.7511	8.2651	2120	1.6394
1.1094	8.3041	2130	1.6526
0.7236	8.3431	2140	1.6619
1.0677	8.3821	2150	1.6602
1.9794	8.4211	2160	1.6436
1.7215	8.4600	2170	1.6370
1.0996	8.4990	2180	1.6489
0.7863	8.5380	2190	1.6542
1.0097	8.5770	2200	1.6568
1.9809	8.6160	2210	1.6434
1.6825	8.6550	2220	1.6360
1.0263	8.6940	2230	1.6434
0.7833	8.7329	2240	1.6501
1.0464	8.7719	2250	1.6508
1.9653	8.8109	2260	1.6401
1.7373	8.8499	2270	1.6344
1.1006	8.8889	2280	1.6413
0.8168	8.9279	2290	1.6541
0.9995	8.9669	2300	1.6534
1.4208	9.0058	2310	1.6379
1.9976	9.0448	2320	1.6321
1.6071	9.0838	2330	1.6334
0.8642	9.1228	2340	1.6474
0.7999	9.1618	2350	1.6597
1.1814	9.2008	2360	1.6597
2.0584	9.2398	2370	1.6371
1.553	9.2788	2380	1.6367
0.8407	9.3177	2390	1.6471
0.794	9.3567	2400	1.6581
1.174	9.3957	2410	1.6569
2.0687	9.4347	2420	1.6380
1.5462	9.4737	2430	1.6351
0.8547	9.5127	2440	1.6441
0.8228	9.5517	2450	1.6526
1.2233	9.5906	2460	1.6499
2.0733	9.6296	2470	1.6349
1.538	9.6686	2480	1.6347
0.8964	9.7076	2490	1.6424
0.8674	9.7466	2500	1.6536
1.2348	9.7856	2510	1.6496
2.0887	9.8246	2520	1.6320
1.5475	9.8635	2530	1.6309
0.8144	9.9025	2540	1.6412
0.8235	9.9415	2550	1.6502
1.178	9.9805	2560	1.6487
1.6384	10.0195	2570	1.6345
1.8175	10.0585	2580	1.6300
1.2408	10.0975	2590	1.6356
0.7216	10.1365	2600	1.6489
0.8593	10.1754	2610	1.6568
1.607	10.2144	2620	1.6480
1.8762	10.2534	2630	1.6334
1.3494	10.2924	2640	1.6354
0.7465	10.3314	2650	1.6458
0.8743	10.3704	2660	1.6558
1.6364	10.4094	2670	1.6468
1.8216	10.4483	2680	1.6318
1.2798	10.4873	2690	1.6353
0.726	10.5263	2700	1.6452
0.9583	10.5653	2710	1.6488
1.5887	10.6043	2720	1.6462
1.84	10.6433	2730	1.6327
1.2454	10.6823	2740	1.6320
0.7697	10.7212	2750	1.6397
0.8999	10.7602	2760	1.6519
1.5935	10.7992	2770	1.6455
1.8429	10.8382	2780	1.6322
1.3332	10.8772	2790	1.6325
0.6932	10.9162	2800	1.6411
0.9339	10.9552	2810	1.6452
1.1425	10.9942	2820	1.6433
1.889	11.0331	2830	1.6314
1.6236	11.0721	2840	1.6297
0.9341	11.1111	2850	1.6386
0.7124	11.1501	2860	1.6499
0.9938	11.1891	2870	1.6523
1.9533	11.2281	2880	1.6402
1.652	11.2671	2890	1.6317
1.0537	11.3060	2900	1.6360
0.7237	11.3450	2910	1.6447
0.9102	11.3840	2920	1.6498
1.9928	11.4230	2930	1.6416
1.6343	11.4620	2940	1.6310
1.0092	11.5010	2950	1.6310
0.7098	11.5400	2960	1.6398
0.9438	11.5789	2970	1.6471
1.9748	11.6179	2980	1.6426
1.6513	11.6569	2990	1.6329
0.9409	11.6959	3000	1.6320
0.7378	11.7349	3010	1.6380
0.9646	11.7739	3020	1.6446
1.9973	11.8129	3030	1.6421
1.6751	11.8519	3040	1.6340
1.0134	11.8908	3050	1.6319
0.7059	11.9298	3060	1.6380
0.9441	11.9688	3070	1.6446
1.4615	12.0078	3080	1.6384
1.9982	12.0468	3090	1.6308
1.443	12.0858	3100	1.6314
0.757	12.1248	3110	1.6383
0.8057	12.1637	3120	1.6475
1.2138	12.2027	3130	1.6491
1.9368	12.2417	3140	1.6360
1.3835	12.2807	3150	1.6329
0.7351	12.3197	3160	1.6369
0.7449	12.3587	3170	1.6447
1.1674	12.3977	3180	1.6486
1.9846	12.4366	3190	1.6387
1.427	12.4756	3200	1.6337
0.8101	12.5146	3210	1.6330
0.784	12.5536	3220	1.6422
1.1873	12.5926	3230	1.6475
2.0327	12.6316	3240	1.6411
1.5589	12.6706	3250	1.6352
0.8416	12.7096	3260	1.6339
0.7736	12.7485	3270	1.6395
1.1923	12.7875	3280	1.6452
1.982	12.8265	3290	1.6388
1.4829	12.8655	3300	1.6325
0.7603	12.9045	3310	1.6321
0.7794	12.9435	3320	1.6398
1.1345	12.9825	3330	1.6440
1.6627	13.0214	3340	1.6384
1.7906	13.0604	3350	1.6331
1.156	13.0994	3360	1.6328
0.7005	13.1384	3370	1.6374
0.8508	13.1774	3380	1.6448
1.6253	13.2164	3390	1.6488
1.7616	13.2554	3400	1.6398
1.2173	13.2943	3410	1.6350
0.6789	13.3333	3420	1.6361
0.8691	13.3723	3430	1.6443
1.6237	13.4113	3440	1.6468
1.7865	13.4503	3450	1.6403
1.2722	13.4893	3460	1.6344
0.6823	13.5283	3470	1.6343
0.8759	13.5673	3480	1.6424
1.5915	13.6062	3490	1.6451
1.7412	13.6452	3500	1.6398
1.1348	13.6842	3510	1.6353
0.6631	13.7232	3520	1.6351
0.8673	13.7622	3530	1.6405
1.6407	13.8012	3540	1.6414
1.7293	13.8402	3550	1.6381
1.2252	13.8791	3560	1.6345
0.673	13.9181	3570	1.6352
0.8808	13.9571	3580	1.6424
1.1056	13.9961	3590	1.6425
2.0034	14.0351	3600	1.6371
1.6216	14.0741	3610	1.6333
0.9085	14.1131	3620	1.6347
0.7023	14.1520	3630	1.6385
0.8838	14.1910	3640	1.6462
1.9691	14.2300	3650	1.6449
1.5183	14.2690	3660	1.6392
0.8066	14.3080	3670	1.6345
0.7142	14.3470	3680	1.6370
0.9286	14.3860	3690	1.6435
2.0038	14.4250	3700	1.6448
1.601	14.4639	3710	1.6402
0.9244	14.5029	3720	1.6338
0.6977	14.5419	3730	1.6354
0.8881	14.5809	3740	1.6426
1.9395	14.6199	3750	1.6441
1.6051	14.6589	3760	1.6402
1.008	14.6979	3770	1.6356
0.6612	14.7368	3780	1.6344
0.8928	14.7758	3790	1.6385
2.0018	14.8148	3800	1.6408
1.6221	14.8538	3810	1.6379
0.921	14.8928	3820	1.6342
0.6683	14.9318	3830	1.6352
0.9269	14.9708	3840	1.6378
1.4392	15.0097	3850	1.6385
1.9238	15.0487	3860	1.6371
1.4064	15.0877	3870	1.6364
0.739	15.1267	3880	1.6363
0.7393	15.1657	3890	1.6388
1.2143	15.2047	3900	1.6432
1.9835	15.2437	3910	1.6404
1.4888	15.2827	3920	1.6371
0.751	15.3216	3930	1.6354
0.7866	15.3606	3940	1.6358
1.2281	15.3996	3950	1.6399
1.9032	15.4386	3960	1.6397
1.3723	15.4776	3970	1.6383
0.7189	15.5166	3980	1.6355
0.7309	15.5556	3990	1.6376
1.2181	15.5945	4000	1.6403
1.9174	15.6335	4010	1.6391
1.2947	15.6725	4020	1.6375
0.7134	15.7115	4030	1.6362
0.7807	15.7505	4040	1.6373
1.2161	15.7895	4050	1.6405
1.939	15.8285	4060	1.6384
1.3711	15.8674	4070	1.6366
0.7455	15.9064	4080	1.6348
0.7517	15.9454	4090	1.6357
1.1461	15.9844	4100	1.6382
1.6604	16.0234	4110	1.6382
1.7005	16.0624	4120	1.6373
1.1151	16.1014	4130	1.6374
0.6724	16.1404	4140	1.6371
0.8738	16.1793	4150	1.6379
1.682	16.2183	4160	1.6391
1.6665	16.2573	4170	1.6375
1.1856	16.2963	4180	1.6366
0.6374	16.3353	4190	1.6366
0.8564	16.3743	4200	1.6389
1.651	16.4133	4210	1.6406
1.7028	16.4522	4220	1.6383
1.1576	16.4912	4230	1.6365
0.6747	16.5302	4240	1.6363
0.8639	16.5692	4250	1.6377
1.6352	16.6082	4260	1.6396
1.757	16.6472	4270	1.6390
1.0778	16.6862	4280	1.6388
0.644	16.7251	4290	1.6382
0.8102	16.7641	4300	1.6384
1.6998	16.8031	4310	1.6392
1.7384	16.8421	4320	1.6372
1.1362	16.8811	4330	1.6362
0.6682	16.9201	4340	1.6362
0.8349	16.9591	4350	1.6368
1.0263	16.9981	4360	1.6384
1.9781	17.0370	4370	1.6386
1.5533	17.0760	4380	1.6385
0.8585	17.1150	4390	1.6383
0.6612	17.1540	4400	1.6383
0.863	17.1930	4410	1.6391
1.9873	17.2320	4420	1.6395
1.5251	17.2710	4430	1.6384
0.8887	17.3099	4440	1.6377
0.6743	17.3489	4450	1.6377
0.8481	17.3879	4460	1.6383
1.9994	17.4269	4470	1.6386
1.5665	17.4659	4480	1.6382
0.8675	17.5049	4490	1.6379
0.6853	17.5439	4500	1.6382
0.9162	17.5828	4510	1.6391
2.0417	17.6218	4520	1.6391
1.5883	17.6608	4530	1.6380
0.9551	17.6998	4540	1.6376
0.6642	17.7388	4550	1.6377
0.9115	17.7778	4560	1.6378
2.05	17.8168	4570	1.6376
1.5172	17.8558	4580	1.6370
0.7651	17.8947	4590	1.6368
0.707	17.9337	4600	1.6370
0.8869	17.9727	4610	1.6373
1.4506	18.0117	4620	1.6377
1.9137	18.0507	4630	1.6374
1.3038	18.0897	4640	1.6371
0.6498	18.1287	4650	1.6371
0.7462	18.1676	4660	1.6374
1.2987	18.2066	4670	1.6378
1.938	18.2456	4680	1.6376
1.4307	18.2846	4690	1.6374
0.7548	18.3236	4700	1.6374
0.7381	18.3626	4710	1.6375
1.2753	18.4016	4720	1.6379
1.8916	18.4405	4730	1.6379
1.3062	18.4795	4740	1.6377
0.6819	18.5185	4750	1.6377
0.7771	18.5575	4760	1.6377
1.2419	18.5965	4770	1.6378
1.8299	18.6355	4780	1.6377
1.2819	18.6745	4790	1.6378
0.74	18.7135	4800	1.6376
0.7668	18.7524	4810	1.6377
1.2643	18.7914	4820	1.6378
1.9668	18.8304	4830	1.6378
1.365	18.8694	4840	1.6377
0.7506	18.9084	4850	1.6377
0.7601	18.9474	4860	1.6378
1.084	18.9864	4870	1.6380
1.6764	19.0253	4880	1.6380
1.6286	19.0643	4890	1.6380
1.111	19.1033	4900	1.6379
0.6536	19.1423	4910	1.6379
0.8533	19.1813	4920	1.6380
1.7116	19.2203	4930	1.6381
1.7198	19.2593	4940	1.6380
1.0983	19.2982	4950	1.6380
0.6609	19.3372	4960	1.6380
0.8579	19.3762	4970	1.6380
1.7292	19.4152	4980	1.6380
1.7108	19.4542	4990	1.6380
1.1063	19.4932	5000	1.6380
0.6559	19.5322	5010	1.6380
0.8649	19.5712	5020	1.6380
1.7209	19.6101	5030	1.6380
1.6668	19.6491	5040	1.6380
1.0578	19.6881	5050	1.6380
0.6209	19.7271	5060	1.6380
0.8345	19.7661	5070	1.6380
1.726	19.8051	5080	1.6380
1.641	19.8441	5090	1.6380
1.0477	19.8830	5100	1.6380
0.6487	19.9220	5110	1.6380
0.838	19.9610	5120	1.6380

Framework versions

• PEFT 0.12.0
• Transformers 4.45.0
• Pytorch 2.4.0+cu121
• Datasets 2.21.0
• Tokenizers 0.20.1