ModernBERT_wine_quality_reviews_ft

This model is a fine-tuned version of answerdotai/ModernBERT-base on the None dataset. It achieves the following results on the evaluation set:

• Loss: 0.6671
• Accuracy: 0.7019
• F1: 0.7024

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 0.0001
• train_batch_size: 32
• eval_batch_size: 32
• seed: 42
• optimizer: Use OptimizerNames.ADAMW_TORCH with betas=(0.8,0.8) and epsilon=1e-06 and optimizer_args=No additional optimizer arguments
• lr_scheduler_type: linear
• lr_scheduler_warmup_ratio: 0.15
• num_epochs: 5

Training results

Training Loss	Epoch	Step	Validation Loss	Accuracy	F1
1.1457	0.0826	350	0.9894	0.5461	0.5305
0.9441	0.1653	700	1.1213	0.4977	0.4827
0.8589	0.2479	1050	0.8232	0.6297	0.6277
0.8131	0.3306	1400	0.8268	0.6177	0.5956
0.7837	0.4132	1750	0.7474	0.6679	0.6663
0.7726	0.4959	2100	0.8008	0.6397	0.6269
0.7576	0.5785	2450	0.7571	0.6533	0.6550
0.7528	0.6612	2800	0.7414	0.6666	0.6598
0.7588	0.7438	3150	0.7627	0.6588	0.6397
0.7416	0.8264	3500	0.7259	0.6736	0.6739
0.7303	0.9091	3850	0.7052	0.6847	0.6812
0.7313	0.9917	4200	0.7059	0.6860	0.6799
0.6647	1.0744	4550	0.7002	0.6890	0.6887
0.6606	1.1570	4900	0.7712	0.6583	0.6502
0.65	1.2397	5250	0.6868	0.6917	0.6904
0.6464	1.3223	5600	0.7371	0.6757	0.6673
0.6494	1.4050	5950	0.7323	0.6751	0.6724
0.6505	1.4876	6300	0.6952	0.6877	0.6856
0.6499	1.5702	6650	0.6935	0.6893	0.6812
0.6399	1.6529	7000	0.7099	0.6873	0.6826
0.632	1.7355	7350	0.6912	0.6942	0.6915
0.6488	1.8182	7700	0.6741	0.6971	0.6972
0.6331	1.9008	8050	0.6881	0.6933	0.6932
0.6339	1.9835	8400	0.6671	0.7019	0.7024
0.4914	2.0661	8750	0.7598	0.6989	0.6982
0.4498	2.1488	9100	0.7617	0.6997	0.6996
0.4407	2.2314	9450	0.7674	0.6950	0.6945
0.4468	2.3140	9800	0.7978	0.6946	0.6932
0.4486	2.3967	10150	0.7718	0.6929	0.6926
0.4462	2.4793	10500	0.7928	0.6808	0.6811
0.4483	2.5620	10850	0.7678	0.6957	0.6966
0.4347	2.6446	11200	0.7687	0.6935	0.6938
0.4429	2.7273	11550	0.7496	0.6969	0.6973
0.4415	2.8099	11900	0.7621	0.6968	0.6963

Framework versions

• Transformers 4.48.1
• Pytorch 2.5.1+cu121
• Datasets 3.2.0
• Tokenizers 0.21.0