nb-bert-base-edu-scorer-lr3e4-bs32-swe

This model is a fine-tuned version of NbAiLab/nb-bert-base on an unknown dataset. It achieves the following results on the evaluation set:

• Loss: 0.7996
• Mse: 0.7996
• Mae: 0.6982
• Rmse: 0.8942
• R2: 0.5844

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 0.0003
• train_batch_size: 32
• eval_batch_size: 32
• seed: 42
• optimizer: Use OptimizerNames.ADAMW_TORCH with betas=(0.9,0.999) and epsilon=1e-08 and optimizer_args=No additional optimizer arguments
• lr_scheduler_type: linear
• num_epochs: 20
• mixed_precision_training: Native AMP

Training results

Training Loss	Epoch	Step	Validation Loss	Mse	Mae	Rmse	R2
No log	0	0	6.0700	6.0700	2.0111	2.4637	-2.0534
1.1272	0.3397	1000	1.0319	1.0319	0.7925	1.0158	0.4809
1.0837	0.6793	2000	1.0182	1.0182	0.7850	1.0091	0.4878
1.0446	1.0190	3000	0.9967	0.9967	0.7683	0.9983	0.4986
1.0863	1.3587	4000	0.9580	0.9580	0.7534	0.9788	0.5181
1.0601	1.6984	5000	1.0061	1.0061	0.7796	1.0030	0.4939
0.9957	2.0380	6000	1.3005	1.3005	0.8945	1.1404	0.3458
1.0104	2.3777	7000	0.9569	0.9569	0.7483	0.9782	0.5187
1.04	2.7174	8000	0.9457	0.9457	0.7648	0.9724	0.5243
1.0445	3.0571	9000	0.9641	0.9641	0.7445	0.9819	0.5150
0.9931	3.3967	10000	0.9549	0.9549	0.7430	0.9772	0.5197
1.0134	3.7364	11000	0.9791	0.9791	0.7549	0.9895	0.5075
1.0366	4.0761	12000	1.0248	1.0248	0.7673	1.0123	0.4845
1.0106	4.4158	13000	0.9321	0.9321	0.7378	0.9654	0.5311
0.9409	4.7554	14000	0.9553	0.9553	0.7420	0.9774	0.5194
0.925	5.0951	15000	1.1885	1.1885	0.8538	1.0902	0.4021
0.961	5.4348	16000	0.9201	0.9201	0.7341	0.9592	0.5372
1.0096	5.7745	17000	0.9192	0.9192	0.7448	0.9587	0.5376
0.9696	6.1141	18000	0.9543	0.9543	0.7445	0.9769	0.5199
0.9737	6.4538	19000	0.9287	0.9287	0.7281	0.9637	0.5328
0.9725	6.7935	20000	0.9589	0.9589	0.7557	0.9792	0.5176
0.9683	7.1332	21000	0.9079	0.9079	0.7354	0.9528	0.5433
0.9606	7.4728	22000	0.9885	0.9885	0.7481	0.9943	0.5027
0.9846	7.8125	23000	1.0081	1.0081	0.7895	1.0041	0.4929
0.9671	8.1522	24000	0.9174	0.9174	0.7251	0.9578	0.5385
0.9679	8.4918	25000	0.9212	0.9212	0.7447	0.9598	0.5366
0.9503	8.8315	26000	0.9418	0.9418	0.7343	0.9705	0.5262
0.9858	9.1712	27000	0.9186	0.9186	0.7325	0.9584	0.5379
0.969	9.5109	28000	0.9219	0.9219	0.7352	0.9602	0.5362
1.0022	9.8505	29000	0.9458	0.9458	0.7400	0.9725	0.5242
0.942	10.1902	30000	0.9746	0.9746	0.7416	0.9872	0.5097
0.9633	10.5299	31000	0.9173	0.9173	0.7218	0.9577	0.5386
0.9463	10.8696	32000	0.9528	0.9528	0.7443	0.9761	0.5207
0.9803	11.2092	33000	0.9042	0.9042	0.7226	0.9509	0.5452
0.9318	11.5489	34000	0.9030	0.9030	0.7270	0.9502	0.5458
0.9176	11.8886	35000	0.9378	0.9378	0.7314	0.9684	0.5283
0.9063	12.2283	36000	0.8946	0.8946	0.7191	0.9458	0.5500
0.9754	12.5679	37000	0.8938	0.8938	0.7207	0.9454	0.5504
0.9291	12.9076	38000	0.9565	0.9565	0.7503	0.9780	0.5188
0.9142	13.2473	39000	0.9238	0.9238	0.7278	0.9611	0.5353
0.9579	13.5870	40000	0.9267	0.9267	0.7335	0.9627	0.5338
0.9556	13.9266	41000	0.9083	0.9083	0.7197	0.9531	0.5431
0.9465	14.2663	42000	0.9228	0.9228	0.7287	0.9606	0.5358
0.9455	14.6060	43000	0.9122	0.9122	0.7201	0.9551	0.5411
0.9294	14.9457	44000	0.9241	0.9241	0.7307	0.9613	0.5351
0.9038	15.2853	45000	0.8985	0.8985	0.7229	0.9479	0.5480
0.9154	15.625	46000	0.9374	0.9374	0.7451	0.9682	0.5285
0.9482	15.9647	47000	0.9487	0.9487	0.7413	0.9740	0.5228
0.9568	16.3043	48000	0.9006	0.9006	0.7224	0.9490	0.5470
0.9902	16.6440	49000	0.9042	0.9042	0.7200	0.9509	0.5451
0.9364	16.9837	50000	0.9053	0.9053	0.7263	0.9515	0.5446
0.9432	17.3234	51000	0.9139	0.9139	0.7331	0.9560	0.5403
0.9288	17.6630	52000	0.9165	0.9165	0.7285	0.9573	0.5390
0.9385	18.0027	53000	0.9081	0.9081	0.7243	0.9529	0.5432
0.9157	18.3424	54000	0.9449	0.9449	0.7435	0.9720	0.5247
0.9666	18.6821	55000	0.8962	0.8962	0.7174	0.9467	0.5492
0.931	19.0217	56000	0.8971	0.8971	0.7222	0.9471	0.5487
0.96	19.3614	57000	0.8975	0.8975	0.7230	0.9473	0.5485
0.9257	19.7011	58000	0.9041	0.9041	0.7252	0.9508	0.5452

Framework versions

• Transformers 4.55.0
• Pytorch 2.5.1+cu121
• Datasets 4.0.0
• Tokenizers 0.21.4