PhoBert_Lexical_Hosting_Dataset10K

This model is a fine-tuned version of vinai/phobert-base-v2 on an unknown dataset. It achieves the following results on the evaluation set:

• Loss: 0.4273
• Accuracy: 0.8794
• F1: 0.8974

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 2e-05
• train_batch_size: 64
• eval_batch_size: 64
• seed: 42
• optimizer: Use adamw_torch with betas=(0.9,0.999) and epsilon=1e-08 and optimizer_args=No additional optimizer arguments
• lr_scheduler_type: linear
• num_epochs: 15

Training results

Training Loss	Epoch	Step	Validation Loss	Accuracy	F1
1.7448	1.2422	200	0.5205	0.7188	0.7907
1.7737	2.4845	400	0.3096	0.9020	0.9067
1.3408	3.7267	600	0.3578	0.8771	0.8938
1.7858	4.9689	800	0.3340	0.8927	0.9039
1.1154	6.2112	1000	0.2603	0.9155	0.9179
1.2382	7.4534	1200	0.4342	0.8434	0.8738
1.1267	8.6957	1400	0.3936	0.8697	0.8904
1.0833	9.9379	1600	0.3876	0.8677	0.8892
1.1027	11.1801	1800	0.4320	0.8757	0.8948
0.8329	12.4224	2000	0.3797	0.8832	0.8995
1.1287	13.6646	2200	0.4414	0.8677	0.8900
1.1326	14.9068	2400	0.4273	0.8794	0.8974

Framework versions

• Transformers 4.46.2
• Pytorch 2.1.2
• Datasets 2.20.0
• Tokenizers 0.20.3