layoutlm-sroie

This model is a fine-tuned version of microsoft/layoutlm-base-uncased on the sroie dataset. It achieves the following results on the evaluation set:

• Loss: 0.0363
• Address: {'precision': 0.901685393258427, 'recall': 0.9250720461095101, 'f1': 0.9132290184921764, 'number': 347}
• Company: {'precision': 0.904891304347826, 'recall': 0.9596541786743515, 'f1': 0.9314685314685315, 'number': 347}
• Date: {'precision': 0.9913544668587896, 'recall': 0.9913544668587896, 'f1': 0.9913544668587896, 'number': 347}
• Total: {'precision': 0.8155080213903744, 'recall': 0.8789625360230547, 'f1': 0.8460471567267684, 'number': 347}
• Overall Precision: 0.9017
• Overall Recall: 0.9388
• Overall F1: 0.9199
• Overall Accuracy: 0.9930

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 3e-05
• train_batch_size: 16
• eval_batch_size: 8
• seed: 42
• optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
• lr_scheduler_type: linear
• num_epochs: 15

Training results

Training Loss	Epoch	Step	Validation Loss	Address	Company	Date	Total	Overall Precision	Overall Recall	Overall F1	Overall Accuracy
0.5189	1.0	40	0.1280	{'precision': 0.7891891891891892, 'recall': 0.8414985590778098, 'f1': 0.8145048814504882, 'number': 347}	{'precision': 0.6987012987012987, 'recall': 0.7752161383285303, 'f1': 0.7349726775956286, 'number': 347}	{'precision': 0.6651982378854625, 'recall': 0.8703170028818443, 'f1': 0.7540574282147315, 'number': 347}	{'precision': 0.0, 'recall': 0.0, 'f1': 0.0, 'number': 347}	0.7138	0.6218	0.6646	0.9650
0.0849	2.0	80	0.0558	{'precision': 0.8753462603878116, 'recall': 0.9106628242074928, 'f1': 0.8926553672316384, 'number': 347}	{'precision': 0.8102189781021898, 'recall': 0.9596541786743515, 'f1': 0.8786279683377309, 'number': 347}	{'precision': 0.9178082191780822, 'recall': 0.9654178674351584, 'f1': 0.9410112359550562, 'number': 347}	{'precision': 0.5179282868525896, 'recall': 0.3746397694524496, 'f1': 0.43478260869565216, 'number': 347}	0.8026	0.8026	0.8026	0.9851
0.0447	3.0	120	0.0435	{'precision': 0.8997214484679665, 'recall': 0.930835734870317, 'f1': 0.9150141643059491, 'number': 347}	{'precision': 0.8954423592493298, 'recall': 0.962536023054755, 'f1': 0.9277777777777777, 'number': 347}	{'precision': 0.96045197740113, 'recall': 0.9798270893371758, 'f1': 0.9700427960057061, 'number': 347}	{'precision': 0.6222222222222222, 'recall': 0.6455331412103746, 'f1': 0.6336633663366337, 'number': 347}	0.8444	0.8797	0.8617	0.9890
0.0318	4.0	160	0.0347	{'precision': 0.8777777777777778, 'recall': 0.9106628242074928, 'f1': 0.8939179632248939, 'number': 347}	{'precision': 0.9153005464480874, 'recall': 0.9654178674351584, 'f1': 0.9396914446002805, 'number': 347}	{'precision': 0.9913544668587896, 'recall': 0.9913544668587896, 'f1': 0.9913544668587896, 'number': 347}	{'precision': 0.671957671957672, 'recall': 0.7319884726224783, 'f1': 0.7006896551724137, 'number': 347}	0.8608	0.8999	0.8799	0.9910
0.0245	5.0	200	0.0360	{'precision': 0.8885793871866295, 'recall': 0.9193083573487032, 'f1': 0.9036827195467422, 'number': 347}	{'precision': 0.8909574468085106, 'recall': 0.9654178674351584, 'f1': 0.9266943291839556, 'number': 347}	{'precision': 0.9913294797687862, 'recall': 0.9884726224783862, 'f1': 0.98989898989899, 'number': 347}	{'precision': 0.7873754152823921, 'recall': 0.6829971181556196, 'f1': 0.7314814814814815, 'number': 347}	0.8929	0.8890	0.8910	0.9910
0.0171	6.0	240	0.0325	{'precision': 0.8932584269662921, 'recall': 0.9164265129682997, 'f1': 0.9046941678520626, 'number': 347}	{'precision': 0.912568306010929, 'recall': 0.962536023054755, 'f1': 0.9368863955119215, 'number': 347}	{'precision': 0.991304347826087, 'recall': 0.9855907780979827, 'f1': 0.9884393063583815, 'number': 347}	{'precision': 0.823170731707317, 'recall': 0.7780979827089337, 'f1': 0.8, 'number': 347}	0.9061	0.9107	0.9084	0.9926
0.0133	7.0	280	0.0352	{'precision': 0.8969359331476323, 'recall': 0.9279538904899135, 'f1': 0.9121813031161472, 'number': 347}	{'precision': 0.9103260869565217, 'recall': 0.9654178674351584, 'f1': 0.937062937062937, 'number': 347}	{'precision': 0.9885057471264368, 'recall': 0.9913544668587896, 'f1': 0.9899280575539569, 'number': 347}	{'precision': 0.7801608579088471, 'recall': 0.8386167146974063, 'f1': 0.8083333333333332, 'number': 347}	0.8923	0.9308	0.9111	0.9922
0.013	8.0	320	0.0338	{'precision': 0.889196675900277, 'recall': 0.9250720461095101, 'f1': 0.9067796610169492, 'number': 347}	{'precision': 0.9103260869565217, 'recall': 0.9654178674351584, 'f1': 0.937062937062937, 'number': 347}	{'precision': 0.9885057471264368, 'recall': 0.9913544668587896, 'f1': 0.9899280575539569, 'number': 347}	{'precision': 0.7887700534759359, 'recall': 0.8501440922190202, 'f1': 0.8183079056865465, 'number': 347}	0.8925	0.9330	0.9123	0.9927
0.0105	9.0	360	0.0378	{'precision': 0.8885793871866295, 'recall': 0.9193083573487032, 'f1': 0.9036827195467422, 'number': 347}	{'precision': 0.9081081081081082, 'recall': 0.968299711815562, 'f1': 0.9372384937238494, 'number': 347}	{'precision': 0.9913294797687862, 'recall': 0.9884726224783862, 'f1': 0.98989898989899, 'number': 347}	{'precision': 0.8096590909090909, 'recall': 0.8213256484149856, 'f1': 0.8154506437768241, 'number': 347}	0.8991	0.9244	0.9115	0.9923
0.0094	10.0	400	0.0353	{'precision': 0.901685393258427, 'recall': 0.9250720461095101, 'f1': 0.9132290184921764, 'number': 347}	{'precision': 0.904891304347826, 'recall': 0.9596541786743515, 'f1': 0.9314685314685315, 'number': 347}	{'precision': 0.9913544668587896, 'recall': 0.9913544668587896, 'f1': 0.9913544668587896, 'number': 347}	{'precision': 0.8142076502732241, 'recall': 0.8587896253602305, 'f1': 0.8359046283309958, 'number': 347}	0.9019	0.9337	0.9175	0.9929
0.0078	11.0	440	0.0373	{'precision': 0.8938547486033519, 'recall': 0.9221902017291066, 'f1': 0.9078014184397163, 'number': 347}	{'precision': 0.9098360655737705, 'recall': 0.9596541786743515, 'f1': 0.9340813464235624, 'number': 347}	{'precision': 0.9913544668587896, 'recall': 0.9913544668587896, 'f1': 0.9913544668587896, 'number': 347}	{'precision': 0.8150134048257373, 'recall': 0.8760806916426513, 'f1': 0.8444444444444444, 'number': 347}	0.9010	0.9373	0.9188	0.9928
0.0074	12.0	480	0.0379	{'precision': 0.8994413407821229, 'recall': 0.9279538904899135, 'f1': 0.9134751773049646, 'number': 347}	{'precision': 0.9128065395095368, 'recall': 0.9654178674351584, 'f1': 0.938375350140056, 'number': 347}	{'precision': 0.9913544668587896, 'recall': 0.9913544668587896, 'f1': 0.9913544668587896, 'number': 347}	{'precision': 0.835195530726257, 'recall': 0.861671469740634, 'f1': 0.8482269503546098, 'number': 347}	0.9091	0.9366	0.9226	0.9931
0.007	13.0	520	0.0357	{'precision': 0.9019607843137255, 'recall': 0.9279538904899135, 'f1': 0.9147727272727272, 'number': 347}	{'precision': 0.9024390243902439, 'recall': 0.9596541786743515, 'f1': 0.9301675977653631, 'number': 347}	{'precision': 0.9913544668587896, 'recall': 0.9913544668587896, 'f1': 0.9913544668587896, 'number': 347}	{'precision': 0.8328767123287671, 'recall': 0.8760806916426513, 'f1': 0.853932584269663, 'number': 347}	0.9061	0.9388	0.9222	0.9932
0.0069	14.0	560	0.0361	{'precision': 0.901685393258427, 'recall': 0.9250720461095101, 'f1': 0.9132290184921764, 'number': 347}	{'precision': 0.9051490514905149, 'recall': 0.962536023054755, 'f1': 0.9329608938547486, 'number': 347}	{'precision': 0.9913544668587896, 'recall': 0.9913544668587896, 'f1': 0.9913544668587896, 'number': 347}	{'precision': 0.8046875, 'recall': 0.8904899135446686, 'f1': 0.8454172366621068, 'number': 347}	0.8984	0.9424	0.9198	0.9930
0.0065	15.0	600	0.0363	{'precision': 0.901685393258427, 'recall': 0.9250720461095101, 'f1': 0.9132290184921764, 'number': 347}	{'precision': 0.904891304347826, 'recall': 0.9596541786743515, 'f1': 0.9314685314685315, 'number': 347}	{'precision': 0.9913544668587896, 'recall': 0.9913544668587896, 'f1': 0.9913544668587896, 'number': 347}	{'precision': 0.8155080213903744, 'recall': 0.8789625360230547, 'f1': 0.8460471567267684, 'number': 347}	0.9017	0.9388	0.9199	0.9930

Framework versions

• Transformers 4.28.0
• Pytorch 2.1.0+cu118
• Datasets 2.15.0
• Tokenizers 0.12.1