layoutlm-funsd

This model is a fine-tuned version of microsoft/layoutlm-base-uncased on the funsd dataset. It achieves the following results on the evaluation set:

• Loss: 0.6650
• Answer: {'precision': 0.7158712541620422, 'recall': 0.7972805933250927, 'f1': 0.7543859649122808, 'number': 809}
• Header: {'precision': 0.2982456140350877, 'recall': 0.2857142857142857, 'f1': 0.2918454935622318, 'number': 119}
• Question: {'precision': 0.7667238421955404, 'recall': 0.8394366197183099, 'f1': 0.8014343343792021, 'number': 1065}
• Overall Precision: 0.7212
• Overall Recall: 0.7893
• Overall F1: 0.7537
• Overall Accuracy: 0.8191

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 3e-05
• train_batch_size: 16
• eval_batch_size: 8
• seed: 42
• optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
• lr_scheduler_type: linear
• num_epochs: 15

Training results

Training Loss	Epoch	Step	Validation Loss	Answer	Header	Question	Overall Precision	Overall Recall	Overall F1	Overall Accuracy
1.7902	1.0	10	1.6058	{'precision': 0.0174496644295302, 'recall': 0.016069221260815822, 'f1': 0.01673101673101673, 'number': 809}	{'precision': 0.0, 'recall': 0.0, 'f1': 0.0, 'number': 119}	{'precision': 0.24484848484848484, 'recall': 0.18967136150234742, 'f1': 0.21375661375661376, 'number': 1065}	0.1369	0.1079	0.1207	0.3425
1.4512	2.0	20	1.2477	{'precision': 0.22826086956521738, 'recall': 0.23362175525339926, 'f1': 0.23091020158827122, 'number': 809}	{'precision': 0.0, 'recall': 0.0, 'f1': 0.0, 'number': 119}	{'precision': 0.4611066559743384, 'recall': 0.539906103286385, 'f1': 0.49740484429065746, 'number': 1065}	0.3680	0.3833	0.3755	0.5802
1.0772	3.0	30	0.9579	{'precision': 0.47790055248618785, 'recall': 0.4276885043263288, 'f1': 0.45140247879973905, 'number': 809}	{'precision': 0.05555555555555555, 'recall': 0.01680672268907563, 'f1': 0.025806451612903226, 'number': 119}	{'precision': 0.6270125223613596, 'recall': 0.6582159624413145, 'f1': 0.6422354557947779, 'number': 1065}	0.5586	0.5263	0.5420	0.6919
0.8282	4.0	40	0.7735	{'precision': 0.6132368148914168, 'recall': 0.7330037082818294, 'f1': 0.6677927927927928, 'number': 809}	{'precision': 0.17647058823529413, 'recall': 0.10084033613445378, 'f1': 0.1283422459893048, 'number': 119}	{'precision': 0.6726649528706083, 'recall': 0.7370892018779343, 'f1': 0.703405017921147, 'number': 1065}	0.6312	0.6974	0.6627	0.7621
0.6763	5.0	50	0.7086	{'precision': 0.6333333333333333, 'recall': 0.7515451174289246, 'f1': 0.6873940079140758, 'number': 809}	{'precision': 0.325, 'recall': 0.2184873949579832, 'f1': 0.26130653266331655, 'number': 119}	{'precision': 0.6769731489015459, 'recall': 0.7812206572769953, 'f1': 0.7253705318221447, 'number': 1065}	0.6461	0.7356	0.6879	0.7869
0.5577	6.0	60	0.6736	{'precision': 0.6542155816435432, 'recall': 0.757725587144623, 'f1': 0.7021764032073311, 'number': 809}	{'precision': 0.32926829268292684, 'recall': 0.226890756302521, 'f1': 0.26865671641791045, 'number': 119}	{'precision': 0.6952822892498066, 'recall': 0.844131455399061, 'f1': 0.7625106022052586, 'number': 1065}	0.6657	0.7722	0.7150	0.7955
0.4901	7.0	70	0.6510	{'precision': 0.6706263498920086, 'recall': 0.7676143386897404, 'f1': 0.7158501440922191, 'number': 809}	{'precision': 0.27927927927927926, 'recall': 0.2605042016806723, 'f1': 0.26956521739130435, 'number': 119}	{'precision': 0.7412765957446809, 'recall': 0.8178403755868544, 'f1': 0.7776785714285714, 'number': 1065}	0.6885	0.7642	0.7244	0.7998
0.4474	8.0	80	0.6389	{'precision': 0.6828478964401294, 'recall': 0.7824474660074165, 'f1': 0.7292626728110598, 'number': 809}	{'precision': 0.3137254901960784, 'recall': 0.2689075630252101, 'f1': 0.2895927601809955, 'number': 119}	{'precision': 0.7523564695801199, 'recall': 0.8244131455399061, 'f1': 0.7867383512544801, 'number': 1065}	0.7026	0.7742	0.7367	0.8049
0.4055	9.0	90	0.6371	{'precision': 0.6855277475516867, 'recall': 0.7787391841779975, 'f1': 0.7291666666666666, 'number': 809}	{'precision': 0.288135593220339, 'recall': 0.2857142857142857, 'f1': 0.2869198312236287, 'number': 119}	{'precision': 0.7368852459016394, 'recall': 0.844131455399061, 'f1': 0.7868708971553611, 'number': 1065}	0.6925	0.7842	0.7355	0.8111
0.3597	10.0	100	0.6547	{'precision': 0.7027932960893855, 'recall': 0.7775030902348579, 'f1': 0.7382629107981221, 'number': 809}	{'precision': 0.25925925925925924, 'recall': 0.29411764705882354, 'f1': 0.2755905511811024, 'number': 119}	{'precision': 0.7463330457290768, 'recall': 0.812206572769953, 'f1': 0.7778776978417264, 'number': 1065}	0.6985	0.7672	0.7312	0.8070
0.3295	11.0	110	0.6618	{'precision': 0.709070796460177, 'recall': 0.792336217552534, 'f1': 0.7483946293053124, 'number': 809}	{'precision': 0.3333333333333333, 'recall': 0.25210084033613445, 'f1': 0.28708133971291866, 'number': 119}	{'precision': 0.7857142857142857, 'recall': 0.8366197183098592, 'f1': 0.8103683492496588, 'number': 1065}	0.7340	0.7837	0.7581	0.8106
0.3169	12.0	120	0.6639	{'precision': 0.7094972067039106, 'recall': 0.7849196538936959, 'f1': 0.7453051643192488, 'number': 809}	{'precision': 0.3017241379310345, 'recall': 0.29411764705882354, 'f1': 0.29787234042553185, 'number': 119}	{'precision': 0.7582417582417582, 'recall': 0.8422535211267606, 'f1': 0.7980427046263344, 'number': 1065}	0.7142	0.7863	0.7485	0.8152
0.2951	13.0	130	0.6653	{'precision': 0.7094972067039106, 'recall': 0.7849196538936959, 'f1': 0.7453051643192488, 'number': 809}	{'precision': 0.3063063063063063, 'recall': 0.2857142857142857, 'f1': 0.2956521739130435, 'number': 119}	{'precision': 0.7784588441330998, 'recall': 0.8347417840375587, 'f1': 0.805618486633439, 'number': 1065}	0.7253	0.7817	0.7525	0.8167
0.2872	14.0	140	0.6667	{'precision': 0.7116022099447514, 'recall': 0.796044499381953, 'f1': 0.751458576429405, 'number': 809}	{'precision': 0.2982456140350877, 'recall': 0.2857142857142857, 'f1': 0.2918454935622318, 'number': 119}	{'precision': 0.7737162750217581, 'recall': 0.8347417840375587, 'f1': 0.803071364046974, 'number': 1065}	0.7228	0.7863	0.7532	0.8179
0.2779	15.0	150	0.6650	{'precision': 0.7158712541620422, 'recall': 0.7972805933250927, 'f1': 0.7543859649122808, 'number': 809}	{'precision': 0.2982456140350877, 'recall': 0.2857142857142857, 'f1': 0.2918454935622318, 'number': 119}	{'precision': 0.7667238421955404, 'recall': 0.8394366197183099, 'f1': 0.8014343343792021, 'number': 1065}	0.7212	0.7893	0.7537	0.8191

Framework versions

• Transformers 4.25.1
• Pytorch 1.12.1
• Datasets 2.8.0
• Tokenizers 0.13.2