layoutlm-funsd

This model is a fine-tuned version of microsoft/layoutlm-base-uncased on the funsd dataset. It achieves the following results on the evaluation set:

• Loss: 1.1103
• Answer: {'precision': 0.4171539961013645, 'recall': 0.5290482076637825, 'f1': 0.4664850136239782, 'number': 809}
• Header: {'precision': 0.26595744680851063, 'recall': 0.21008403361344538, 'f1': 0.23474178403755866, 'number': 119}
• Question: {'precision': 0.5105058365758754, 'recall': 0.615962441314554, 'f1': 0.5582978723404256, 'number': 1065}
• Overall Precision: 0.4611
• Overall Recall: 0.5564
• Overall F1: 0.5043
• Overall Accuracy: 0.6256

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 3e-05
• train_batch_size: 16
• eval_batch_size: 8
• seed: 42
• optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
• lr_scheduler_type: linear
• num_epochs: 15
• mixed_precision_training: Native AMP

Training results

Training Loss	Epoch	Step	Validation Loss	Answer	Header	Question	Overall Precision	Overall Recall	Overall F1	Overall Accuracy
1.7545	1.0	10	1.4910	{'precision': 0.04744787922358016, 'recall': 0.0815822002472188, 'f1': 0.06000000000000001, 'number': 809}	{'precision': 0.0, 'recall': 0.0, 'f1': 0.0, 'number': 119}	{'precision': 0.2316715542521994, 'recall': 0.29671361502347415, 'f1': 0.2601893783449979, 'number': 1065}	0.1386	0.1917	0.1608	0.3843
1.4327	2.0	20	1.3684	{'precision': 0.1908983451536643, 'recall': 0.3992583436341162, 'f1': 0.258296681327469, 'number': 809}	{'precision': 0.08333333333333333, 'recall': 0.01680672268907563, 'f1': 0.027972027972027972, 'number': 119}	{'precision': 0.2686771761480466, 'recall': 0.36807511737089205, 'f1': 0.3106180665610142, 'number': 1065}	0.2258	0.3598	0.2775	0.4199
1.3	3.0	30	1.2336	{'precision': 0.23386581469648562, 'recall': 0.45241038318912236, 'f1': 0.3083403538331929, 'number': 809}	{'precision': 0.23404255319148937, 'recall': 0.09243697478991597, 'f1': 0.13253012048192772, 'number': 119}	{'precision': 0.3207196029776675, 'recall': 0.48544600938967136, 'f1': 0.3862532685842361, 'number': 1065}	0.2773	0.4486	0.3427	0.4777
1.1799	4.0	40	1.1284	{'precision': 0.26886145404663925, 'recall': 0.484548825710754, 'f1': 0.3458314953683282, 'number': 809}	{'precision': 0.2903225806451613, 'recall': 0.226890756302521, 'f1': 0.25471698113207547, 'number': 119}	{'precision': 0.369108049311095, 'recall': 0.4779342723004695, 'f1': 0.41653027823240585, 'number': 1065}	0.3167	0.4656	0.3770	0.5629
1.0681	5.0	50	1.1019	{'precision': 0.2949346405228758, 'recall': 0.446229913473424, 'f1': 0.35514018691588783, 'number': 809}	{'precision': 0.3373493975903614, 'recall': 0.23529411764705882, 'f1': 0.2772277227722772, 'number': 119}	{'precision': 0.38892345986309895, 'recall': 0.5868544600938967, 'f1': 0.46781437125748504, 'number': 1065}	0.3480	0.5088	0.4133	0.5724
0.9791	6.0	60	1.2060	{'precision': 0.33286810886252616, 'recall': 0.5896168108776267, 'f1': 0.4255129348795718, 'number': 809}	{'precision': 0.4, 'recall': 0.20168067226890757, 'f1': 0.2681564245810056, 'number': 119}	{'precision': 0.45607476635514016, 'recall': 0.4582159624413146, 'f1': 0.45714285714285713, 'number': 1065}	0.3859	0.4962	0.4342	0.5718
0.9138	7.0	70	1.0604	{'precision': 0.37743589743589745, 'recall': 0.45488257107540175, 'f1': 0.4125560538116592, 'number': 809}	{'precision': 0.3333333333333333, 'recall': 0.25210084033613445, 'f1': 0.28708133971291866, 'number': 119}	{'precision': 0.4469026548672566, 'recall': 0.5690140845070423, 'f1': 0.5006195786864932, 'number': 1065}	0.4147	0.5038	0.4549	0.5983
0.8555	8.0	80	1.0361	{'precision': 0.3559928443649374, 'recall': 0.4919653893695921, 'f1': 0.4130773222625843, 'number': 809}	{'precision': 0.3076923076923077, 'recall': 0.20168067226890757, 'f1': 0.2436548223350254, 'number': 119}	{'precision': 0.45045045045045046, 'recall': 0.6103286384976526, 'f1': 0.5183413078149921, 'number': 1065}	0.4062	0.5379	0.4629	0.6104
0.8062	9.0	90	1.0676	{'precision': 0.37511520737327186, 'recall': 0.5030902348578492, 'f1': 0.4297782470960929, 'number': 809}	{'precision': 0.31521739130434784, 'recall': 0.24369747899159663, 'f1': 0.27488151658767773, 'number': 119}	{'precision': 0.4796310530361261, 'recall': 0.5859154929577465, 'f1': 0.5274725274725274, 'number': 1065}	0.4278	0.5319	0.4742	0.6094
0.7981	10.0	100	1.0901	{'precision': 0.3904109589041096, 'recall': 0.4932014833127318, 'f1': 0.4358274167121791, 'number': 809}	{'precision': 0.3132530120481928, 'recall': 0.2184873949579832, 'f1': 0.25742574257425743, 'number': 119}	{'precision': 0.47112462006079026, 'recall': 0.5821596244131455, 'f1': 0.5207895842083158, 'number': 1065}	0.4316	0.5243	0.4735	0.6113
0.7159	11.0	110	1.1141	{'precision': 0.3889908256880734, 'recall': 0.5241038318912238, 'f1': 0.4465508162190627, 'number': 809}	{'precision': 0.26732673267326734, 'recall': 0.226890756302521, 'f1': 0.24545454545454548, 'number': 119}	{'precision': 0.5027844073190135, 'recall': 0.5934272300469483, 'f1': 0.5443583118001722, 'number': 1065}	0.4424	0.5434	0.4877	0.6139
0.7242	12.0	120	1.0786	{'precision': 0.39233576642335766, 'recall': 0.5315203955500618, 'f1': 0.4514435695538058, 'number': 809}	{'precision': 0.2926829268292683, 'recall': 0.20168067226890757, 'f1': 0.23880597014925373, 'number': 119}	{'precision': 0.5096674400618716, 'recall': 0.6187793427230047, 'f1': 0.5589482612383375, 'number': 1065}	0.4504	0.5585	0.4987	0.6172
0.6895	13.0	130	1.1184	{'precision': 0.4066427289048474, 'recall': 0.5599505562422744, 'f1': 0.4711388455538222, 'number': 809}	{'precision': 0.2696629213483146, 'recall': 0.20168067226890757, 'f1': 0.23076923076923078, 'number': 119}	{'precision': 0.5230125523012552, 'recall': 0.5868544600938967, 'f1': 0.5530973451327434, 'number': 1065}	0.4595	0.5529	0.5019	0.6134
0.6605	14.0	140	1.1015	{'precision': 0.4114737883283877, 'recall': 0.5142150803461063, 'f1': 0.45714285714285713, 'number': 809}	{'precision': 0.2631578947368421, 'recall': 0.21008403361344538, 'f1': 0.23364485981308414, 'number': 119}	{'precision': 0.5068702290076336, 'recall': 0.6234741784037559, 'f1': 0.5591578947368421, 'number': 1065}	0.4574	0.5544	0.5012	0.6242
0.6498	15.0	150	1.1103	{'precision': 0.4171539961013645, 'recall': 0.5290482076637825, 'f1': 0.4664850136239782, 'number': 809}	{'precision': 0.26595744680851063, 'recall': 0.21008403361344538, 'f1': 0.23474178403755866, 'number': 119}	{'precision': 0.5105058365758754, 'recall': 0.615962441314554, 'f1': 0.5582978723404256, 'number': 1065}	0.4611	0.5564	0.5043	0.6256

Framework versions

• Transformers 4.38.1
• Pytorch 2.2.2+cu118
• Datasets 2.18.0
• Tokenizers 0.15.2