layoutlm-funsd

This model is a fine-tuned version of microsoft/layoutlm-base-uncased on the funsd dataset. It achieves the following results on the evaluation set:

• Loss: 0.6877
• Answer: {'precision': 0.7073707370737073, 'recall': 0.7948084054388134, 'f1': 0.7485448195576251, 'number': 809}
• Header: {'precision': 0.3435114503816794, 'recall': 0.37815126050420167, 'f1': 0.36, 'number': 119}
• Question: {'precision': 0.7785651018600531, 'recall': 0.8253521126760563, 'f1': 0.8012762078395624, 'number': 1065}
• Overall Precision: 0.7225
• Overall Recall: 0.7863
• Overall F1: 0.7530
• Overall Accuracy: 0.7977

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 3e-05
• train_batch_size: 16
• eval_batch_size: 8
• seed: 42
• optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
• lr_scheduler_type: linear
• num_epochs: 15
• mixed_precision_training: Native AMP

Training results

Training Loss	Epoch	Step	Validation Loss	Answer	Header	Question	Overall Precision	Overall Recall	Overall F1	Overall Accuracy
1.7597	1.0	10	1.5908	{'precision': 0.028469750889679714, 'recall': 0.029666254635352288, 'f1': 0.029055690072639227, 'number': 809}	{'precision': 0.0, 'recall': 0.0, 'f1': 0.0, 'number': 119}	{'precision': 0.2727272727272727, 'recall': 0.2507042253521127, 'f1': 0.261252446183953, 'number': 1065}	0.1597	0.1460	0.1526	0.3668
1.4236	2.0	20	1.2445	{'precision': 0.1969872537659328, 'recall': 0.21013597033374537, 'f1': 0.2033492822966507, 'number': 809}	{'precision': 0.0, 'recall': 0.0, 'f1': 0.0, 'number': 119}	{'precision': 0.446, 'recall': 0.6281690140845071, 'f1': 0.5216374269005849, 'number': 1065}	0.3551	0.4210	0.3852	0.5787
1.0716	3.0	30	0.9339	{'precision': 0.48671726755218214, 'recall': 0.6341161928306551, 'f1': 0.5507246376811593, 'number': 809}	{'precision': 0.0, 'recall': 0.0, 'f1': 0.0, 'number': 119}	{'precision': 0.5635276532137519, 'recall': 0.707981220657277, 'f1': 0.6275488972118186, 'number': 1065}	0.5249	0.6357	0.5750	0.7085
0.8242	4.0	40	0.7904	{'precision': 0.5719591457753017, 'recall': 0.761433868974042, 'f1': 0.6532343584305408, 'number': 809}	{'precision': 0.08695652173913043, 'recall': 0.05042016806722689, 'f1': 0.06382978723404256, 'number': 119}	{'precision': 0.6635672020287405, 'recall': 0.7370892018779343, 'f1': 0.6983985765124555, 'number': 1065}	0.6041	0.7060	0.6511	0.7541
0.6816	5.0	50	0.7204	{'precision': 0.643956043956044, 'recall': 0.7243510506798516, 'f1': 0.6817917393833623, 'number': 809}	{'precision': 0.19148936170212766, 'recall': 0.15126050420168066, 'f1': 0.16901408450704225, 'number': 119}	{'precision': 0.6732751784298177, 'recall': 0.7971830985915493, 'f1': 0.7300085984522787, 'number': 1065}	0.6415	0.7291	0.6825	0.7767
0.5706	6.0	60	0.7060	{'precision': 0.6353754940711462, 'recall': 0.7948084054388134, 'f1': 0.7062053816584294, 'number': 809}	{'precision': 0.1919191919191919, 'recall': 0.15966386554621848, 'f1': 0.17431192660550457, 'number': 119}	{'precision': 0.7103139013452915, 'recall': 0.7436619718309859, 'f1': 0.726605504587156, 'number': 1065}	0.6532	0.7296	0.6893	0.7740
0.5032	7.0	70	0.6764	{'precision': 0.6708994708994709, 'recall': 0.7836835599505563, 'f1': 0.7229190421892816, 'number': 809}	{'precision': 0.232, 'recall': 0.24369747899159663, 'f1': 0.2377049180327869, 'number': 119}	{'precision': 0.7590149516270889, 'recall': 0.8103286384976526, 'f1': 0.7838328792007266, 'number': 1065}	0.6914	0.7657	0.7267	0.7902
0.4396	8.0	80	0.6664	{'precision': 0.6763129689174705, 'recall': 0.7799752781211372, 'f1': 0.7244546498277842, 'number': 809}	{'precision': 0.25833333333333336, 'recall': 0.2605042016806723, 'f1': 0.2594142259414226, 'number': 119}	{'precision': 0.7695035460992907, 'recall': 0.8150234741784037, 'f1': 0.791609667122663, 'number': 1065}	0.7015	0.7677	0.7331	0.7919
0.3889	9.0	90	0.6626	{'precision': 0.7093541202672605, 'recall': 0.7873918417799752, 'f1': 0.7463386057410661, 'number': 809}	{'precision': 0.2857142857142857, 'recall': 0.2857142857142857, 'f1': 0.2857142857142857, 'number': 119}	{'precision': 0.7683566433566433, 'recall': 0.8253521126760563, 'f1': 0.7958352195563604, 'number': 1065}	0.7173	0.7777	0.7463	0.7966
0.3711	10.0	100	0.6630	{'precision': 0.702433628318584, 'recall': 0.7849196538936959, 'f1': 0.7413893753648569, 'number': 809}	{'precision': 0.3064516129032258, 'recall': 0.31932773109243695, 'f1': 0.31275720164609055, 'number': 119}	{'precision': 0.7749110320284698, 'recall': 0.8178403755868544, 'f1': 0.7957971676564642, 'number': 1065}	0.7175	0.7747	0.7450	0.7996
0.3207	11.0	110	0.6584	{'precision': 0.6994594594594594, 'recall': 0.799752781211372, 'f1': 0.7462514417531719, 'number': 809}	{'precision': 0.3235294117647059, 'recall': 0.3697478991596639, 'f1': 0.3450980392156863, 'number': 119}	{'precision': 0.7726075504828798, 'recall': 0.8262910798122066, 'f1': 0.7985480943738658, 'number': 1065}	0.7141	0.7883	0.7493	0.8011
0.3129	12.0	120	0.6771	{'precision': 0.7112359550561798, 'recall': 0.7824474660074165, 'f1': 0.7451442024720423, 'number': 809}	{'precision': 0.3308270676691729, 'recall': 0.3697478991596639, 'f1': 0.3492063492063492, 'number': 119}	{'precision': 0.7837354781054513, 'recall': 0.8234741784037559, 'f1': 0.8031135531135531, 'number': 1065}	0.7255	0.7797	0.7516	0.7980
0.2863	13.0	130	0.6822	{'precision': 0.7075575027382256, 'recall': 0.7985166872682324, 'f1': 0.7502903600464577, 'number': 809}	{'precision': 0.3359375, 'recall': 0.36134453781512604, 'f1': 0.3481781376518218, 'number': 119}	{'precision': 0.7793594306049823, 'recall': 0.8225352112676056, 'f1': 0.8003654636820466, 'number': 1065}	0.7229	0.7852	0.7528	0.7993
0.2682	14.0	140	0.6865	{'precision': 0.709070796460177, 'recall': 0.792336217552534, 'f1': 0.7483946293053124, 'number': 809}	{'precision': 0.3435114503816794, 'recall': 0.37815126050420167, 'f1': 0.36, 'number': 119}	{'precision': 0.7726075504828798, 'recall': 0.8262910798122066, 'f1': 0.7985480943738658, 'number': 1065}	0.7203	0.7858	0.7516	0.7976
0.2665	15.0	150	0.6877	{'precision': 0.7073707370737073, 'recall': 0.7948084054388134, 'f1': 0.7485448195576251, 'number': 809}	{'precision': 0.3435114503816794, 'recall': 0.37815126050420167, 'f1': 0.36, 'number': 119}	{'precision': 0.7785651018600531, 'recall': 0.8253521126760563, 'f1': 0.8012762078395624, 'number': 1065}	0.7225	0.7863	0.7530	0.7977

Framework versions

• Transformers 4.45.1
• Pytorch 2.4.0+cpu
• Datasets 3.0.1
• Tokenizers 0.20.0