layoutlm-funsd

This model is a fine-tuned version of microsoft/layoutlm-base-uncased on the funsd dataset. It achieves the following results on the evaluation set:

• Loss: 0.6978
• Answer: {'precision': 0.7016216216216217, 'recall': 0.8022249690976514, 'f1': 0.748558246828143, 'number': 809}
• Header: {'precision': 0.30327868852459017, 'recall': 0.31092436974789917, 'f1': 0.3070539419087137, 'number': 119}
• Question: {'precision': 0.7697022767075307, 'recall': 0.8253521126760563, 'f1': 0.7965564114182148, 'number': 1065}
• Overall Precision: 0.7149
• Overall Recall: 0.7852
• Overall F1: 0.7484
• Overall Accuracy: 0.7991

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 3e-05
• train_batch_size: 16
• eval_batch_size: 8
• seed: 42
• optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
• lr_scheduler_type: linear
• num_epochs: 15
• mixed_precision_training: Native AMP

Training results

Training Loss	Epoch	Step	Validation Loss	Answer	Header	Question	Overall Precision	Overall Recall	Overall F1	Overall Accuracy
1.8306	1.0	10	1.6241	{'precision': 0.015978695073235686, 'recall': 0.014833127317676144, 'f1': 0.015384615384615385, 'number': 809}	{'precision': 0.0, 'recall': 0.0, 'f1': 0.0, 'number': 119}	{'precision': 0.18994413407821228, 'recall': 0.12769953051643193, 'f1': 0.1527231892195396, 'number': 1065}	0.1009	0.0743	0.0855	0.3533
1.4993	2.0	20	1.2681	{'precision': 0.11957950065703023, 'recall': 0.11248454882571075, 'f1': 0.11592356687898088, 'number': 809}	{'precision': 0.0, 'recall': 0.0, 'f1': 0.0, 'number': 119}	{'precision': 0.43696829079659705, 'recall': 0.5305164319248826, 'f1': 0.4792196776929601, 'number': 1065}	0.3192	0.3292	0.3241	0.5668
1.1217	3.0	30	0.9584	{'precision': 0.4690157958687728, 'recall': 0.47713226205191595, 'f1': 0.47303921568627455, 'number': 809}	{'precision': 0.08333333333333333, 'recall': 0.025210084033613446, 'f1': 0.038709677419354833, 'number': 119}	{'precision': 0.6071741032370953, 'recall': 0.6516431924882629, 'f1': 0.6286231884057971, 'number': 1065}	0.5410	0.5434	0.5422	0.7000
0.8403	4.0	40	0.7788	{'precision': 0.6163265306122448, 'recall': 0.7466007416563659, 'f1': 0.6752375628842929, 'number': 809}	{'precision': 0.2127659574468085, 'recall': 0.08403361344537816, 'f1': 0.12048192771084337, 'number': 119}	{'precision': 0.6675603217158177, 'recall': 0.7014084507042253, 'f1': 0.6840659340659341, 'number': 1065}	0.6342	0.6829	0.6576	0.7495
0.6807	5.0	50	0.7110	{'precision': 0.6525871172122492, 'recall': 0.7639060568603214, 'f1': 0.7038724373576309, 'number': 809}	{'precision': 0.26865671641791045, 'recall': 0.15126050420168066, 'f1': 0.19354838709677416, 'number': 119}	{'precision': 0.7077059344552702, 'recall': 0.7502347417840376, 'f1': 0.7283500455788514, 'number': 1065}	0.6696	0.7200	0.6939	0.7799
0.5615	6.0	60	0.6839	{'precision': 0.6663135593220338, 'recall': 0.7775030902348579, 'f1': 0.7176269252709641, 'number': 809}	{'precision': 0.3225806451612903, 'recall': 0.16806722689075632, 'f1': 0.22099447513812157, 'number': 119}	{'precision': 0.7101214574898785, 'recall': 0.8234741784037559, 'f1': 0.7626086956521739, 'number': 1065}	0.6809	0.7657	0.7208	0.7886
0.4954	7.0	70	0.6647	{'precision': 0.6813304721030042, 'recall': 0.7849196538936959, 'f1': 0.7294658242389431, 'number': 809}	{'precision': 0.28865979381443296, 'recall': 0.23529411764705882, 'f1': 0.2592592592592593, 'number': 119}	{'precision': 0.7263681592039801, 'recall': 0.8225352112676056, 'f1': 0.7714663143989432, 'number': 1065}	0.6886	0.7722	0.7280	0.7957
0.4479	8.0	80	0.6529	{'precision': 0.6748663101604279, 'recall': 0.7799752781211372, 'f1': 0.7236238532110092, 'number': 809}	{'precision': 0.25742574257425743, 'recall': 0.2184873949579832, 'f1': 0.23636363636363636, 'number': 119}	{'precision': 0.740016992353441, 'recall': 0.8178403755868544, 'f1': 0.7769848349687779, 'number': 1065}	0.6905	0.7667	0.7266	0.8053
0.3961	9.0	90	0.6535	{'precision': 0.6924754634678298, 'recall': 0.7849196538936959, 'f1': 0.7358053302433372, 'number': 809}	{'precision': 0.25217391304347825, 'recall': 0.24369747899159663, 'f1': 0.24786324786324784, 'number': 119}	{'precision': 0.7459505541346974, 'recall': 0.8215962441314554, 'f1': 0.7819481680071492, 'number': 1065}	0.6980	0.7722	0.7332	0.8020
0.3516	10.0	100	0.6645	{'precision': 0.6899141630901288, 'recall': 0.7948084054388134, 'f1': 0.7386559448592762, 'number': 809}	{'precision': 0.2857142857142857, 'recall': 0.25210084033613445, 'f1': 0.26785714285714285, 'number': 119}	{'precision': 0.7667814113597247, 'recall': 0.8366197183098592, 'f1': 0.8001796138302649, 'number': 1065}	0.7112	0.7847	0.7462	0.8046
0.3197	11.0	110	0.6868	{'precision': 0.6927194860813705, 'recall': 0.799752781211372, 'f1': 0.7423981640849111, 'number': 809}	{'precision': 0.2966101694915254, 'recall': 0.29411764705882354, 'f1': 0.2953586497890296, 'number': 119}	{'precision': 0.7722513089005235, 'recall': 0.8309859154929577, 'f1': 0.8005427408412483, 'number': 1065}	0.7129	0.7863	0.7478	0.7996
0.2986	12.0	120	0.6912	{'precision': 0.6914778856526429, 'recall': 0.792336217552534, 'f1': 0.7384792626728109, 'number': 809}	{'precision': 0.3162393162393162, 'recall': 0.31092436974789917, 'f1': 0.3135593220338983, 'number': 119}	{'precision': 0.7845057880676759, 'recall': 0.8272300469483568, 'f1': 0.8053016453382084, 'number': 1065}	0.7194	0.7822	0.7495	0.7971
0.2882	13.0	130	0.7016	{'precision': 0.6961748633879782, 'recall': 0.7873918417799752, 'f1': 0.7389791183294663, 'number': 809}	{'precision': 0.3076923076923077, 'recall': 0.3025210084033613, 'f1': 0.30508474576271183, 'number': 119}	{'precision': 0.7696969696969697, 'recall': 0.8347417840375587, 'f1': 0.8009009009009008, 'number': 1065}	0.7142	0.7837	0.7474	0.8028
0.2789	14.0	140	0.6994	{'precision': 0.6989247311827957, 'recall': 0.8034610630407911, 'f1': 0.747556066705003, 'number': 809}	{'precision': 0.30327868852459017, 'recall': 0.31092436974789917, 'f1': 0.3070539419087137, 'number': 119}	{'precision': 0.7698343504795118, 'recall': 0.8291079812206573, 'f1': 0.7983725135623869, 'number': 1065}	0.7140	0.7878	0.7490	0.7990
0.274	15.0	150	0.6978	{'precision': 0.7016216216216217, 'recall': 0.8022249690976514, 'f1': 0.748558246828143, 'number': 809}	{'precision': 0.30327868852459017, 'recall': 0.31092436974789917, 'f1': 0.3070539419087137, 'number': 119}	{'precision': 0.7697022767075307, 'recall': 0.8253521126760563, 'f1': 0.7965564114182148, 'number': 1065}	0.7149	0.7852	0.7484	0.7991

Framework versions

• Transformers 4.28.1
• Pytorch 2.0.0+cu118
• Datasets 2.11.0
• Tokenizers 0.13.3