layoutlm-funsd

This model is a fine-tuned version of microsoft/layoutlm-base-uncased on the funsd dataset. It achieves the following results on the evaluation set:

• Loss: 0.7034
• Answer: {'precision': 0.6974697469746974, 'recall': 0.7836835599505563, 'f1': 0.7380675203725262, 'number': 809}
• Header: {'precision': 0.3106060606060606, 'recall': 0.3445378151260504, 'f1': 0.32669322709163345, 'number': 119}
• Question: {'precision': 0.7759226713532513, 'recall': 0.8291079812206573, 'f1': 0.8016341352700863, 'number': 1065}
• Overall Precision: 0.7150
• Overall Recall: 0.7817
• Overall F1: 0.7469
• Overall Accuracy: 0.8170

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 3e-05
• train_batch_size: 16
• eval_batch_size: 8
• seed: 42
• optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
• lr_scheduler_type: linear
• num_epochs: 15

Training results

Training Loss	Epoch	Step	Validation Loss	Answer	Header	Question	Overall Precision	Overall Recall	Overall F1	Overall Accuracy
1.817	1.0	10	1.6092	{'precision': 0.002197802197802198, 'recall': 0.0012360939431396785, 'f1': 0.0015822784810126582, 'number': 809}	{'precision': 0.0, 'recall': 0.0, 'f1': 0.0, 'number': 119}	{'precision': 0.22, 'recall': 0.07230046948356808, 'f1': 0.10883392226148411, 'number': 1065}	0.0968	0.0391	0.0557	0.3198
1.4741	2.0	20	1.2390	{'precision': 0.22916666666666666, 'recall': 0.24474660074165636, 'f1': 0.23670053795576806, 'number': 809}	{'precision': 0.0, 'recall': 0.0, 'f1': 0.0, 'number': 119}	{'precision': 0.4344262295081967, 'recall': 0.5474178403755868, 'f1': 0.4844204403822185, 'number': 1065}	0.3540	0.3919	0.3720	0.6109
1.1045	3.0	30	0.9397	{'precision': 0.493006993006993, 'recall': 0.522867737948084, 'f1': 0.5074985002999399, 'number': 809}	{'precision': 0.0, 'recall': 0.0, 'f1': 0.0, 'number': 119}	{'precision': 0.5923515052888527, 'recall': 0.6835680751173709, 'f1': 0.6346992153443767, 'number': 1065}	0.5473	0.5775	0.5620	0.7110
0.8369	4.0	40	0.7900	{'precision': 0.6087408949011447, 'recall': 0.723114956736712, 'f1': 0.6610169491525424, 'number': 809}	{'precision': 0.0, 'recall': 0.0, 'f1': 0.0, 'number': 119}	{'precision': 0.6610025488530161, 'recall': 0.7305164319248826, 'f1': 0.6940231935771632, 'number': 1065}	0.6229	0.6839	0.6520	0.7567
0.6855	5.0	50	0.7185	{'precision': 0.6293706293706294, 'recall': 0.7787391841779975, 'f1': 0.696132596685083, 'number': 809}	{'precision': 0.1, 'recall': 0.06722689075630252, 'f1': 0.08040201005025126, 'number': 119}	{'precision': 0.6999125109361329, 'recall': 0.7511737089201878, 'f1': 0.7246376811594203, 'number': 1065}	0.6466	0.7215	0.6820	0.7805
0.5663	6.0	60	0.6795	{'precision': 0.6485655737704918, 'recall': 0.7824474660074165, 'f1': 0.7092436974789915, 'number': 809}	{'precision': 0.1728395061728395, 'recall': 0.11764705882352941, 'f1': 0.13999999999999999, 'number': 119}	{'precision': 0.6875502008032128, 'recall': 0.8037558685446009, 'f1': 0.7411255411255411, 'number': 1065}	0.6529	0.7541	0.6999	0.7940
0.4995	7.0	70	0.6814	{'precision': 0.6630552546045504, 'recall': 0.7564894932014833, 'f1': 0.7066974595842955, 'number': 809}	{'precision': 0.21929824561403508, 'recall': 0.21008403361344538, 'f1': 0.2145922746781116, 'number': 119}	{'precision': 0.721465076660988, 'recall': 0.7953051643192488, 'f1': 0.7565877623939258, 'number': 1065}	0.6712	0.7446	0.7060	0.7994
0.454	8.0	80	0.6688	{'precision': 0.6716738197424893, 'recall': 0.7737948084054388, 'f1': 0.7191269385410684, 'number': 809}	{'precision': 0.24324324324324326, 'recall': 0.226890756302521, 'f1': 0.23478260869565218, 'number': 119}	{'precision': 0.7363481228668942, 'recall': 0.8103286384976526, 'f1': 0.7715690657130085, 'number': 1065}	0.6844	0.7607	0.7205	0.8080
0.4132	9.0	90	0.6665	{'precision': 0.6782231852654388, 'recall': 0.7737948084054388, 'f1': 0.7228637413394918, 'number': 809}	{'precision': 0.29508196721311475, 'recall': 0.3025210084033613, 'f1': 0.2987551867219917, 'number': 119}	{'precision': 0.739460370994941, 'recall': 0.8234741784037559, 'f1': 0.7792092403376277, 'number': 1065}	0.6898	0.7722	0.7287	0.8095
0.3671	10.0	100	0.6719	{'precision': 0.6879049676025918, 'recall': 0.7873918417799752, 'f1': 0.7342939481268012, 'number': 809}	{'precision': 0.319672131147541, 'recall': 0.3277310924369748, 'f1': 0.32365145228215775, 'number': 119}	{'precision': 0.7710526315789473, 'recall': 0.8253521126760563, 'f1': 0.7972789115646259, 'number': 1065}	0.7107	0.7802	0.7438	0.8165
0.334	11.0	110	0.6857	{'precision': 0.6862955032119914, 'recall': 0.792336217552534, 'f1': 0.7355134825014343, 'number': 809}	{'precision': 0.3584905660377358, 'recall': 0.31932773109243695, 'f1': 0.3377777777777778, 'number': 119}	{'precision': 0.7785778577857786, 'recall': 0.812206572769953, 'f1': 0.7950367647058822, 'number': 1065}	0.7178	0.7747	0.7452	0.8160
0.3234	12.0	120	0.6966	{'precision': 0.6926454445664105, 'recall': 0.7799752781211372, 'f1': 0.7337209302325581, 'number': 809}	{'precision': 0.3125, 'recall': 0.33613445378151263, 'f1': 0.3238866396761134, 'number': 119}	{'precision': 0.7707786526684165, 'recall': 0.8272300469483568, 'f1': 0.7980072463768116, 'number': 1065}	0.7113	0.7787	0.7435	0.8154
0.3019	13.0	130	0.6940	{'precision': 0.7010869565217391, 'recall': 0.7972805933250927, 'f1': 0.746096009253904, 'number': 809}	{'precision': 0.3252032520325203, 'recall': 0.33613445378151263, 'f1': 0.3305785123966942, 'number': 119}	{'precision': 0.7724444444444445, 'recall': 0.815962441314554, 'f1': 0.7936073059360731, 'number': 1065}	0.7168	0.7797	0.7469	0.8166
0.2888	14.0	140	0.7011	{'precision': 0.6946564885496184, 'recall': 0.7873918417799752, 'f1': 0.7381228273464657, 'number': 809}	{'precision': 0.29927007299270075, 'recall': 0.3445378151260504, 'f1': 0.3203125, 'number': 119}	{'precision': 0.7726872246696035, 'recall': 0.8234741784037559, 'f1': 0.7972727272727272, 'number': 1065}	0.7104	0.7802	0.7437	0.8170
0.2838	15.0	150	0.7034	{'precision': 0.6974697469746974, 'recall': 0.7836835599505563, 'f1': 0.7380675203725262, 'number': 809}	{'precision': 0.3106060606060606, 'recall': 0.3445378151260504, 'f1': 0.32669322709163345, 'number': 119}	{'precision': 0.7759226713532513, 'recall': 0.8291079812206573, 'f1': 0.8016341352700863, 'number': 1065}	0.7150	0.7817	0.7469	0.8170

Framework versions

• Transformers 4.27.4
• Pytorch 1.13.1
• Datasets 2.11.0
• Tokenizers 0.13.3