End of training

README.md

@@ -0,0 +1,79 @@
+---
+base_model: microsoft/layoutlm-base-uncased
+tags:
+- generated_from_trainer
+datasets:
+- funsd
+model-index:
+- name: layoutlm-funsd
+  results: []
+---
+
+<!-- This model card has been generated automatically according to the information the Trainer had access to. You
+should probably proofread and complete it, then remove this comment. -->
+
+# layoutlm-funsd
+
+This model is a fine-tuned version of [microsoft/layoutlm-base-uncased](https://huggingface.co/microsoft/layoutlm-base-uncased) on the funsd dataset.
+It achieves the following results on the evaluation set:
+- Loss: 0.7080
+- Answer: {'precision': 0.7122381477398015, 'recall': 0.7985166872682324, 'f1': 0.752913752913753, 'number': 809}
+- Header: {'precision': 0.3359375, 'recall': 0.36134453781512604, 'f1': 0.3481781376518218, 'number': 119}
+- Question: {'precision': 0.7817531305903399, 'recall': 0.8206572769953052, 'f1': 0.8007329363261567, 'number': 1065}
+- Overall Precision: 0.7260
+- Overall Recall: 0.7842
+- Overall F1: 0.7540
+- Overall Accuracy: 0.8073
+
+## Model description
+
+More information needed
+
+## Intended uses & limitations
+
+More information needed
+
+## Training and evaluation data
+
+More information needed
+
+## Training procedure
+
+### Training hyperparameters
+
+The following hyperparameters were used during training:
+- learning_rate: 3e-05
+- train_batch_size: 16
+- eval_batch_size: 8
+- seed: 42
+- optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
+- lr_scheduler_type: linear
+- num_epochs: 15
+
+### Training results
+
+| Training Loss | Epoch | Step | Validation Loss | Answer                                                                                                    | Header                                                                                                        | Question                                                                                                   | Overall Precision | Overall Recall | Overall F1 | Overall Accuracy |
+|:-------------:|:-----:|:----:|:---------------:|:---------------------------------------------------------------------------------------------------------:|:-------------------------------------------------------------------------------------------------------------:|:----------------------------------------------------------------------------------------------------------:|:-----------------:|:--------------:|:----------:|:----------------:|
+| 1.4164        | 1.0   | 10   | 1.1867          | {'precision': 0.21566110397946084, 'recall': 0.207663782447466, 'f1': 0.21158690176322417, 'number': 809} | {'precision': 0.0, 'recall': 0.0, 'f1': 0.0, 'number': 119}                                                   | {'precision': 0.48124557678697805, 'recall': 0.6384976525821596, 'f1': 0.5488297013720743, 'number': 1065} | 0.3869            | 0.4255         | 0.4053     | 0.6139           |
+| 1.0235        | 2.0   | 20   | 0.8815          | {'precision': 0.578494623655914, 'recall': 0.6650185414091471, 'f1': 0.6187464059804485, 'number': 809}   | {'precision': 0.05555555555555555, 'recall': 0.008403361344537815, 'f1': 0.014598540145985401, 'number': 119} | {'precision': 0.6398687448728466, 'recall': 0.7323943661971831, 'f1': 0.6830122591943958, 'number': 1065}  | 0.6087            | 0.6618         | 0.6341     | 0.7403           |
+| 0.7822        | 3.0   | 30   | 0.7564          | {'precision': 0.6335403726708074, 'recall': 0.7564894932014833, 'f1': 0.6895774647887324, 'number': 809}  | {'precision': 0.13559322033898305, 'recall': 0.06722689075630252, 'f1': 0.0898876404494382, 'number': 119}    | {'precision': 0.6905158069883528, 'recall': 0.7793427230046949, 'f1': 0.7322452580502868, 'number': 1065}  | 0.6511            | 0.7275         | 0.6872     | 0.7697           |
+| 0.6495        | 4.0   | 40   | 0.6955          | {'precision': 0.6533333333333333, 'recall': 0.7873918417799752, 'f1': 0.7141255605381165, 'number': 809}  | {'precision': 0.19480519480519481, 'recall': 0.12605042016806722, 'f1': 0.15306122448979592, 'number': 119}   | {'precision': 0.7162276975361087, 'recall': 0.7915492957746478, 'f1': 0.752007136485281, 'number': 1065}   | 0.6707            | 0.7501         | 0.7082     | 0.7915           |
+| 0.5641        | 5.0   | 50   | 0.6796          | {'precision': 0.6843267108167771, 'recall': 0.7663782447466008, 'f1': 0.7230320699708457, 'number': 809}  | {'precision': 0.275, 'recall': 0.18487394957983194, 'f1': 0.22110552763819097, 'number': 119}                 | {'precision': 0.7565217391304347, 'recall': 0.8169014084507042, 'f1': 0.7855530474040633, 'number': 1065}  | 0.7079            | 0.7587         | 0.7324     | 0.7899           |
+| 0.4862        | 6.0   | 60   | 0.6563          | {'precision': 0.6844978165938864, 'recall': 0.7750309023485785, 'f1': 0.7269565217391305, 'number': 809}  | {'precision': 0.28, 'recall': 0.23529411764705882, 'f1': 0.2557077625570776, 'number': 119}                   | {'precision': 0.7420168067226891, 'recall': 0.8291079812206573, 'f1': 0.7831485587583149, 'number': 1065}  | 0.6972            | 0.7717         | 0.7326     | 0.8007           |
+| 0.4389        | 7.0   | 70   | 0.6444          | {'precision': 0.6868365180467091, 'recall': 0.799752781211372, 'f1': 0.7390062821245003, 'number': 809}   | {'precision': 0.28703703703703703, 'recall': 0.2605042016806723, 'f1': 0.27312775330396477, 'number': 119}    | {'precision': 0.7411167512690355, 'recall': 0.8225352112676056, 'f1': 0.7797062750333779, 'number': 1065}  | 0.6962            | 0.7797         | 0.7356     | 0.8040           |
+| 0.3912        | 8.0   | 80   | 0.6505          | {'precision': 0.7074527252502781, 'recall': 0.7861557478368356, 'f1': 0.7447306791569087, 'number': 809}  | {'precision': 0.3392857142857143, 'recall': 0.31932773109243695, 'f1': 0.32900432900432897, 'number': 119}    | {'precision': 0.7689594356261023, 'recall': 0.8187793427230047, 'f1': 0.793087767166894, 'number': 1065}   | 0.7207            | 0.7757         | 0.7472     | 0.8073           |
+| 0.3511        | 9.0   | 90   | 0.6696          | {'precision': 0.7147577092511013, 'recall': 0.8022249690976514, 'f1': 0.7559697146185206, 'number': 809}  | {'precision': 0.296, 'recall': 0.31092436974789917, 'f1': 0.30327868852459017, 'number': 119}                 | {'precision': 0.7589833479404031, 'recall': 0.8131455399061033, 'f1': 0.7851314596554851, 'number': 1065}  | 0.7139            | 0.7787         | 0.7449     | 0.8042           |
+| 0.3166        | 10.0  | 100  | 0.6746          | {'precision': 0.7190265486725663, 'recall': 0.8034610630407911, 'f1': 0.7589025102159953, 'number': 809}  | {'precision': 0.35398230088495575, 'recall': 0.33613445378151263, 'f1': 0.3448275862068966, 'number': 119}    | {'precision': 0.7753108348134992, 'recall': 0.819718309859155, 'f1': 0.7968963943404839, 'number': 1065}   | 0.7294            | 0.7842         | 0.7558     | 0.8081           |
+| 0.2925        | 11.0  | 110  | 0.6839          | {'precision': 0.7160356347438753, 'recall': 0.7948084054388134, 'f1': 0.753368482718219, 'number': 809}   | {'precision': 0.3208955223880597, 'recall': 0.36134453781512604, 'f1': 0.33992094861660077, 'number': 119}    | {'precision': 0.7803780378037803, 'recall': 0.8140845070422535, 'f1': 0.796875, 'number': 1065}            | 0.7247            | 0.7792         | 0.7510     | 0.8087           |
+| 0.2837        | 12.0  | 120  | 0.6853          | {'precision': 0.7161862527716186, 'recall': 0.7985166872682324, 'f1': 0.7551139684395091, 'number': 809}  | {'precision': 0.3333333333333333, 'recall': 0.3445378151260504, 'f1': 0.33884297520661155, 'number': 119}     | {'precision': 0.7751322751322751, 'recall': 0.8253521126760563, 'f1': 0.7994542974079127, 'number': 1065}  | 0.7253            | 0.7858         | 0.7543     | 0.8064           |
+| 0.265         | 13.0  | 130  | 0.7016          | {'precision': 0.7069154774972558, 'recall': 0.796044499381953, 'f1': 0.7488372093023256, 'number': 809}   | {'precision': 0.31654676258992803, 'recall': 0.3697478991596639, 'f1': 0.3410852713178294, 'number': 119}     | {'precision': 0.7867513611615246, 'recall': 0.8140845070422535, 'f1': 0.8001845869866173, 'number': 1065}  | 0.7226            | 0.7802         | 0.7503     | 0.8076           |
+| 0.2475        | 14.0  | 140  | 0.7055          | {'precision': 0.7084708470847084, 'recall': 0.796044499381953, 'f1': 0.749708963911525, 'number': 809}    | {'precision': 0.32575757575757575, 'recall': 0.36134453781512604, 'f1': 0.3426294820717131, 'number': 119}    | {'precision': 0.771806167400881, 'recall': 0.8225352112676056, 'f1': 0.7963636363636363, 'number': 1065}   | 0.7183            | 0.7842         | 0.7498     | 0.8054           |
+| 0.2423        | 15.0  | 150  | 0.7080          | {'precision': 0.7122381477398015, 'recall': 0.7985166872682324, 'f1': 0.752913752913753, 'number': 809}   | {'precision': 0.3359375, 'recall': 0.36134453781512604, 'f1': 0.3481781376518218, 'number': 119}              | {'precision': 0.7817531305903399, 'recall': 0.8206572769953052, 'f1': 0.8007329363261567, 'number': 1065}  | 0.7260            | 0.7842         | 0.7540     | 0.8073           |
+
+
+### Framework versions
+
+- Transformers 4.34.1
+- Pytorch 2.1.0+cu118
+- Datasets 2.14.6
+- Tokenizers 0.14.1

preprocessor_config.json

@@ -0,0 +1,14 @@
+{
+  "apply_ocr": true,
+  "do_resize": true,
+  "feature_extractor_type": "LayoutLMv2FeatureExtractor",
+  "image_processor_type": "LayoutLMv2ImageProcessor",
+  "ocr_lang": null,
+  "processor_class": "LayoutLMv2Processor",
+  "resample": 2,
+  "size": {
+    "height": 224,
+    "width": 224
+  },
+  "tesseract_config": ""
+}

pytorch_model.bin

@@ -1,3 +1,3 @@
 version https://git-lfs.github.com/spec/v1
-oid sha256:4663fe5b140a097699ac7e52e5a71814038e3905c45b0fb0e564abc5f25dc6de
+oid sha256:fb511165d82e51218ca1d884dbb3daa5db074d330b5f337e44dfd44176fd2519
 size 450604414

special_tokens_map.json

@@ -0,0 +1,37 @@
+{
+  "cls_token": {
+    "content": "[CLS]",
+    "lstrip": false,
+    "normalized": false,
+    "rstrip": false,
+    "single_word": false
+  },
+  "mask_token": {
+    "content": "[MASK]",
+    "lstrip": false,
+    "normalized": false,
+    "rstrip": false,
+    "single_word": false
+  },
+  "pad_token": {
+    "content": "[PAD]",
+    "lstrip": false,
+    "normalized": false,
+    "rstrip": false,
+    "single_word": false
+  },
+  "sep_token": {
+    "content": "[SEP]",
+    "lstrip": false,
+    "normalized": false,
+    "rstrip": false,
+    "single_word": false
+  },
+  "unk_token": {
+    "content": "[UNK]",
+    "lstrip": false,
+    "normalized": false,
+    "rstrip": false,
+    "single_word": false
+  }
+}

tokenizer_config.json

@@ -0,0 +1,80 @@
+{
+  "added_tokens_decoder": {
+    "0": {
+      "content": "[PAD]",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "100": {
+      "content": "[UNK]",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "101": {
+      "content": "[CLS]",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "102": {
+      "content": "[SEP]",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "103": {
+      "content": "[MASK]",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    }
+  },
+  "additional_special_tokens": [],
+  "apply_ocr": false,
+  "clean_up_tokenization_spaces": true,
+  "cls_token": "[CLS]",
+  "cls_token_box": [
+    0,
+    0,
+    0,
+    0
+  ],
+  "do_basic_tokenize": true,
+  "do_lower_case": true,
+  "mask_token": "[MASK]",
+  "model_max_length": 512,
+  "never_split": null,
+  "only_label_first_subword": true,
+  "pad_token": "[PAD]",
+  "pad_token_box": [
+    0,
+    0,
+    0,
+    0
+  ],
+  "pad_token_label": -100,
+  "processor_class": "LayoutLMv2Processor",
+  "sep_token": "[SEP]",
+  "sep_token_box": [
+    1000,
+    1000,
+    1000,
+    1000
+  ],
+  "strip_accents": null,
+  "tokenize_chinese_chars": true,
+  "tokenizer_class": "LayoutLMv2Tokenizer",
+  "unk_token": "[UNK]"
+}