End of training

README.md

@@ -0,0 +1,137 @@
+---
+library_name: transformers
+license: agpl-3.0
+base_model: vinai/phobert-base-v2
+tags:
+- generated_from_trainer
+metrics:
+- accuracy
+- f1
+model-index:
+- name: PhoBert_Lexical_15-10
+  results: []
+---
+
+<!-- This model card has been generated automatically according to the information the Trainer had access to. You
+should probably proofread and complete it, then remove this comment. -->
+
+# PhoBert_Lexical_15-10
+
+This model is a fine-tuned version of [vinai/phobert-base-v2](https://huggingface.co/vinai/phobert-base-v2) on an unknown dataset.
+It achieves the following results on the evaluation set:
+- Loss: 0.0211
+- Accuracy: 0.9945
+- F1: 0.9945
+
+## Model description
+
+More information needed
+
+## Intended uses & limitations
+
+More information needed
+
+## Training and evaluation data
+
+More information needed
+
+## Training procedure
+
+### Training hyperparameters
+
+The following hyperparameters were used during training:
+- learning_rate: 2e-05
+- train_batch_size: 64
+- eval_batch_size: 64
+- seed: 42
+- optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
+- lr_scheduler_type: linear
+- num_epochs: 15
+
+### Training results
+
+| Training Loss | Epoch   | Step  | Validation Loss | Accuracy | F1     |
+|:-------------:|:-------:|:-----:|:---------------:|:--------:|:------:|
+| No log        | 0.1996  | 200   | 0.3681          | 0.8393   | 0.8397 |
+| No log        | 0.3992  | 400   | 0.3333          | 0.8558   | 0.8563 |
+| No log        | 0.5988  | 600   | 0.3128          | 0.8681   | 0.8684 |
+| No log        | 0.7984  | 800   | 0.3062          | 0.8698   | 0.8703 |
+| No log        | 0.9980  | 1000  | 0.2766          | 0.8822   | 0.8827 |
+| 0.3857        | 1.1976  | 1200  | 0.2619          | 0.8896   | 0.8900 |
+| 0.3857        | 1.3972  | 1400  | 0.2524          | 0.8939   | 0.8943 |
+| 0.3857        | 1.5968  | 1600  | 0.2386          | 0.8997   | 0.9000 |
+| 0.3857        | 1.7964  | 1800  | 0.2422          | 0.9013   | 0.9013 |
+| 0.3857        | 1.9960  | 2000  | 0.2229          | 0.9073   | 0.9077 |
+| 0.2899        | 2.1956  | 2200  | 0.2121          | 0.9135   | 0.9137 |
+| 0.2899        | 2.3952  | 2400  | 0.2311          | 0.9028   | 0.9032 |
+| 0.2899        | 2.5948  | 2600  | 0.2070          | 0.9163   | 0.9166 |
+| 0.2899        | 2.7944  | 2800  | 0.1931          | 0.9224   | 0.9226 |
+| 0.2899        | 2.9940  | 3000  | 0.1894          | 0.9236   | 0.9238 |
+| 0.248         | 3.1936  | 3200  | 0.1797          | 0.9285   | 0.9287 |
+| 0.248         | 3.3932  | 3400  | 0.1714          | 0.9330   | 0.9331 |
+| 0.248         | 3.5928  | 3600  | 0.1628          | 0.9369   | 0.9371 |
+| 0.248         | 3.7924  | 3800  | 0.1642          | 0.9353   | 0.9355 |
+| 0.248         | 3.9920  | 4000  | 0.1541          | 0.9403   | 0.9405 |
+| 0.2167        | 4.1916  | 4200  | 0.1430          | 0.9467   | 0.9468 |
+| 0.2167        | 4.3912  | 4400  | 0.1510          | 0.9425   | 0.9427 |
+| 0.2167        | 4.5908  | 4600  | 0.1509          | 0.9433   | 0.9435 |
+| 0.2167        | 4.7904  | 4800  | 0.1323          | 0.9514   | 0.9515 |
+| 0.2167        | 4.9900  | 5000  | 0.1248          | 0.9559   | 0.9559 |
+| 0.1901        | 5.1896  | 5200  | 0.1159          | 0.9577   | 0.9577 |
+| 0.1901        | 5.3892  | 5400  | 0.1123          | 0.9595   | 0.9595 |
+| 0.1901        | 5.5888  | 5600  | 0.1128          | 0.9590   | 0.9591 |
+| 0.1901        | 5.7884  | 5800  | 0.1016          | 0.9636   | 0.9636 |
+| 0.1901        | 5.9880  | 6000  | 0.1080          | 0.9628   | 0.9629 |
+| 0.1659        | 6.1876  | 6200  | 0.0910          | 0.9679   | 0.9679 |
+| 0.1659        | 6.3872  | 6400  | 0.0939          | 0.9674   | 0.9675 |
+| 0.1659        | 6.5868  | 6600  | 0.0948          | 0.9665   | 0.9665 |
+| 0.1659        | 6.7864  | 6800  | 0.0846          | 0.9711   | 0.9711 |
+| 0.1659        | 6.9860  | 7000  | 0.0866          | 0.9691   | 0.9691 |
+| 0.1451        | 7.1856  | 7200  | 0.0778          | 0.9733   | 0.9734 |
+| 0.1451        | 7.3852  | 7400  | 0.0823          | 0.9707   | 0.9708 |
+| 0.1451        | 7.5848  | 7600  | 0.0702          | 0.9771   | 0.9772 |
+| 0.1451        | 7.7844  | 7800  | 0.0684          | 0.9776   | 0.9776 |
+| 0.1451        | 7.9840  | 8000  | 0.0695          | 0.9765   | 0.9766 |
+| 0.128         | 8.1836  | 8200  | 0.0635          | 0.9793   | 0.9793 |
+| 0.128         | 8.3832  | 8400  | 0.0586          | 0.9805   | 0.9805 |
+| 0.128         | 8.5828  | 8600  | 0.0698          | 0.9747   | 0.9748 |
+| 0.128         | 8.7824  | 8800  | 0.0570          | 0.9815   | 0.9815 |
+| 0.128         | 8.9820  | 9000  | 0.0514          | 0.9849   | 0.9849 |
+| 0.1113        | 9.1816  | 9200  | 0.0514          | 0.9836   | 0.9836 |
+| 0.1113        | 9.3812  | 9400  | 0.0470          | 0.9862   | 0.9862 |
+| 0.1113        | 9.5808  | 9600  | 0.0480          | 0.9850   | 0.9850 |
+| 0.1113        | 9.7804  | 9800  | 0.0482          | 0.9849   | 0.9849 |
+| 0.1113        | 9.9800  | 10000 | 0.0423          | 0.9872   | 0.9872 |
+| 0.0969        | 10.1796 | 10200 | 0.0410          | 0.9871   | 0.9871 |
+| 0.0969        | 10.3792 | 10400 | 0.0374          | 0.9887   | 0.9887 |
+| 0.0969        | 10.5788 | 10600 | 0.0372          | 0.9884   | 0.9884 |
+| 0.0969        | 10.7784 | 10800 | 0.0350          | 0.9903   | 0.9903 |
+| 0.0969        | 10.9780 | 11000 | 0.0386          | 0.9878   | 0.9878 |
+| 0.0864        | 11.1776 | 11200 | 0.0327          | 0.9908   | 0.9908 |
+| 0.0864        | 11.3772 | 11400 | 0.0305          | 0.9915   | 0.9916 |
+| 0.0864        | 11.5768 | 11600 | 0.0304          | 0.9916   | 0.9916 |
+| 0.0864        | 11.7764 | 11800 | 0.0336          | 0.9895   | 0.9895 |
+| 0.0864        | 11.9760 | 12000 | 0.0276          | 0.9925   | 0.9925 |
+| 0.0734        | 12.1756 | 12200 | 0.0265          | 0.9929   | 0.9929 |
+| 0.0734        | 12.3752 | 12400 | 0.0275          | 0.9924   | 0.9924 |
+| 0.0734        | 12.5749 | 12600 | 0.0265          | 0.9928   | 0.9928 |
+| 0.0734        | 12.7745 | 12800 | 0.0256          | 0.9927   | 0.9928 |
+| 0.0734        | 12.9741 | 13000 | 0.0238          | 0.9940   | 0.9940 |
+| 0.0665        | 13.1737 | 13200 | 0.0251          | 0.9929   | 0.9929 |
+| 0.0665        | 13.3733 | 13400 | 0.0242          | 0.9934   | 0.9934 |
+| 0.0665        | 13.5729 | 13600 | 0.0228          | 0.9942   | 0.9942 |
+| 0.0665        | 13.7725 | 13800 | 0.0232          | 0.9938   | 0.9938 |
+| 0.0665        | 13.9721 | 14000 | 0.0228          | 0.9940   | 0.9940 |
+| 0.0623        | 14.1717 | 14200 | 0.0215          | 0.9944   | 0.9944 |
+| 0.0623        | 14.3713 | 14400 | 0.0217          | 0.9943   | 0.9943 |
+| 0.0623        | 14.5709 | 14600 | 0.0213          | 0.9944   | 0.9944 |
+| 0.0623        | 14.7705 | 14800 | 0.0213          | 0.9944   | 0.9944 |
+| 0.0623        | 14.9701 | 15000 | 0.0211          | 0.9945   | 0.9945 |
+
+
+### Framework versions
+
+- Transformers 4.45.2
+- Pytorch 2.1.2
+- Datasets 2.20.0
+- Tokenizers 0.20.1

added_tokens.json

@@ -0,0 +1,3 @@
+{
+  "<mask>": 64000
+}

config.json

@@ -0,0 +1,37 @@
+{
+  "_name_or_path": "vinai/phobert-base-v2",
+  "architectures": [
+    "PhoBertLexical"
+  ],
+  "attention_probs_dropout_prob": 0.1,
+  "bos_token_id": 0,
+  "classifier_dropout": null,
+  "eos_token_id": 2,
+  "hidden_act": "gelu",
+  "hidden_dropout_prob": 0.1,
+  "hidden_size": 768,
+  "id2label": {
+    "0": "B\u00ecnh th\u01b0\u1eddng",
+    "1": "T\u00ednh nhi\u1ec7m th\u1ea5p"
+  },
+  "initializer_range": 0.02,
+  "intermediate_size": 3072,
+  "label2id": {
+    "B\u00ecnh th\u01b0\u1eddng": 0,
+    "T\u00ednh nhi\u1ec7m th\u1ea5p": 1
+  },
+  "layer_norm_eps": 1e-05,
+  "max_position_embeddings": 258,
+  "model_type": "roberta",
+  "num_attention_heads": 12,
+  "num_classes": 2,
+  "num_hidden_layers": 12,
+  "pad_token_id": 1,
+  "position_embedding_type": "absolute",
+  "tokenizer_class": "PhobertTokenizer",
+  "torch_dtype": "float32",
+  "transformers_version": "4.45.2",
+  "type_vocab_size": 1,
+  "use_cache": true,
+  "vocab_size": 64001
+}

model.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:2be7b012f1072f4dc3b7bcfd2c923b8e0335580bbc62669742153ba5e9ca509e
+size 538468008

special_tokens_map.json

@@ -0,0 +1,9 @@
+{
+  "bos_token": "<s>",
+  "cls_token": "<s>",
+  "eos_token": "</s>",
+  "mask_token": "<mask>",
+  "pad_token": "<pad>",
+  "sep_token": "</s>",
+  "unk_token": "<unk>"
+}

tokenizer_config.json

@@ -0,0 +1,54 @@
+{
+  "added_tokens_decoder": {
+    "0": {
+      "content": "<s>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "1": {
+      "content": "<pad>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "2": {
+      "content": "</s>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "3": {
+      "content": "<unk>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "64000": {
+      "content": "<mask>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    }
+  },
+  "bos_token": "<s>",
+  "clean_up_tokenization_spaces": false,
+  "cls_token": "<s>",
+  "eos_token": "</s>",
+  "mask_token": "<mask>",
+  "model_max_length": 1000000000000000019884624838656,
+  "pad_token": "<pad>",
+  "sep_token": "</s>",
+  "tokenizer_class": "PhobertTokenizer",
+  "unk_token": "<unk>"
+}

training_args.bin

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:35d110083428d52f5a746a6b1502c254ec9a05402b6c307d2f6be1bd2cd3cbf6
+size 5176