|
--- |
|
license: apache-2.0 |
|
tags: |
|
- generated_from_trainer |
|
datasets: |
|
- finer-139 |
|
metrics: |
|
- precision |
|
- recall |
|
- f1 |
|
- accuracy |
|
base_model: google/bert_uncased_L-2_H-128_A-2 |
|
model-index: |
|
- name: bertiny-finetuned-finer-full |
|
results: |
|
- task: |
|
type: token-classification |
|
name: Token Classification |
|
dataset: |
|
name: finer-139 |
|
type: finer-139 |
|
args: finer-139 |
|
metrics: |
|
- type: precision |
|
value: 0.555368475586064 |
|
name: Precision |
|
- type: recall |
|
value: 0.5164398410213176 |
|
name: Recall |
|
- type: f1 |
|
value: 0.5351972041937094 |
|
name: F1 |
|
- type: accuracy |
|
value: 0.988733187308122 |
|
name: Accuracy |
|
--- |
|
|
|
<!-- This model card has been generated automatically according to the information the Trainer had access to. You |
|
should probably proofread and complete it, then remove this comment. --> |
|
|
|
# bertiny-finetuned-finer-full |
|
|
|
This model is a fine-tuned version of [google/bert_uncased_L-2_H-128_A-2](https://huggingface.co/google/bert_uncased_L-2_H-128_A-2) on the 10% of finer-139 dataset for 40 epochs according to paper. |
|
It achieves the following results on the evaluation set: |
|
- Loss: 0.0788 |
|
- Precision: 0.5554 |
|
- Recall: 0.5164 |
|
- F1: 0.5352 |
|
- Accuracy: 0.9887 |
|
|
|
## Model description |
|
|
|
More information needed |
|
|
|
## Intended uses & limitations |
|
|
|
More information needed |
|
|
|
## Training and evaluation data |
|
|
|
More information needed |
|
|
|
## Training procedure |
|
|
|
### Training hyperparameters |
|
|
|
The following hyperparameters were used during training: |
|
- learning_rate: 2e-05 |
|
- train_batch_size: 8 |
|
- eval_batch_size: 8 |
|
- seed: 42 |
|
- optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08 |
|
- lr_scheduler_type: linear |
|
- num_epochs: 40 |
|
|
|
### Training results |
|
|
|
| Training Loss | Epoch | Step | Validation Loss | Precision | Recall | F1 | Accuracy | |
|
|:-------------:|:-----:|:------:|:---------------:|:---------:|:------:|:------:|:--------:| |
|
| 0.0852 | 1.0 | 11255 | 0.0929 | 1.0 | 0.0001 | 0.0002 | 0.9843 | |
|
| 0.08 | 2.0 | 22510 | 0.0840 | 0.4626 | 0.0730 | 0.1261 | 0.9851 | |
|
| 0.0759 | 3.0 | 33765 | 0.0750 | 0.5113 | 0.2035 | 0.2912 | 0.9865 | |
|
| 0.0569 | 4.0 | 45020 | 0.0673 | 0.4973 | 0.3281 | 0.3953 | 0.9872 | |
|
| 0.0488 | 5.0 | 56275 | 0.0635 | 0.5289 | 0.3749 | 0.4388 | 0.9878 | |
|
| 0.0422 | 6.0 | 67530 | 0.0606 | 0.5258 | 0.4068 | 0.4587 | 0.9880 | |
|
| 0.0364 | 7.0 | 78785 | 0.0600 | 0.5588 | 0.4186 | 0.4787 | 0.9883 | |
|
| 0.0307 | 8.0 | 90040 | 0.0589 | 0.5223 | 0.4916 | 0.5065 | 0.9883 | |
|
| 0.0284 | 9.0 | 101295 | 0.0595 | 0.5588 | 0.4813 | 0.5171 | 0.9887 | |
|
| 0.0255 | 10.0 | 112550 | 0.0597 | 0.5606 | 0.4944 | 0.5254 | 0.9888 | |
|
| 0.0223 | 11.0 | 123805 | 0.0600 | 0.5533 | 0.4998 | 0.5252 | 0.9888 | |
|
| 0.0228 | 12.0 | 135060 | 0.0608 | 0.5290 | 0.5228 | 0.5259 | 0.9885 | |
|
| 0.0225 | 13.0 | 146315 | 0.0612 | 0.5480 | 0.5111 | 0.5289 | 0.9887 | |
|
| 0.0204 | 14.0 | 157570 | 0.0634 | 0.5646 | 0.5120 | 0.5370 | 0.9890 | |
|
| 0.0176 | 15.0 | 168825 | 0.0639 | 0.5611 | 0.5135 | 0.5363 | 0.9889 | |
|
| 0.0167 | 16.0 | 180080 | 0.0647 | 0.5631 | 0.5120 | 0.5363 | 0.9888 | |
|
| 0.0161 | 17.0 | 191335 | 0.0665 | 0.5607 | 0.5081 | 0.5331 | 0.9889 | |
|
| 0.0145 | 18.0 | 202590 | 0.0673 | 0.5437 | 0.5280 | 0.5357 | 0.9887 | |
|
| 0.0166 | 19.0 | 213845 | 0.0687 | 0.5722 | 0.5008 | 0.5341 | 0.9889 | |
|
| 0.0155 | 20.0 | 225100 | 0.0685 | 0.5325 | 0.5337 | 0.5331 | 0.9885 | |
|
| 0.0142 | 21.0 | 236355 | 0.0705 | 0.5626 | 0.5166 | 0.5386 | 0.9890 | |
|
| 0.0127 | 22.0 | 247610 | 0.0694 | 0.5426 | 0.5358 | 0.5392 | 0.9887 | |
|
| 0.0112 | 23.0 | 258865 | 0.0721 | 0.5591 | 0.5129 | 0.5351 | 0.9888 | |
|
| 0.0123 | 24.0 | 270120 | 0.0733 | 0.5715 | 0.5081 | 0.5380 | 0.9889 | |
|
| 0.0116 | 25.0 | 281375 | 0.0735 | 0.5621 | 0.5123 | 0.5361 | 0.9888 | |
|
| 0.0112 | 26.0 | 292630 | 0.0739 | 0.5634 | 0.5181 | 0.5398 | 0.9889 | |
|
| 0.0108 | 27.0 | 303885 | 0.0753 | 0.5548 | 0.5155 | 0.5344 | 0.9887 | |
|
| 0.0125 | 28.0 | 315140 | 0.0746 | 0.5507 | 0.5221 | 0.5360 | 0.9886 | |
|
| 0.0093 | 29.0 | 326395 | 0.0762 | 0.5602 | 0.5156 | 0.5370 | 0.9888 | |
|
| 0.0094 | 30.0 | 337650 | 0.0762 | 0.5625 | 0.5157 | 0.5381 | 0.9889 | |
|
| 0.0117 | 31.0 | 348905 | 0.0767 | 0.5519 | 0.5195 | 0.5352 | 0.9887 | |
|
| 0.0091 | 32.0 | 360160 | 0.0772 | 0.5501 | 0.5198 | 0.5345 | 0.9887 | |
|
| 0.0109 | 33.0 | 371415 | 0.0775 | 0.5635 | 0.5097 | 0.5353 | 0.9888 | |
|
| 0.0094 | 34.0 | 382670 | 0.0776 | 0.5467 | 0.5216 | 0.5339 | 0.9887 | |
|
| 0.009 | 35.0 | 393925 | 0.0782 | 0.5601 | 0.5139 | 0.5360 | 0.9889 | |
|
| 0.0093 | 36.0 | 405180 | 0.0780 | 0.5568 | 0.5156 | 0.5354 | 0.9888 | |
|
| 0.0087 | 37.0 | 416435 | 0.0783 | 0.5588 | 0.5143 | 0.5356 | 0.9888 | |
|
| 0.009 | 38.0 | 427690 | 0.0785 | 0.5483 | 0.5178 | 0.5326 | 0.9887 | |
|
| 0.0094 | 39.0 | 438945 | 0.0787 | 0.5541 | 0.5154 | 0.5340 | 0.9887 | |
|
| 0.0088 | 40.0 | 450200 | 0.0788 | 0.5554 | 0.5164 | 0.5352 | 0.9887 | |
|
|
|
|
|
### Framework versions |
|
|
|
- Transformers 4.20.1 |
|
- Pytorch 1.12.0+cu113 |
|
- Datasets 2.3.2 |
|
- Tokenizers 0.12.1 |
|
|
