Hugging Face's logo

dl-ru
/
rubert-tiny2-srl

Token Classification
Transformers
PyTorch
bert
Generated from Trainer
Model card Files
xet
 Community
2
dl-ru commited on
Commit
92aa913
·
verified ·
1 Parent(s): 9f9e1b6

New version with explicit predicate marking

Browse files
Files changed (4) hide show
  1. README.md +37 -40
  2. config.json +42 -44
  3. pytorch_model.bin +2 -2
  4. training_args.bin +2 -2
README.md CHANGED
@@ -15,42 +15,42 @@ should probably proofread and complete it, then remove this comment. -->
15
 
16
  This model is a fine-tuned version of [cointegrated/rubert-tiny2](https://huggingface.co/cointegrated/rubert-tiny2) on an unknown dataset.
17
  It achieves the following results on the evaluation set:
18
- - Loss: 0.1559
19
- - Addressee Precision: 0.7778
20
  - Addressee Recall: 0.875
21
- - Addressee F1: 0.8235
22
  - Addressee Number: 8
23
  - Benefactive Precision: 0.0
24
  - Benefactive Recall: 0.0
25
  - Benefactive F1: 0.0
26
  - Benefactive Number: 2
27
- - Causator Precision: 0.8667
28
  - Causator Recall: 0.8125
29
- - Causator F1: 0.8387
30
  - Causator Number: 16
31
- - Cause Precision: 0.5
32
- - Cause Recall: 0.1667
33
- - Cause F1: 0.25
34
  - Cause Number: 12
35
- - Contrsubject Precision: 0.7333
36
- - Contrsubject Recall: 0.6471
37
- - Contrsubject F1: 0.6875
38
  - Contrsubject Number: 17
39
- - Deliberative Precision: 0.8333
40
- - Deliberative Recall: 0.8333
41
- - Deliberative F1: 0.8333
42
  - Deliberative Number: 6
43
  - Destinative Precision: 1.0
44
- - Destinative Recall: 0.75
45
- - Destinative F1: 0.8571
46
  - Destinative Number: 4
47
  - Directivefinal Precision: 1.0
48
  - Directivefinal Recall: 1.0
49
  - Directivefinal F1: 1.0
50
  - Directivefinal Number: 2
51
- - Experiencer Precision: 0.7870
52
- - Experiencer Recall: 0.8947
53
- - Experiencer F1: 0.8374
54
  - Experiencer Number: 95
55
  - Instrument Precision: 0.0
56
  - Instrument Recall: 0.0
@@ -60,18 +60,14 @@ It achieves the following results on the evaluation set:
60
  - Limitative Recall: 0.0
61
  - Limitative F1: 0.0
62
  - Limitative Number: 1
63
- - Object Precision: 0.7714
64
- - Object Recall: 0.7875
65
- - Object F1: 0.7794
66
  - Object Number: 240
67
- - Predicate Precision: 0.9928
68
- - Predicate Recall: 1.0
69
- - Predicate F1: 0.9964
70
- - Predicate Number: 274
71
- - Overall Precision: 0.8653
72
- - Overall Recall: 0.8691
73
- - Overall F1: 0.8672
74
- - Overall Accuracy: 0.9566
75
 
76
  ## Model description
77
 
@@ -90,24 +86,25 @@ More information needed
90
  ### Training hyperparameters
91
 
92
  The following hyperparameters were used during training:
93
- - learning_rate: 0.0001722734324185955
94
- - train_batch_size: 8
95
  - eval_batch_size: 1
96
- - seed: 815951
97
  - gradient_accumulation_steps: 4
98
- - total_train_batch_size: 32
99
  - optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
100
  - lr_scheduler_type: linear
101
  - lr_scheduler_warmup_ratio: 0.04
102
- - num_epochs: 3
103
 
104
  ### Training results
105
 
106
- | Training Loss | Epoch | Step | Validation Loss | Addressee Precision | Addressee Recall | Addressee F1 | Addressee Number | Benefactive Precision | Benefactive Recall | Benefactive F1 | Benefactive Number | Causator Precision | Causator Recall | Causator F1 | Causator Number | Cause Precision | Cause Recall | Cause F1 | Cause Number | Contrsubject Precision | Contrsubject Recall | Contrsubject F1 | Contrsubject Number | Deliberative Precision | Deliberative Recall | Deliberative F1 | Deliberative Number | Destinative Precision | Destinative Recall | Destinative F1 | Destinative Number | Directivefinal Precision | Directivefinal Recall | Directivefinal F1 | Directivefinal Number | Experiencer Precision | Experiencer Recall | Experiencer F1 | Experiencer Number | Instrument Precision | Instrument Recall | Instrument F1 | Instrument Number | Limitative Precision | Limitative Recall | Limitative F1 | Limitative Number | Object Precision | Object Recall | Object F1 | Object Number | Predicate Precision | Predicate Recall | Predicate F1 | Predicate Number | Overall Precision | Overall Recall | Overall F1 | Overall Accuracy |
107
- |:-------------:|:-----:|:----:|:---------------:|:-------------------:|:----------------:|:------------:|:----------------:|:---------------------:|:------------------:|:--------------:|:------------------:|:------------------:|:---------------:|:-----------:|:---------------:|:---------------:|:------------:|:--------:|:------------:|:----------------------:|:-------------------:|:---------------:|:-------------------:|:----------------------:|:-------------------:|:---------------:|:-------------------:|:---------------------:|:------------------:|:--------------:|:------------------:|:------------------------:|:---------------------:|:-----------------:|:---------------------:|:---------------------:|:------------------:|:--------------:|:------------------:|:--------------------:|:-----------------:|:-------------:|:-----------------:|:--------------------:|:-----------------:|:-------------:|:-----------------:|:----------------:|:-------------:|:---------:|:-------------:|:-------------------:|:----------------:|:------------:|:----------------:|:-----------------:|:--------------:|:----------:|:----------------:|
108
- | 0.2353 | 1.0 | 245 | 0.1957 | 0.6 | 0.375 | 0.4615 | 8 | 0.0 | 0.0 | 0.0 | 2 | 0.7368 | 0.875 | 0.8000 | 16 | 0.2857 | 0.1667 | 0.2105 | 12 | 0.6667 | 0.2353 | 0.3478 | 17 | 0.0 | 0.0 | 0.0 | 6 | 0.0 | 0.0 | 0.0 | 4 | 0.0 | 0.0 | 0.0 | 2 | 0.7477 | 0.8421 | 0.7921 | 95 | 0.0 | 0.0 | 0.0 | 3 | 0.0 | 0.0 | 0.0 | 1 | 0.7696 | 0.6958 | 0.7309 | 240 | 0.9783 | 0.9854 | 0.9818 | 274 | 0.8477 | 0.7941 | 0.8200 | 0.9466 |
109
- | 0.148 | 2.0 | 491 | 0.1635 | 0.7 | 0.875 | 0.7778 | 8 | 0.0 | 0.0 | 0.0 | 2 | 0.8333 | 0.9375 | 0.8824 | 16 | 0.4 | 0.1667 | 0.2353 | 12 | 0.6923 | 0.5294 | 0.6000 | 17 | 0.8 | 0.6667 | 0.7273 | 6 | 1.0 | 0.25 | 0.4 | 4 | 1.0 | 1.0 | 1.0 | 2 | 0.7436 | 0.9158 | 0.8208 | 95 | 0.0 | 0.0 | 0.0 | 3 | 0.0 | 0.0 | 0.0 | 1 | 0.7864 | 0.7208 | 0.7522 | 240 | 0.9892 | 1.0 | 0.9946 | 274 | 0.8593 | 0.8441 | 0.8516 | 0.9538 |
110
- | 0.1046 | 2.99 | 735 | 0.1559 | 0.7778 | 0.875 | 0.8235 | 8 | 0.0 | 0.0 | 0.0 | 2 | 0.8667 | 0.8125 | 0.8387 | 16 | 0.5 | 0.1667 | 0.25 | 12 | 0.7333 | 0.6471 | 0.6875 | 17 | 0.8333 | 0.8333 | 0.8333 | 6 | 1.0 | 0.75 | 0.8571 | 4 | 1.0 | 1.0 | 1.0 | 2 | 0.7870 | 0.8947 | 0.8374 | 95 | 0.0 | 0.0 | 0.0 | 3 | 0.0 | 0.0 | 0.0 | 1 | 0.7714 | 0.7875 | 0.7794 | 240 | 0.9928 | 1.0 | 0.9964 | 274 | 0.8653 | 0.8691 | 0.8672 | 0.9566 |
 
111
 
112
 
113
  ### Framework versions
 
15
 
16
  This model is a fine-tuned version of [cointegrated/rubert-tiny2](https://huggingface.co/cointegrated/rubert-tiny2) on an unknown dataset.
17
  It achieves the following results on the evaluation set:
18
+ - Loss: 0.1428
19
+ - Addressee Precision: 0.6364
20
  - Addressee Recall: 0.875
21
+ - Addressee F1: 0.7368
22
  - Addressee Number: 8
23
  - Benefactive Precision: 0.0
24
  - Benefactive Recall: 0.0
25
  - Benefactive F1: 0.0
26
  - Benefactive Number: 2
27
+ - Causator Precision: 0.9286
28
  - Causator Recall: 0.8125
29
+ - Causator F1: 0.8667
30
  - Causator Number: 16
31
+ - Cause Precision: 0.6
32
+ - Cause Recall: 0.25
33
+ - Cause F1: 0.3529
34
  - Cause Number: 12
35
+ - Contrsubject Precision: 0.6364
36
+ - Contrsubject Recall: 0.4118
37
+ - Contrsubject F1: 0.5
38
  - Contrsubject Number: 17
39
+ - Deliberative Precision: 1.0
40
+ - Deliberative Recall: 0.6667
41
+ - Deliberative F1: 0.8
42
  - Deliberative Number: 6
43
  - Destinative Precision: 1.0
44
+ - Destinative Recall: 0.5
45
+ - Destinative F1: 0.6667
46
  - Destinative Number: 4
47
  - Directivefinal Precision: 1.0
48
  - Directivefinal Recall: 1.0
49
  - Directivefinal F1: 1.0
50
  - Directivefinal Number: 2
51
+ - Experiencer Precision: 0.8018
52
+ - Experiencer Recall: 0.9368
53
+ - Experiencer F1: 0.8641
54
  - Experiencer Number: 95
55
  - Instrument Precision: 0.0
56
  - Instrument Recall: 0.0
 
60
  - Limitative Recall: 0.0
61
  - Limitative F1: 0.0
62
  - Limitative Number: 1
63
+ - Object Precision: 0.7589
64
+ - Object Recall: 0.8
65
+ - Object F1: 0.7789
66
  - Object Number: 240
67
+ - Overall Precision: 0.7724
68
+ - Overall Recall: 0.7857
69
+ - Overall F1: 0.7790
70
+ - Overall Accuracy: 0.9589
 
 
 
 
71
 
72
  ## Model description
73
 
 
86
  ### Training hyperparameters
87
 
88
  The following hyperparameters were used during training:
89
+ - learning_rate: 8.017672397578385e-05
90
+ - train_batch_size: 4
91
  - eval_batch_size: 1
92
+ - seed: 678943
93
  - gradient_accumulation_steps: 4
94
+ - total_train_batch_size: 16
95
  - optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
96
  - lr_scheduler_type: linear
97
  - lr_scheduler_warmup_ratio: 0.04
98
+ - num_epochs: 4
99
 
100
  ### Training results
101
 
102
+ | Training Loss | Epoch | Step | Validation Loss | Addressee Precision | Addressee Recall | Addressee F1 | Addressee Number | Benefactive Precision | Benefactive Recall | Benefactive F1 | Benefactive Number | Causator Precision | Causator Recall | Causator F1 | Causator Number | Cause Precision | Cause Recall | Cause F1 | Cause Number | Contrsubject Precision | Contrsubject Recall | Contrsubject F1 | Contrsubject Number | Deliberative Precision | Deliberative Recall | Deliberative F1 | Deliberative Number | Destinative Precision | Destinative Recall | Destinative F1 | Destinative Number | Directivefinal Precision | Directivefinal Recall | Directivefinal F1 | Directivefinal Number | Experiencer Precision | Experiencer Recall | Experiencer F1 | Experiencer Number | Instrument Precision | Instrument Recall | Instrument F1 | Instrument Number | Limitative Precision | Limitative Recall | Limitative F1 | Limitative Number | Object Precision | Object Recall | Object F1 | Object Number | Overall Precision | Overall Recall | Overall F1 | Overall Accuracy |
103
+ |:-------------:|:-----:|:----:|:---------------:|:-------------------:|:----------------:|:------------:|:----------------:|:---------------------:|:------------------:|:--------------:|:------------------:|:------------------:|:---------------:|:-----------:|:---------------:|:---------------:|:------------:|:--------:|:------------:|:----------------------:|:-------------------:|:---------------:|:-------------------:|:----------------------:|:-------------------:|:---------------:|:-------------------:|:---------------------:|:------------------:|:--------------:|:------------------:|:------------------------:|:---------------------:|:-----------------:|:---------------------:|:---------------------:|:------------------:|:--------------:|:------------------:|:--------------------:|:-----------------:|:-------------:|:-----------------:|:--------------------:|:-----------------:|:-------------:|:-----------------:|:----------------:|:-------------:|:---------:|:-------------:|:-----------------:|:--------------:|:----------:|:----------------:|
104
+ | 0.2206 | 1.0 | 490 | 0.1959 | 0.6667 | 0.25 | 0.3636 | 8 | 0.0 | 0.0 | 0.0 | 2 | 0.8667 | 0.8125 | 0.8387 | 16 | 0.0 | 0.0 | 0.0 | 12 | 1.0 | 0.0588 | 0.1111 | 17 | 0.0 | 0.0 | 0.0 | 6 | 0.0 | 0.0 | 0.0 | 4 | 0.0 | 0.0 | 0.0 | 2 | 0.7203 | 0.8947 | 0.7981 | 95 | 0.0 | 0.0 | 0.0 | 3 | 0.0 | 0.0 | 0.0 | 1 | 0.6692 | 0.725 | 0.696 | 240 | 0.6927 | 0.6773 | 0.6849 | 0.9445 |
105
+ | 0.1507 | 2.0 | 981 | 0.1492 | 0.5556 | 0.625 | 0.5882 | 8 | 0.0 | 0.0 | 0.0 | 2 | 0.8667 | 0.8125 | 0.8387 | 16 | 0.6 | 0.25 | 0.3529 | 12 | 0.75 | 0.3529 | 0.48 | 17 | 1.0 | 0.1667 | 0.2857 | 6 | 1.0 | 0.25 | 0.4 | 4 | 1.0 | 1.0 | 1.0 | 2 | 0.8646 | 0.8737 | 0.8691 | 95 | 0.0 | 0.0 | 0.0 | 3 | 0.0 | 0.0 | 0.0 | 1 | 0.7635 | 0.7667 | 0.7651 | 240 | 0.7884 | 0.7340 | 0.7602 | 0.9566 |
106
+ | 0.1146 | 3.0 | 1472 | 0.1437 | 0.6364 | 0.875 | 0.7368 | 8 | 0.0 | 0.0 | 0.0 | 2 | 0.9286 | 0.8125 | 0.8667 | 16 | 0.6 | 0.25 | 0.3529 | 12 | 0.6429 | 0.5294 | 0.5806 | 17 | 1.0 | 0.5 | 0.6667 | 6 | 1.0 | 0.5 | 0.6667 | 4 | 1.0 | 1.0 | 1.0 | 2 | 0.8 | 0.9263 | 0.8585 | 95 | 0.0 | 0.0 | 0.0 | 3 | 0.0 | 0.0 | 0.0 | 1 | 0.7443 | 0.8125 | 0.7769 | 240 | 0.7612 | 0.7931 | 0.7768 | 0.9584 |
107
+ | 0.0842 | 3.99 | 1960 | 0.1428 | 0.6364 | 0.875 | 0.7368 | 8 | 0.0 | 0.0 | 0.0 | 2 | 0.9286 | 0.8125 | 0.8667 | 16 | 0.6 | 0.25 | 0.3529 | 12 | 0.6364 | 0.4118 | 0.5 | 17 | 1.0 | 0.6667 | 0.8 | 6 | 1.0 | 0.5 | 0.6667 | 4 | 1.0 | 1.0 | 1.0 | 2 | 0.8018 | 0.9368 | 0.8641 | 95 | 0.0 | 0.0 | 0.0 | 3 | 0.0 | 0.0 | 0.0 | 1 | 0.7589 | 0.8 | 0.7789 | 240 | 0.7724 | 0.7857 | 0.7790 | 0.9589 |
108
 
109
 
110
  ### Framework versions
config.json CHANGED
@@ -12,54 +12,52 @@
12
  "hidden_size": 312,
13
  "id2label": {
14
  "0": "O",
15
- "1": "B-Predicate",
16
- "2": "B-Object",
17
- "3": "B-Experiencer",
18
- "4": "B-Cause",
19
- "5": "B-Deliberative",
20
- "6": "B-Causator",
21
- "7": "B-ContrSubject",
22
- "8": "B-Benefactive",
23
- "9": "B-Addressee",
24
- "10": "I-Object",
25
- "11": "B-Destinative",
26
- "12": "I-ContrSubject",
27
- "13": "B-Instrument",
28
- "14": "I-Deliberative",
29
- "15": "B-Limitative",
30
- "16": "B-DirectiveFinal",
31
- "17": "B-Mediative",
32
- "18": "I-DirectiveFinal",
33
- "19": "B-DirectiveInitial",
34
- "20": "I-DirectiveInitial",
35
- "21": "I-Experiencer",
36
- "22": "I-Cause"
37
  },
38
  "initializer_range": 0.02,
39
  "intermediate_size": 600,
40
  "label2id": {
41
- "B-Addressee": 9,
42
- "B-Benefactive": 8,
43
- "B-Causator": 6,
44
- "B-Cause": 4,
45
- "B-ContrSubject": 7,
46
- "B-Deliberative": 5,
47
- "B-Destinative": 11,
48
- "B-DirectiveFinal": 16,
49
- "B-DirectiveInitial": 19,
50
- "B-Experiencer": 3,
51
- "B-Instrument": 13,
52
- "B-Limitative": 15,
53
- "B-Mediative": 17,
54
- "B-Object": 2,
55
- "B-Predicate": 1,
56
- "I-Cause": 22,
57
- "I-ContrSubject": 12,
58
- "I-Deliberative": 14,
59
- "I-DirectiveFinal": 18,
60
- "I-DirectiveInitial": 20,
61
- "I-Experiencer": 21,
62
- "I-Object": 10,
63
  "O": 0
64
  },
65
  "layer_norm_eps": 1e-12,
 
12
  "hidden_size": 312,
13
  "id2label": {
14
  "0": "O",
15
+ "1": "B-Object",
16
+ "2": "B-Experiencer",
17
+ "3": "B-Cause",
18
+ "4": "B-Deliberative",
19
+ "5": "B-Causator",
20
+ "6": "B-ContrSubject",
21
+ "7": "B-Benefactive",
22
+ "8": "B-Addressee",
23
+ "9": "I-Object",
24
+ "10": "B-Destinative",
25
+ "11": "I-ContrSubject",
26
+ "12": "B-Instrument",
27
+ "13": "I-Deliberative",
28
+ "14": "B-Limitative",
29
+ "15": "B-DirectiveFinal",
30
+ "16": "B-Mediative",
31
+ "17": "I-DirectiveFinal",
32
+ "18": "B-DirectiveInitial",
33
+ "19": "I-DirectiveInitial",
34
+ "20": "I-Experiencer",
35
+ "21": "I-Cause"
 
36
  },
37
  "initializer_range": 0.02,
38
  "intermediate_size": 600,
39
  "label2id": {
40
+ "B-Addressee": 8,
41
+ "B-Benefactive": 7,
42
+ "B-Causator": 5,
43
+ "B-Cause": 3,
44
+ "B-ContrSubject": 6,
45
+ "B-Deliberative": 4,
46
+ "B-Destinative": 10,
47
+ "B-DirectiveFinal": 15,
48
+ "B-DirectiveInitial": 18,
49
+ "B-Experiencer": 2,
50
+ "B-Instrument": 12,
51
+ "B-Limitative": 14,
52
+ "B-Mediative": 16,
53
+ "B-Object": 1,
54
+ "I-Cause": 21,
55
+ "I-ContrSubject": 11,
56
+ "I-Deliberative": 13,
57
+ "I-DirectiveFinal": 17,
58
+ "I-DirectiveInitial": 19,
59
+ "I-Experiencer": 20,
60
+ "I-Object": 9,
 
61
  "O": 0
62
  },
63
  "layer_norm_eps": 1e-12,
pytorch_model.bin CHANGED
@@ -1,3 +1,3 @@
1
  version https://git-lfs.github.com/spec/v1
2
- oid sha256:88625d7c7d464a0afa4da7a3df8db8b6a68c9824280018291ff6cfa4ac8cf937
3
- size 116430550
 
1
  version https://git-lfs.github.com/spec/v1
2
+ oid sha256:d557ee951a120b38ed50084bcfead5e3c16fae4e582949932a8e354cdfff96cc
3
+ size 116429334
training_args.bin CHANGED
@@ -1,3 +1,3 @@
1
  version https://git-lfs.github.com/spec/v1
2
- oid sha256:92bccd8424bd2b44800b16c85d657112a9c10b5e891d03dc6382bd1c44d6e5d8
3
- size 4091
 
1
  version https://git-lfs.github.com/spec/v1
2
+ oid sha256:2a0dd85f6470330f97378a28fa85c857715de3b90e12ee9351662122d02a9901
3
+ size 4155