Upload improved README
Browse files
README.md
CHANGED
@@ -8,12 +8,88 @@ tags:
|
|
8 |
- ner
|
9 |
- named-entity-recognition
|
10 |
pipeline_tag: token-classification
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
11 |
---
|
12 |
|
13 |
-
# SpanMarker for Named Entity Recognition
|
14 |
|
15 |
-
This is a [SpanMarker](https://github.com/tomaarsen/SpanMarkerNER) model that can be used for Named Entity Recognition. In particular, this SpanMarker model uses [bert-base-uncased](https://huggingface.co/bert-base-uncased) as the underlying encoder.
|
16 |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
17 |
|
18 |
## Usage
|
19 |
|
@@ -29,9 +105,48 @@ You can then run inference with this model like so:
|
|
29 |
from span_marker import SpanMarkerModel
|
30 |
|
31 |
# Download from the 🤗 Hub
|
32 |
-
model = SpanMarkerModel.from_pretrained("
|
33 |
# Run inference
|
34 |
-
entities = model.predict("
|
35 |
```
|
36 |
|
37 |
See the [SpanMarker](https://github.com/tomaarsen/SpanMarkerNER) repository for documentation and additional information on this library.
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
8 |
- ner
|
9 |
- named-entity-recognition
|
10 |
pipeline_tag: token-classification
|
11 |
+
model-index:
|
12 |
+
- name: SpanMarker w. bert-base-uncased on CrossNER by Tom Aarsen
|
13 |
+
results:
|
14 |
+
- task:
|
15 |
+
type: token-classification
|
16 |
+
name: Named Entity Recognition
|
17 |
+
dataset:
|
18 |
+
type: P3ps/Cross_ner
|
19 |
+
name: CrossNER
|
20 |
+
split: test
|
21 |
+
revision: 7cecbbb3d2eb8c75c8571c53e5a5270cfd0c5a9e
|
22 |
+
metrics:
|
23 |
+
- type: f1
|
24 |
+
value: 0.8708
|
25 |
+
name: F1
|
26 |
+
- type: precision
|
27 |
+
value: 0.8763
|
28 |
+
name: Precision
|
29 |
+
- type: recall
|
30 |
+
value: 0.8654
|
31 |
+
name: Recall
|
32 |
+
datasets:
|
33 |
+
- P3ps/Cross_ner
|
34 |
+
language:
|
35 |
+
- en
|
36 |
+
metrics:
|
37 |
+
- f1
|
38 |
+
- recall
|
39 |
+
- precision
|
40 |
---
|
41 |
|
42 |
+
# SpanMarker for uncased Named Entity Recognition
|
43 |
|
44 |
+
This is a [SpanMarker](https://github.com/tomaarsen/SpanMarkerNER) model that can be used for Named Entity Recognition. In particular, this SpanMarker model uses [bert-base-uncased](https://huggingface.co/bert-base-uncased) as the underlying encoder. See [train.py](train.py) for the training script.
|
45 |
|
46 |
+
Is your data always capitalized correctly? Then consider using the cased variant of this model instead for better performance:
|
47 |
+
[tomaarsen/span-marker-bert-base-cross-ner](https://huggingface.co/tomaarsen/span-marker-bert-base-cross-ner).
|
48 |
+
|
49 |
+
## Labels & Metrics
|
50 |
+
|
51 |
+
| **Label** | **Examples** | **Precision** | **Recall** | **F1** |
|
52 |
+
|:-------------------|-|------------:|---------:|------:|
|
53 |
+
| **all** | - | 87.63 | 86.54 | 87.08 |
|
54 |
+
| academicjournal | "new journal of physics", "epl", "european physical journal b" | 82.22 | 90.24 | 86.05 |
|
55 |
+
| album | "tellin' stories", "generation terrorists", "country airs" | 84.46 | 84.46 | 84.46 |
|
56 |
+
| algorithm | "lda", "pca", "gradient descent" | 82.86 | 76.99 | 79.82 |
|
57 |
+
| astronomicalobject | "earth", "sun", "halley's comet" | 88.61 | 94.59 | 91.50 |
|
58 |
+
| award | "nobel prize for literature", "acamedy award for best actress", "mandelbrot's awards" | 87.76 | 91.63 | 89.66 |
|
59 |
+
| band | "clash", "parliament funkadelic", "sly and the family stone" | 82.72 | 85.35 | 84.01 |
|
60 |
+
| book | "nietzsche contra wagner" , "dionysian-dithyrambs", "the rebel" | 68.51 | 79.49 | 73.59 |
|
61 |
+
| chemicalcompound | "hydrogen sulfide", "starch", "lactic acid" | 73.33 | 66.67 | 69.84 |
|
62 |
+
| chemicalelement | "potassium", "fluorine", "chlorine" | 95.65 | 73.33 | 83.02 |
|
63 |
+
| conference | "siggraph", "ijcai", "ieee transactions on speech and audio processing" | 72.41 | 60.00 | 65.62 |
|
64 |
+
| country | "united arab emirates", "u.s.", "canada" | 81.03 | 86.08 | 83.48 |
|
65 |
+
| discipline | "physics", "meteorology", "geography" | 35.48 | 40.74 | 37.93 |
|
66 |
+
| election | "2004 canadian federal election", "2006 canadian federal election", "1999 scottish parliament election" | 96.22 | 98.28 | 97.24 |
|
67 |
+
| enzyme | "rna polymerase", "phosphoinositide 3-kinase", "protein kinase c" | 72.09 | 83.78 | 77.50 |
|
68 |
+
| event | "cannes film festival", "2019 special olympics world summer games", "2017 western iraq campaign" | 68.12 | 60.22 | 63.93 |
|
69 |
+
| field | "computational imaging", "electronics", "information theory" | 92.13 | 77.36 | 84.10 |
|
70 |
+
| literarygenre | "novel", "satire", "short story" | 65.26 | 72.09 | 68.51 |
|
71 |
+
| location | "china", "bombay", "serbia" | 94.78 | 93.68 | 94.23 |
|
72 |
+
| magazine | "the atlantic", "the american spectator", "astounding science fiction" | 60.71 | 60.71 | 60.71 |
|
73 |
+
| metrics | "bleu", "precision", "dcg" | 77.01 | 82.72 | 79.76 |
|
74 |
+
| misc | "serbian", "belgian", "the birth of a nation" | 80.11 | 72.12 | 75.91 |
|
75 |
+
| musicalartist | "chuck burgi", "john miceli", "john o'reilly" | 78.84 | 84.44 | 81.55 |
|
76 |
+
| musicalinstrument | "koto", "bubens", "def" | 75.00 | 33.33 | 46.15 |
|
77 |
+
| musicgenre | "christian rock", "punk rock", "romantic melodicism" | 88.21 | 88.21 | 88.21 |
|
78 |
+
| organisation | "irish times", "comintern", "wimbledon" | 89.17 | 89.98 | 89.57 |
|
79 |
+
| person | "gong zhichao", "liu lufung", "margret crowley" | 95.87 | 92.65 | 94.23 |
|
80 |
+
| poem | "historia destructionis troiae", "i am joaquin", "the snow man" | 94.29 | 64.71 | 76.74 |
|
81 |
+
| politicalparty | "new democratic party", "bloc québécois", "liberal party of canada" | 87.16 | 84.50 | 85.81 |
|
82 |
+
| politician | "susan kadis", "simon strelchik", "lloyd helferty" | 85.23 | 90.71 | 87.89 |
|
83 |
+
| product | "alphago", "wordnet", "facial recognition system" | 63.95 | 65.48 | 64.71 |
|
84 |
+
| programlang | "r", "c++", "java" | 75.00 | 84.38 | 79.41 |
|
85 |
+
| protein | "dna methyltransferase", "tau protein", "amyloid beta" | 57.50 | 66.67 | 61.74 |
|
86 |
+
| researcher | "sirovich", "kirby", "matthew turk" | 93.06 | 75.28 | 83.23 |
|
87 |
+
| scientist | "matjaž perc", "cotton", "singer" | 80.27 | 93.72 | 86.47 |
|
88 |
+
| song | "right where i'm supposed to be", "easy", "three times a lady" | 89.87 | 82.56 | 86.06 |
|
89 |
+
| task | "robot control", "elevator scheduling", "telecommunications" | 73.86 | 75.58 | 74.71 |
|
90 |
+
| theory | "big bang", "general theory of relativity", "ptolemaic planetary theories" | 0.00 | 0.00 | 0.00 |
|
91 |
+
| university | "university of göttingen", "duke", "imperial academy of sciences" | 79.78 | 79.78 | 79.78 |
|
92 |
+
| writer | "thomas mann", "george bernard shaw", "thomas hardy" | 77.78 | 86.19 | 81.77 |
|
93 |
|
94 |
## Usage
|
95 |
|
|
|
105 |
from span_marker import SpanMarkerModel
|
106 |
|
107 |
# Download from the 🤗 Hub
|
108 |
+
model = SpanMarkerModel.from_pretrained("tomaarsen/span-marker-bert-base-uncased-cross-ner")
|
109 |
# Run inference
|
110 |
+
entities = model.predict("amelia earhart flew her single engine lockheed vega 5b across the atlantic to paris.")
|
111 |
```
|
112 |
|
113 |
See the [SpanMarker](https://github.com/tomaarsen/SpanMarkerNER) repository for documentation and additional information on this library.
|
114 |
+
|
115 |
+
## Training procedure
|
116 |
+
|
117 |
+
### Training hyperparameters
|
118 |
+
|
119 |
+
The following hyperparameters were used during training:
|
120 |
+
- learning_rate: 5e-05
|
121 |
+
- train_batch_size: 32
|
122 |
+
- eval_batch_size: 32
|
123 |
+
- seed: 42
|
124 |
+
- optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
|
125 |
+
- lr_scheduler_type: linear
|
126 |
+
- lr_scheduler_warmup_ratio: 0.1
|
127 |
+
- num_epochs: 3
|
128 |
+
|
129 |
+
### Training results
|
130 |
+
|
131 |
+
| Training Loss | Epoch | Step | Validation Loss | Overall Precision | Overall Recall | Overall F1 | Overall Accuracy |
|
132 |
+
|:-------------:|:-----:|:----:|:---------------:|:-----------------:|:--------------:|:----------:|:----------------:|
|
133 |
+
| 0.0641 | 0.25 | 200 | 0.0445 | 0.7141 | 0.5496 | 0.6212 | 0.8700 |
|
134 |
+
| 0.0268 | 0.5 | 400 | 0.0224 | 0.8171 | 0.7510 | 0.7827 | 0.9314 |
|
135 |
+
| 0.0213 | 0.76 | 600 | 0.0187 | 0.8387 | 0.8013 | 0.8196 | 0.9444 |
|
136 |
+
| 0.017 | 1.01 | 800 | 0.0162 | 0.8623 | 0.8231 | 0.8422 | 0.9497 |
|
137 |
+
| 0.0141 | 1.26 | 1000 | 0.0163 | 0.8571 | 0.8384 | 0.8477 | 0.9535 |
|
138 |
+
| 0.0132 | 1.51 | 1200 | 0.0149 | 0.8711 | 0.8470 | 0.8589 | 0.9563 |
|
139 |
+
| 0.0113 | 1.76 | 1400 | 0.0150 | 0.8603 | 0.8523 | 0.8563 | 0.9556 |
|
140 |
+
| 0.0097 | 2.02 | 1600 | 0.0150 | 0.8710 | 0.8553 | 0.8631 | 0.9573 |
|
141 |
+
| 0.0083 | 2.27 | 1800 | 0.0148 | 0.8809 | 0.8568 | 0.8687 | 0.9586 |
|
142 |
+
| 0.0075 | 2.52 | 2000 | 0.0150 | 0.8733 | 0.8573 | 0.8652 | 0.9583 |
|
143 |
+
| 0.0068 | 2.77 | 2200 | 0.0148 | 0.8745 | 0.8642 | 0.8693 | 0.9600 |
|
144 |
+
|
145 |
+
|
146 |
+
### Framework versions
|
147 |
+
|
148 |
+
- SpanMarker 1.2.4
|
149 |
+
- Transformers 4.31.0
|
150 |
+
- Pytorch 2.0.1+cu118
|
151 |
+
- Datasets 2.14.3
|
152 |
+
- Tokenizers 0.13.2
|