Update README.md
Browse files
README.md
CHANGED
@@ -55,15 +55,28 @@ You can use this model directly with a pipeline for masked language modeling:
|
|
55 |
'sequence': 'ከሀገራቸው ከኢትዮጵያ ከወጡ ግማሽ ምዕተዓመት ተቆጥሯል ።'}]
|
56 |
```
|
57 |
|
58 |
-
#
|
59 |
|
60 |
This model was finetuned and evaluated on the following amharic nlp tasks
|
61 |
|
62 |
-
- Text Classification
|
63 |
- Sentiment Classification
|
|
|
64 |
- Named Entity Recognition
|
65 |
-
|
66 |
-
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
67 |
|
68 |
### Amharic News Category Classification
|
69 |
|
@@ -73,29 +86,4 @@ The following github repository contains a [notebook](https://github.com/rasyose
|
|
73 |
|-----|--------------|--------|---------|------|--|
|
74 |
|**bert-small-amharic**|25.7M|0.89|0.86|0.87|0.86|
|
75 |
|bert-mini-amharic|9.67M|0.87|0.83|0.83|0.83|
|
76 |
-
|xlm-roberta-base|279M|0.9|0.88|0.88|0.88|
|
77 |
-
|
78 |
-
### Sentiment Classification
|
79 |
-
The model was finetuned on the [amharic-sentiment](https://huggingface.co/datasets/rasyosef/amharic-sentiment)
|
80 |
-
dataset to classify the given text as having `positive` or `negative` sentiment
|
81 |
-
|
82 |
-
|Model|Size (# params)| Accuracy | Precision | Recall | F1 |
|
83 |
-
| --- | ------------- | -------- | --------- | ------ | -- |
|
84 |
-
|bert-medium-amharic|40.5M|0.83|0.83|0.82|0.83|
|
85 |
-
|**bert-small-amharic**|27.8M|0.83|0.83|0.82|0.83|
|
86 |
-
|bert-mini-amharic|10.7M|0.81|0.81|0.81|0.81|
|
87 |
-
|bert-tiny-amharic|4.18M|0.79|0.79|0.79|0.79|
|
88 |
-
|xlm-roberta-base|279M|0.83|0.83|0.83|0.83|
|
89 |
-
|am-roberta|443M|0.82|0.83|0.82|0.82|
|
90 |
-
|
91 |
-
### Named Entity Recognition
|
92 |
-
The model was finetuned on the [amharic-named-entity-recognition](https://huggingface.co/datasets/rasyosef/amharic-named-entity-recognition) dataset.
|
93 |
-
|
94 |
-
|Model|Size (# params)| Precision | Recall | F1 |
|
95 |
-
| --- | ------------- | --------- |------- | -- |
|
96 |
-
|bert-medium-amharic|40.5M|0.64|0.73|0.68|
|
97 |
-
|**bert-small-amharic**|27.8M|0.64|0.72|0.68|
|
98 |
-
|bert-mini-amharic|10.7M|0.60|0.67|0.64|
|
99 |
-
|bert-tiny-amharic|4.18M|0.50|0.59|0.54|
|
100 |
-
|xlm-roberta-base|279M|0.69|0.79|0.73|
|
101 |
-
|am-roberta|443M|0.67|0.72|0.69|
|
|
|
55 |
'sequence': 'ከሀገራቸው ከኢትዮጵያ ከወጡ ግማሽ ምዕተዓመት ተቆጥሯል ።'}]
|
56 |
```
|
57 |
|
58 |
+
# Finetuning
|
59 |
|
60 |
This model was finetuned and evaluated on the following amharic nlp tasks
|
61 |
|
|
|
62 |
- Sentiment Classification
|
63 |
+
- Dataset: [amharic-sentiment](https://huggingface.co/datasets/rasyosef/amharic-sentiment)
|
64 |
- Named Entity Recognition
|
65 |
+
- Dataset: [amharic-named-entity-recognition](https://huggingface.co/datasets/rasyosef/amharic-named-entity-recognition)
|
66 |
+
- News Category Classification
|
67 |
+
- Dataset: [amharic-news-category-classification](https://github.com/rasyosef/amharic-news-category-classification)
|
68 |
+
|
69 |
+
### Finetuned Model Performance
|
70 |
+
The reported F1 scores are macro averages.
|
71 |
+
|
72 |
+
|Model|Size (# params)| Perplexity|Sentiment (F1)| Named Entity Recognition (F1)|
|
73 |
+
|-----|---------------|-----------|--------------|------------------------------|
|
74 |
+
|bert-medium-amharic|40.5M|13.74|0.83|0.68|
|
75 |
+
|**bert-small-amharic**|**27.8M**|**15.96**|**0.83**|**0.68**|
|
76 |
+
|bert-mini-amharic|10.7M|22.42|0.81|0.64|
|
77 |
+
|bert-tiny-amharic|4.18M|71.52|0.79|0.54|
|
78 |
+
|xlm-roberta-base|279M||0.83|0.73|
|
79 |
+
|am-roberta|443M||0.82|0.69|
|
80 |
|
81 |
### Amharic News Category Classification
|
82 |
|
|
|
86 |
|-----|--------------|--------|---------|------|--|
|
87 |
|**bert-small-amharic**|25.7M|0.89|0.86|0.87|0.86|
|
88 |
|bert-mini-amharic|9.67M|0.87|0.83|0.83|0.83|
|
89 |
+
|xlm-roberta-base|279M|0.9|0.88|0.88|0.88|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|