rasyosef commited on
Commit
4298ca1
1 Parent(s): 57dc01c

Update README.md

Browse files
Files changed (1) hide show
  1. README.md +18 -30
README.md CHANGED
@@ -55,15 +55,28 @@ You can use this model directly with a pipeline for masked language modeling:
55
  'sequence': 'ከሀገራቸው ከኢትዮጵያ ከወጡ ግማሽ ምዕተዓመት ተቆጥሯል ።'}]
56
  ```
57
 
58
- # Fine-tuning
59
 
60
  This model was finetuned and evaluated on the following amharic nlp tasks
61
 
62
- - Text Classification
63
  - Sentiment Classification
 
64
  - Named Entity Recognition
65
-
66
- The reported precision, recall, and f1 metrics are macro averages.
 
 
 
 
 
 
 
 
 
 
 
 
 
67
 
68
  ### Amharic News Category Classification
69
 
@@ -73,29 +86,4 @@ The following github repository contains a [notebook](https://github.com/rasyose
73
  |-----|--------------|--------|---------|------|--|
74
  |**bert-small-amharic**|25.7M|0.89|0.86|0.87|0.86|
75
  |bert-mini-amharic|9.67M|0.87|0.83|0.83|0.83|
76
- |xlm-roberta-base|279M|0.9|0.88|0.88|0.88|
77
-
78
- ### Sentiment Classification
79
- The model was finetuned on the [amharic-sentiment](https://huggingface.co/datasets/rasyosef/amharic-sentiment)
80
- dataset to classify the given text as having `positive` or `negative` sentiment
81
-
82
- |Model|Size (# params)| Accuracy | Precision | Recall | F1 |
83
- | --- | ------------- | -------- | --------- | ------ | -- |
84
- |bert-medium-amharic|40.5M|0.83|0.83|0.82|0.83|
85
- |**bert-small-amharic**|27.8M|0.83|0.83|0.82|0.83|
86
- |bert-mini-amharic|10.7M|0.81|0.81|0.81|0.81|
87
- |bert-tiny-amharic|4.18M|0.79|0.79|0.79|0.79|
88
- |xlm-roberta-base|279M|0.83|0.83|0.83|0.83|
89
- |am-roberta|443M|0.82|0.83|0.82|0.82|
90
-
91
- ### Named Entity Recognition
92
- The model was finetuned on the [amharic-named-entity-recognition](https://huggingface.co/datasets/rasyosef/amharic-named-entity-recognition) dataset.
93
-
94
- |Model|Size (# params)| Precision | Recall | F1 |
95
- | --- | ------------- | --------- |------- | -- |
96
- |bert-medium-amharic|40.5M|0.64|0.73|0.68|
97
- |**bert-small-amharic**|27.8M|0.64|0.72|0.68|
98
- |bert-mini-amharic|10.7M|0.60|0.67|0.64|
99
- |bert-tiny-amharic|4.18M|0.50|0.59|0.54|
100
- |xlm-roberta-base|279M|0.69|0.79|0.73|
101
- |am-roberta|443M|0.67|0.72|0.69|
 
55
  'sequence': 'ከሀገራቸው ከኢትዮጵያ ከወጡ ግማሽ ምዕተዓመት ተቆጥሯል ።'}]
56
  ```
57
 
58
+ # Finetuning
59
 
60
  This model was finetuned and evaluated on the following amharic nlp tasks
61
 
 
62
  - Sentiment Classification
63
+ - Dataset: [amharic-sentiment](https://huggingface.co/datasets/rasyosef/amharic-sentiment)
64
  - Named Entity Recognition
65
+ - Dataset: [amharic-named-entity-recognition](https://huggingface.co/datasets/rasyosef/amharic-named-entity-recognition)
66
+ - News Category Classification
67
+ - Dataset: [amharic-news-category-classification](https://github.com/rasyosef/amharic-news-category-classification)
68
+
69
+ ### Finetuned Model Performance
70
+ The reported F1 scores are macro averages.
71
+
72
+ |Model|Size (# params)| Perplexity|Sentiment (F1)| Named Entity Recognition (F1)|
73
+ |-----|---------------|-----------|--------------|------------------------------|
74
+ |bert-medium-amharic|40.5M|13.74|0.83|0.68|
75
+ |**bert-small-amharic**|**27.8M**|**15.96**|**0.83**|**0.68**|
76
+ |bert-mini-amharic|10.7M|22.42|0.81|0.64|
77
+ |bert-tiny-amharic|4.18M|71.52|0.79|0.54|
78
+ |xlm-roberta-base|279M||0.83|0.73|
79
+ |am-roberta|443M||0.82|0.69|
80
 
81
  ### Amharic News Category Classification
82
 
 
86
  |-----|--------------|--------|---------|------|--|
87
  |**bert-small-amharic**|25.7M|0.89|0.86|0.87|0.86|
88
  |bert-mini-amharic|9.67M|0.87|0.83|0.83|0.83|
89
+ |xlm-roberta-base|279M|0.9|0.88|0.88|0.88|