|
--- |
|
language: "it" |
|
tags: |
|
- bert |
|
- sarcasm-detection |
|
- text-classification |
|
widget: |
|
- text: "Auto, stop a diesel e benzina dal 2035. Ecco cosa cambia per i consumatori" |
|
--- |
|
|
|
# Italian Sarcasm Detector |
|
|
|
Italian Sarcasm Detector is a text classification model built to detect sarcasm from news article titles. It is fine-tuned on [dbmdz/bert-base-italian-uncased](https://huggingface.co/dbmdz/bert-base-italian-uncased) and the training data consists of scraped data from Italian non-sarcastic newspaper (Il Giornale) and sarcastic newspaper (Lercio). |
|
|
|
|
|
<b>Labels</b>: |
|
0 -> Not Sarcastic; |
|
1 -> Sarcastic |
|
|
|
|
|
## Training Data |
|
|
|
Scraped data: |
|
- Italian non-sarcastic news from [Il Giornale](https://www.ilgiornale.it) |
|
- Italian sarcastic news from [Lercio](https://www.lercio.it) |
|
|
|
Codebase: |
|
- Git Repo: [Official repository](https://github.com/helinivan/multilingual-sarcasm-detector) |
|
|
|
--- |
|
|
|
## Example of classification |
|
|
|
```python |
|
from transformers import AutoModelForSequenceClassification |
|
from transformers import AutoTokenizer |
|
import string |
|
|
|
def preprocess_data(text: str) -> str: |
|
return text.lower().translate(str.maketrans("", "", string.punctuation)).strip() |
|
|
|
MODEL_PATH = "helinivan/italian-sarcasm-detector" |
|
|
|
tokenizer = AutoTokenizer.from_pretrained(MODEL_PATH) |
|
model = AutoModelForSequenceClassification.from_pretrained(MODEL_PATH) |
|
|
|
text = "Auto, stop a diesel e benzina dal 2035. Ecco cosa cambia per i consumatori" |
|
tokenized_text = tokenizer([preprocess_data(text)], padding=True, truncation=True, max_length=512, return_tensors="pt") |
|
output = model(**tokenized_text) |
|
probs = output.logits.softmax(dim=-1).tolist()[0] |
|
confidence = max(probs) |
|
prediction = probs.index(confidence) |
|
results = {"is_sarcastic": prediction, "confidence": confidence} |
|
|
|
``` |
|
|
|
Output: |
|
|
|
``` |
|
{'is_sarcastic': 1, 'confidence': 0.9999909400939941} |
|
``` |
|
|
|
## Performance |
|
| Model-Name | F1 | Precision | Recall | Accuracy |
|
| ------------- |:-------------| -----| -----| ----| |
|
| [helinivan/english-sarcasm-detector ](https://huggingface.co/helinivan/english-sarcasm-detector)| 92.38 | 92.75 | 92.38 | 92.42 |
|
| [helinivan/italian-sarcasm-detector ](https://huggingface.co/helinivan/italian-sarcasm-detector) | **88.26** | 87.66 | 89.66 | 88.69 |
|
| [helinivan/multilingual-sarcasm-detector ](https://huggingface.co/helinivan/multilingual-sarcasm-detector) | 87.23 | 88.65 | 86.33 | 88.30 |
|
| [helinivan/dutch-sarcasm-detector ](https://huggingface.co/helinivan/dutch-sarcasm-detector) | 83.02 | 84.27 | 82.01 | 86.81 |
