metadata

language: it
tags:
  - bert
  - sarcasm-detection
  - text-classification
widget:
  - text: Auto, stop a diesel e benzina dal 2035. Ecco cosa cambia per i consumatori

Italian Sarcasm Detector

Italian Sarcasm Detector is a text classification model built to detect sarcasm from news article titles. It is fine-tuned on dbmdz/bert-base-italian-uncased and the training data consists of scraped data from Italian non-sarcastic newspaper (Il Giornale) and sarcastic newspaper (Lercio).

Labels: 0 -> Not Sarcastic; 1 -> Sarcastic

Training Data

Scraped data:

Italian non-sarcastic news from Il Giornale
Italian sarcastic news from Lercio

Codebase:

Git Repo: Official repository

Example of classification

from transformers import AutoModelForSequenceClassification
from transformers import AutoTokenizer
import string

def preprocess_data(text: str) -> str:
   return text.lower().translate(str.maketrans("", "", string.punctuation)).strip()

MODEL_PATH = "helinivan/italian-sarcasm-detector"

tokenizer = AutoTokenizer.from_pretrained(MODEL_PATH)
model = AutoModelForSequenceClassification.from_pretrained(MODEL_PATH)

text = "Auto, stop a diesel e benzina dal 2035. Ecco cosa cambia per i consumatori"
tokenized_text = tokenizer([preprocess_data(text)], padding=True, truncation=True, max_length=512, return_tensors="pt")
output = model(**tokenized_text)
probs = output.logits.softmax(dim=-1).tolist()[0]
confidence = max(probs)
prediction = probs.index(confidence)
results = {"is_sarcastic": prediction, "confidence": confidence}

Output:

{'is_sarcastic': 1, 'confidence': 0.9999909400939941}

Performance

Model-Name	F1	Precision	Recall	Accuracy
helinivan/english-sarcasm-detector	94.48	94.46	94.51	94.48
helinivan/italian-sarcasm-detector	92.99	92.77	93.24	93.42
helinivan/multilingual-sarcasm-detector	90.91	91.51	90.44	91.55
helinivan/dutch-sarcasm-detector	89.04	90.16	88.09	91.43