🏆 GermEval 2025: Violence Detection (Class-Weighted)

abullardUR@GermEval Shared Task 2025 Submission

🎯 Model Summary

This model is a fine-tuned version of LSX-UniWue/ModernGBERT_134M, specifically designed for Violence Detection (VIO) in German social media content. It was developed as part of the GermEval 2025 Shared Task on Harmful Content Detection.

🏅 Competition Performance

Final Ranking: 2nd out of 8 teams 🥈
Primary Metric (Macro-F1): 0.82 (+19% over official baseline (0.69 → 0.82))
Approach: Class-weighted cross-entropy loss for severe class imbalance (12.8:1 ratio)

📊 Task Details

Task Type: Binary classification
Classes: False (no violence), True (violence-related content detected)
Domain: German social media (Twitter, 2014-2016)
Data Source: Right-wing extremist network posts

⚠️ Limitations and Bias

Known Limitations

Domain Specificity: Trained on 2014-2016 German Twitter data from right-wing extremist networks
Temporal Bias: Language patterns may not reflect contemporary usage
Class Imbalance: 92.8% non-violent vs 7.2% violent content (12.8:1 ratio)
Cultural Context: May not generalize to other German-speaking regions or contexts
Implicit Violence: May struggle with subtle threats or coded violent language

Ethical Considerations

Model trained on potentially harmful content for research purposes only
Should not be used to amplify or generate violent content
Requires careful handling due to sensitive training data
May exhibit bias toward certain types of violence or contexts

🚀 How to Use

Quick Start

from transformers import AutoProcessor, AutoModelForSequenceClassification

# Load model and processor
model_id = "abullard1/germeval2025-vio-moderngbert-cw"
processor = AutoProcessor.from_pretrained(model_id, trust_remote_code=True)
model = AutoModelForSequenceClassification.from_pretrained(model_id, trust_remote_code=True).eval()

# Run inference
text = "Diese Situation macht mich wütend!"
inputs = processor(text, return_tensors="pt", truncation=True)
probs = model(**inputs).logits.softmax(-1).detach().cpu().numpy()
print(f"Predictions: {probs}")

Class Labels

Label 0: No violence detected
Label 1: Violence-related content detected

📈 Training Details

Training Data

Source: GermEval 2025 Shared Task VIO dataset
Size: 7,783 samples
Split: 80% training (6,226), 20% validation (1,557)
Class Distribution: 92.8% non-violent, 7.2% violent

Training Procedure

Base Model: ModernGBERT-134M (8192 token context)
Architecture: Mean-pooling classification head
Loss Function: Class-weighted cross-entropy (inverse frequency weighting)
Optimizer: AdamW with linear scheduling
Early Stopping: Patience of 5 epochs on validation Macro-F1

Hyperparameters

Learning Rate: 5e-5
Weight Decay: 0.0301
Batch Size: 16/32 (train/eval)
Epochs: 3
Warmup Steps: 300

📚 Citation

@inproceedings{bullard2025germeval,
  title   = {abullardUR@GermEval Shared Task 2025: Fine-tuning ModernGBERT on Highly Imbalanced German Social Media for Harmful Content Detection},
  author  = {Bullard, Samuel},
  year    = {2025},
  booktitle = {Proceedings of KONVENS 2025 Workshops}
}

🙏 Acknowledgments

GermEval 2025 Organizers: University of Stuttgart and University of Mannheim
Prof. Dr. Udo Kruschwitz (University of Regensburg) for supervision
ModernGBERT Team: LSX-UniWue for the ModernGBERT-134M German language base-model