Spaces:
Running
Report for bhadresh-savani/distilbert-base-uncased-emotion
Hey Team!🤗✨
We’re thrilled to share some amazing evaluation results that’ll make your day!🎉📊
We have identified 1 potential vulnerabilities in your model based on an automated scan.
This automated analysis evaluated the model on the dataset dair-ai/emotion (subset split, split validation).
👉Robustness issues (1)
| Vulnerability | Level | Data slice | Metric | Transformation | Deviation |
|---|---|---|---|---|---|
| Robustness | major 🔴 | — | Fail rate = 0.222 | Add typos | 222/1000 tested samples (22.2%) changed prediction after perturbation |
🔍✨Examples
When feature “text” is perturbed with the transformation “Add typos”, the model changes its prediction in 22.2% of the cases. We expected the predictions not to be affected by this transformation.| text | Add typos(text) | Original prediction | Prediction after perturbation | |
|---|---|---|---|---|
| 656 | i feel a little bit more nostalgic when those memories come to mind | i feel a little bit more nosftalic when those memories comwe to mind | love (p = 1.00) | joy (p = 0.99) |
| 734 | i can talk to her about almost anything i want to and she just listens and she doesnt make me feel like a whiney brat and she helps me sort my thoughts and make decisions while keeping me where she feels im safe | i can talk to her about almost anything i want to and she just lisrens and she doesnt make me feel liek a shiney brat and she helps me sort my thoughts and make decisions while keeping me where she fes im safe | sadness (p = 1.00) | joy (p = 1.00) |
| 1403 | i feel the need to preface this by saying that i am strongly in favor of keeping violent or otherwise inappropriate videogames out of the hands of minors and i believe that this is an issue that parents and the government need to work on together | i feel the need to preface this by saying that i am ateongly in faor of keeping volent or otherwise inappropriate videogames outo f yhe hands of minor san di believe that this is an issue that parents and the government need to work on tovether | anger (p = 1.00) | sadness (p = 0.88) |
Disclaimer: it's important to note that automated scans may produce false positives or miss certain vulnerabilities. We encourage you to review the findings and assess the impact accordingly.
💡 What's Next?
- Checkout the Giskard Space and improve your model.
- The Giskard community is always buzzing with ideas. 🐢🤔 What do you want to see next? Your feedback is our favorite fuel, so drop your thoughts in the community forum! 🗣️💬 Together, we're building something extraordinary.
🙌 Big Thanks!
We're grateful to have you on this adventure with us. 🚀🌟 Here's to more breakthroughs, laughter, and code magic! 🥂✨ Keep hugging that code and spreading the love! 💻 #Giskard #Huggingface #AISafety 🌈👏 Your enthusiasm, feedback, and contributions are what seek. 🌟 Keep being awesome!