Spaces:
Running
Report for rafalposwiata/deproberta-large-depression
Hey Team!🤗✨
We’re thrilled to share some amazing evaluation results that’ll make your day!🎉📊
We have identified 2 potential vulnerabilities in your model based on an automated scan.
This automated analysis evaluated the model on the dataset tweet_eval (subset sentiment, split train).
👉Performance issues (2)
| Vulnerability | Level | Data slice | Metric | Transformation | Deviation |
|---|---|---|---|---|---|
| Performance | major 🔴 | text contains "night" |
Precision = 0.094 | — | -39.31% than global |
🔍✨Examples
For records in the dataset where `text` contains "night", the Precision is 39.31% lower than the global Precision.| text | label | Predicted label |
|
|---|---|---|---|
| 2 | Sorry bout the stream last night I crashed out but will be on tonight for sure. Then back to Minecraft in pc tomorrow night. | neutral | negative (p = 0.96) |
| 6 | """"" SOUL TRAIN"""" OCT 27 HALLOWEEN SPECIAL ft T.dot FINEST rocking the mic...CRAZY CACTUS NIGHT CLUB ..ADV ticket $10 wt out costume $15..." | positive | negative (p = 0.93) |
| 25 | @user Work colleague of mine on Thursday night: ""oh, look, they're showing the Bee Gees on video."" 23-yr old colleague: ""who?""" | neutral | negative (p = 0.86) |
| Vulnerability | Level | Data slice | Metric | Transformation | Deviation |
|---|---|---|---|---|---|
| Performance | major 🔴 | text contains "day" |
Precision = 0.116 | — | -25.54% than global |
🔍✨Examples
For records in the dataset where `text` contains "day", the Precision is 25.54% lower than the global Precision.| text | label | Predicted label |
|
|---|---|---|---|
| 27 | Yes glass of red\u002c Rammstein and day off tomorrow (thank you @user just what I needed. | positive | negative (p = 0.98) |
| 35 | "last day of august, waiting for frank ocean to pull a beyonce. | neutral | negative (p = 0.51) |
| 37 | Sunday (tomorrow) is National Ice Cream Day and have we got a gift for you! Join us for an ice cream sundae and... | positive | negative (p = 0.98) |
Checkout out the Giskard Space and improve your model.
Disclaimer: it's important to note that automated scans may produce false positives or miss certain vulnerabilities. We encourage you to review the findings and assess the impact accordingly.
💡 What's Next?
- The Giskard community is always buzzing with ideas. 🐢🤔 What do you want to see next? Your feedback is our favorite fuel, so drop your thoughts in the community forum! 🗣️💬 Together, we're building something extraordinary.
🙌 Big Thanks!
We're grateful to have you on this adventure with us. 🚀🌟 Here's to more breakthroughs, laughter, and code magic! 🥂✨ Keep hugging that code and spreading the love! 💻 #Giskard #Huggingface #AISafety 🌈👏 Your enthusiasm, feedback, and contributions are what seek. 🌟 Keep being awesome!