Spaces:
Sleeping
Sleeping
Report for ProsusAI/finbert on financial_phrasebank (sentences_allagree, train set)
#3
by
giskard-bot
- opened
Performance issues (7)
| Vulnerability | Level | Data slice | Metric | Transformation | Deviation | Description |
|---|---|---|---|---|---|---|
| Performance | major | text contains "share" |
Precision = 0.000e+00 | — | -100.00% than global | For records in the dataset where text contains "share", the Precision is 100.0% lower than the global Precision. |
| Performance | major | text contains "finland" |
Balanced Accuracy = 0.003 | — | -66.72% than global | For records in the dataset where text contains "finland", the Balanced Accuracy is 66.72% lower than the global Balanced Accuracy. |
| Performance | major | text contains "mn" |
Precision = 0.004 | — | -59.69% than global | For records in the dataset where text contains "mn", the Precision is 59.69% lower than the global Precision. |
| Performance | major | text contains "year" |
Precision = 0.005 | — | -56.53% than global | For records in the dataset where text contains "year", the Precision is 56.53% lower than the global Precision. |
| Performance | major | text contains "operating" |
Precision = 0.005 | — | -52.60% than global | For records in the dataset where text contains "operating", the Precision is 52.6% lower than the global Precision. |
| Performance | major | text contains "million" |
Precision = 0.006 | — | -47.00% than global | For records in the dataset where text contains "million", the Precision is 47.0% lower than the global Precision. |
| Performance | major | text contains "eur" |
Precision = 0.007 | — | -36.26% than global | For records in the dataset where text contains "eur", the Precision is 36.26% lower than the global Precision. |
Robustness issues (1)
| Vulnerability | Level | Data slice | Metric | Transformation | Deviation | Description |
|---|---|---|---|---|---|---|
| Robustness | medium | — | Fail rate = 0.061 | Add typos | 61/1000 tested samples (6.1%) changed prediction after perturbation | When feature “text” is perturbed with the transformation “Add typos”, the model changes its prediction in 6.1% of the cases. We expected the predictions not to be affected by this transformation. |