Update README.md

README.md

@@ -29,33 +29,24 @@ language:
 pipeline_tag: token-classification
 ---
 
-
-
 # piiranha-v1
 Piiranha is trained to detect 17 types of Personally Identifiable Information (PII) across six languages. It successfully catches 98.27% of PII tokens, with an overall classification accuracy of 99.44%.
+Piiranha is especially accurate at detecting passwords, emails (100%), phone numbers, and usernames.
 
 Supported languages: English, Spanish, French, German, Italian, Dutch
 Supported PII types: Account Number, Building Number, City, Credit Card Number, Date of Birth, Driver's License, Email, First Name, Last Name, ID Card, Password, Social Security Number, Street Address, Tax Number, Phone Number, Username, Zipcode. 
 
-      ACCOUNTNUM       0.84      0.87      0.85      3575
-     BUILDINGNUM       0.92      0.90      0.91      3252
-            CITY       0.95      0.97      0.96      7270
-CREDITCARDNUMBER       0.94      0.96      0.95      2308
-     DATEOFBIRTH       0.93      0.85      0.89      3389
-DRIVERLICENSENUM       0.96      0.96      0.96      2244
-           EMAIL       1.00      1.00      1.00      6892
-       GIVENNAME       0.87      0.93      0.90     12150
-       IDCARDNUM       0.89      0.94      0.91      3700
-        PASSWORD       0.98      0.98      0.98      2387
-       SOCIALNUM       0.93      0.94      0.93      2709
-          STREET       0.97      0.95      0.96      3331
-         SURNAME       0.89      0.78      0.83      8267
-          TAXNUM       0.97      0.89      0.93      2322
-    TELEPHONENUM       0.99      1.00      0.99      5039
-        USERNAME       0.98      0.98      0.98      7680
-         ZIPCODE       0.94      0.97      0.95      3191
-
-It is a fine-tuned version of [microsoft/mdeberta-v3-base](https://huggingface.co/microsoft/mdeberta-v3-base).
+Performance on PII vs. Non PII classification task:
+**Precision: 98.48%** (98.48% of tokens classified as PII are actually PII)
+**Recall: 98.27%** (correctly identifies 98.27% of PII tokens)
+**Specificity: 99.84%** (correctly identifies 99.84% of Non PII tokens)
+
+<img src="https://cloud-3i4ld6u5y-hack-club-bot.vercel.app/0home.png" alt="Akash Network logo" width="400"/>
+
+Piiranha was trained on an H100 GPU rented through the [Akash Network](https://akash.network/).
+
+## Model Description
+Piiranha is a fine-tuned version of [microsoft/mdeberta-v3-base](https://huggingface.co/microsoft/mdeberta-v3-base).
 It achieves the following results on a test set of ~73,000 sentences containing PII:
 - Accuracy: 99.44%
 - Loss: 0.0173
@@ -63,18 +54,41 @@ It achieves the following results on a test set of ~73,000 sentences containing
 - Recall: 93.08%
 - F1: 93.12%
 
-## Model description
-
-More information needed
+Note that the above metrics factor in the eighteen possible categories (17 PII and 1 Non PII), so the metrics are lower than the metrics for just PII vs. Non PII (binary classification).
+
+## Performance by PII type
+Reported performance metrics are lower than the overall accuracy of 99.44% due to class imbalance (most tokens are not PII).
+However, the model is more useful than the below results suggest, due to the intent behind PII detection. The model sometimes misclassifies one PII type for another, but at the end of the day, it still recognizes the token as PII.
+For instance, the model often confuses first names for last names, but that's fine because it still flags the name as PII.
+
+| Entity              | Precision | Recall | F1-Score | Support |
+|---------------------|-----------|--------|----------|---------|
+| ACCOUNTNUM          | 0.84      | 0.87   | 0.85     | 3575    |
+| BUILDINGNUM         | 0.92      | 0.90   | 0.91     | 3252    |
+| CITY                | 0.95      | 0.97   | 0.96     | 7270    |
+| CREDITCARDNUMBER    | 0.94      | 0.96   | 0.95     | 2308    |
+| DATEOFBIRTH         | 0.93      | 0.85   | 0.89     | 3389    |
+| DRIVERLICENSENUM    | 0.96      | 0.96   | 0.96     | 2244    |
+| EMAIL               | 1.00      | 1.00   | 1.00     | 6892    |
+| GIVENNAME           | 0.87      | 0.93   | 0.90     | 12150   |
+| IDCARDNUM           | 0.89      | 0.94   | 0.91     | 3700    |
+| PASSWORD            | 0.98      | 0.98   | 0.98     | 2387    |
+| SOCIALNUM           | 0.93      | 0.94   | 0.93     | 2709    |
+| STREET              | 0.97      | 0.95   | 0.96     | 3331    |
+| SURNAME             | 0.89      | 0.78   | 0.83     | 8267    |
+| TAXNUM              | 0.97      | 0.89   | 0.93     | 2322    |
+| TELEPHONENUM        | 0.99      | 1.00   | 0.99     | 5039    |
+| USERNAME            | 0.98      | 0.98   | 0.98     | 7680    |
+| ZIPCODE             | 0.94      | 0.97   | 0.95     | 3191    |
+| **micro avg**       | 0.93      | 0.93   | 0.93     | 79706   |
+| **macro avg**       | 0.94      | 0.93   | 0.93     | 79706   |
+| **weighted avg**    | 0.93      | 0.93   | 0.93     | 79706   |
 
 ## Intended uses & limitations
 
-More information needed
-
+Piiranha can be used to assist with redacting PII from texts. Use at your own risk. We do not accept responsibility for any incorrect model predictions.
 ## Training and evaluation data
 
-More information needed
-
 ## Training procedure
 
 ### Training hyperparameters