ai-chat-censor6 / README.md
andriadze's picture
Update README.md
0ce5964 verified
|
raw
history blame
2.19 kB
---
library_name: transformers
license: apache-2.0
base_model: distilbert/distilbert-base-uncased
tags:
- generated_from_trainer
metrics:
- accuracy
model-index:
- name: ai-chat-censor6
results: []
---
<!-- This model card has been generated automatically according to the information the Trainer had access to. You
should probably proofread and complete it, then remove this comment. -->
# ai-chat-censor6
The primary focus of the model is detecting sexual/minors category of messages.
Possible flags are: regular, racist, underage, sexual
# BEWARE
The model might categorize any talk about race as racism, for example: "Black people suffer so much in America" will be flagged as "racist".
Model also might flag any comment containing numbers below 18 as underage. This is an issue that will be addressed in next version.
Here's the next version:
https://huggingface.co/andriadze/ai-chat-censor8
## Training and evaluation data
Model was trained on a fully synthetic dataset generated by uncensored 72b models based on qwen2.
This model is a fine-tuned version of [distilbert/distilbert-base-uncased](https://huggingface.co/distilbert/distilbert-base-uncased) on an unknown dataset.
It achieves the following results on the evaluation set:
- Loss: 0.0637
- Accuracy: 0.9903
### Training hyperparameters
The following hyperparameters were used during training:
- learning_rate: 2e-05
- train_batch_size: 16
- eval_batch_size: 16
- seed: 42
- optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
- lr_scheduler_type: linear
- num_epochs: 6
### Training results
| Training Loss | Epoch | Step | Validation Loss | Accuracy |
|:-------------:|:-----:|:----:|:---------------:|:--------:|
| 0.0471 | 1.0 | 1175 | 0.0729 | 0.9854 |
| 0.0282 | 2.0 | 2350 | 0.0529 | 0.9900 |
| 0.0105 | 3.0 | 3525 | 0.0680 | 0.9888 |
| 0.0079 | 4.0 | 4700 | 0.0558 | 0.9911 |
| 0.0017 | 5.0 | 5875 | 0.0595 | 0.9902 |
| 0.0001 | 6.0 | 7050 | 0.0637 | 0.9903 |
### Framework versions
- Transformers 4.44.2
- Pytorch 2.4.0+cu121
- Datasets 3.0.0
- Tokenizers 0.19.1