|
--- |
|
language: |
|
- en |
|
|
|
tags: |
|
- text-classification |
|
- fake-news |
|
- pytorch |
|
datasets: |
|
- Fake News https://www.kaggle.com/datasets/clmentbisaillon/fake-and-real-news-dataset |
|
metrics: |
|
- Accuracy, AUC |
|
--- |
|
|
|
## Model description: |
|
[Distilbert](https://arxiv.org/abs/1910.01108) is created with knowledge distillation during the pre-training phase which reduces the size of a BERT model by 40%, while retaining 97% of its language understanding. It's smaller, faster than Bert and any other Bert-based model. |
|
|
|
[Distilbert-base-uncased](https://huggingface.co/distilbert-base-uncased) finetuned on the fake news dataset with below Hyperparameters |
|
``` |
|
learning rate 5e-5, |
|
batch size 32, |
|
num_train_epochs=2, |
|
``` |
|
Full code available @ [DistilBert-FakeNews](https://github.com/anasserhussien/DistilBert-FakeNews) |
|
|
|
Dataset available @ [Fake News dataset](https://www.kaggle.com/datasets/clmentbisaillon/fake-and-real-news-dataset) |
|
|
|
|
