---
license: bigscience-openrail-m
base_model: ehsanaghaei/SecureBERT
tags:
- generated_from_trainer
metrics:
- precision
- recall
- f1
- accuracy
model-index:
- name: Cyber-ThreaD/SecureBERT-DNRTI
  results: []
---

<!-- This model card has been generated automatically according to the information the Trainer had access to. You
should probably proofread and complete it, then remove this comment. -->

# Cyber-ThreaD/SecureBERT-DNRTI

This model is a fine-tuned version of [ehsanaghaei/SecureBERT](https://huggingface.co/ehsanaghaei/SecureBERT) on the [DNRTI](https://github.com/SCreaMxp/DNRTI-A-Large-scale-Dataset-for-Named-Entity-Recognition-in-Threat-Intelligence) dataset.
It achieves the following results on the evaluation set:
- Loss: 0.2427
- Precision: 0.7694
- Recall: 0.7854
- F1: 0.7773
- Accuracy: 0.9382

It achieves the following results on the prediction set:
- Precision: 0.8346
- Recall: 0.8403
- F1: 0.8374
- Accuracy: 0.9554

## Model description

More information needed

## Intended uses & limitations

More information needed

## Training and evaluation data

More information needed

## Training procedure

### Training hyperparameters

The following hyperparameters were used during training:
- learning_rate: 2e-05
- train_batch_size: 8
- eval_batch_size: 8
- seed: 42
- optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
- lr_scheduler_type: linear
- num_epochs: 10.0

### Training results

| Training Loss | Epoch | Step | Validation Loss | Precision | Recall | F1     | Accuracy |
|:-------------:|:-----:|:----:|:---------------:|:---------:|:------:|:------:|:--------:|
| 0.7821        | 0.76  | 500  | 0.4215          | 0.5219    | 0.5919 | 0.5547 | 0.8745   |
| 0.3559        | 1.52  | 1000 | 0.3152          | 0.6272    | 0.6587 | 0.6426 | 0.9008   |
| 0.2807        | 2.28  | 1500 | 0.2952          | 0.6445    | 0.7232 | 0.6816 | 0.9084   |
| 0.2272        | 3.04  | 2000 | 0.2793          | 0.6682    | 0.7513 | 0.7073 | 0.9161   |
| 0.1837        | 3.81  | 2500 | 0.2489          | 0.7151    | 0.7526 | 0.7334 | 0.9258   |
| 0.1497        | 4.57  | 3000 | 0.2511          | 0.7254    | 0.7826 | 0.7529 | 0.9286   |
| 0.1371        | 5.33  | 3500 | 0.2496          | 0.7425    | 0.7757 | 0.7587 | 0.9331   |
| 0.1135        | 6.09  | 4000 | 0.2554          | 0.7289    | 0.8075 | 0.7662 | 0.9325   |
| 0.1018        | 6.85  | 4500 | 0.2427          | 0.7694    | 0.7854 | 0.7773 | 0.9382   |
| 0.0899        | 7.61  | 5000 | 0.2516          | 0.7583    | 0.8167 | 0.7864 | 0.9378   |
| 0.0809        | 8.37  | 5500 | 0.2459          | 0.7717    | 0.8176 | 0.7940 | 0.9406   |
| 0.0763        | 9.13  | 6000 | 0.2553          | 0.7518    | 0.8217 | 0.7852 | 0.9392   |
| 0.0687        | 9.89  | 6500 | 0.2534          | 0.7621    | 0.8204 | 0.7902 | 0.9407   |


### Framework versions

- Transformers 4.36.0.dev0
- Pytorch 2.1.0+cu118
- Datasets 2.14.6
- Tokenizers 0.14.1


### Citing & Authors

If you use the model kindly cite the following work

```
@inproceedings{deka2024attacker,
  title={AttackER: Towards Enhancing Cyber-Attack Attribution with a Named Entity Recognition Dataset},
  author={Deka, Pritam and Rajapaksha, Sampath and Rani, Ruby and Almutairi, Amirah and Karafili, Erisa},
  booktitle={International Conference on Web Information Systems Engineering},
  pages={255--270},
  year={2024},
  organization={Springer}
}

```