multiclass_vulnerability_classifier

This model is a fine-tuned version of google/flan-t5-small on the None dataset. It achieves the following results on the evaluation set:

• Loss: 1.0699

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 2e-05
• train_batch_size: 8
• eval_batch_size: 8
• seed: 42
• gradient_accumulation_steps: 4
• total_train_batch_size: 32
• optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
• lr_scheduler_type: linear
• num_epochs: 20

Training results

Training Loss	Epoch	Step	Validation Loss
No log	1.0	41	30.4020
No log	2.0	82	9.8970
No log	3.0	123	5.3734
No log	4.0	164	4.3631
No log	5.0	205	3.9439
No log	6.0	246	3.5538
No log	7.0	287	3.1623
No log	8.0	328	2.7757
No log	9.0	369	2.4235
No log	10.0	410	2.1290
No log	11.0	451	1.8848
No log	12.0	492	1.6854
8.4834	13.0	533	1.5165
8.4834	14.0	574	1.3829
8.4834	15.0	615	1.2813
8.4834	16.0	656	1.2007
8.4834	17.0	697	1.1428
8.4834	18.0	738	1.1028
8.4834	19.0	779	1.0779
8.4834	20.0	820	1.0699

Framework versions

• Transformers 4.42.4
• Pytorch 2.3.1+cu121
• Datasets 2.21.0
• Tokenizers 0.19.1