hyenadna-medium-450k-seqlen-hf_ft_BioS2_1kbpHG19_DHSs_H3K27AC

This model is a fine-tuned version of LongSafari/hyenadna-medium-450k-seqlen-hf on the None dataset. It achieves the following results on the evaluation set:

• Loss: 0.4664
• F1 Score: 0.8050
• Precision: 0.7960
• Recall: 0.8141
• Accuracy: 0.7932
• Auc: 0.8726
• Prc: 0.8695

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 1e-05
• train_batch_size: 16
• eval_batch_size: 16
• seed: 42
• optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
• lr_scheduler_type: linear
• num_epochs: 20
• mixed_precision_training: Native AMP

Training results

Training Loss	Epoch	Step	Validation Loss	F1 Score	Precision	Recall	Accuracy	Auc	Prc
0.561	0.1682	500	0.5369	0.7760	0.7043	0.8639	0.7385	0.7986	0.7757
0.5175	0.3365	1000	0.5174	0.7666	0.7529	0.7807	0.7508	0.8132	0.7964
0.5015	0.5047	1500	0.4905	0.7942	0.7265	0.8758	0.7621	0.8354	0.8196
0.4923	0.6729	2000	0.4868	0.7845	0.7653	0.8048	0.7683	0.8351	0.8193
0.4786	0.8412	2500	0.4833	0.7929	0.7716	0.8154	0.7767	0.8468	0.8335
0.4767	1.0094	3000	0.4572	0.8062	0.7699	0.8462	0.7868	0.8595	0.8507
0.4431	1.1777	3500	0.4899	0.7679	0.8116	0.7287	0.7691	0.8486	0.8384
0.4272	1.3459	4000	0.4569	0.8054	0.7801	0.8324	0.7892	0.8628	0.8535
0.4394	1.5141	4500	0.4544	0.7961	0.8073	0.7852	0.7892	0.8696	0.8630
0.434	1.6824	5000	0.4483	0.8045	0.7936	0.8157	0.7922	0.8688	0.8644
0.4411	1.8506	5500	0.4454	0.8199	0.7694	0.8774	0.7979	0.8721	0.8649
0.4238	2.0188	6000	0.4617	0.7927	0.8057	0.7801	0.7861	0.8675	0.8643
0.3887	2.1871	6500	0.4518	0.7971	0.8163	0.7788	0.7922	0.8724	0.8714
0.3922	2.3553	7000	0.4510	0.8050	0.8070	0.8029	0.7961	0.8742	0.8709
0.3943	2.5236	7500	0.4423	0.8207	0.7711	0.8770	0.7991	0.8769	0.8724
0.3899	2.6918	8000	0.4576	0.8007	0.8050	0.7965	0.7922	0.8725	0.8687
0.3901	2.8600	8500	0.4486	0.8218	0.7582	0.8970	0.7961	0.8737	0.8687
0.386	3.0283	9000	0.4721	0.8037	0.7910	0.8167	0.7908	0.8668	0.8611
0.3453	3.1965	9500	0.5186	0.7673	0.8195	0.7213	0.7707	0.8586	0.8532
0.3522	3.3647	10000	0.4681	0.8064	0.7772	0.8379	0.7892	0.8693	0.8689
0.3473	3.5330	10500	0.4696	0.8048	0.8042	0.8055	0.7952	0.8702	0.8676
0.3377	3.7012	11000	0.4664	0.8050	0.7960	0.8141	0.7932	0.8726	0.8695

Framework versions

• Transformers 4.42.3
• Pytorch 2.3.0+cu121
• Datasets 2.18.0
• Tokenizers 0.19.0