ProkBERT PhaStyle

Model Name: neuralbioinfo/PhaStyle-mini
Model Type: Genomic Language Model (ProkBERT-based)
Model Description:

ProkBERT PhaStyle is a fine-tuned genomic language model designed for phage lifestyle prediction. It classifies phages as either virulent or temperate directly from nucleotide sequences. The model is based on ProkBERT architecture and was trained on the BACPHLIP dataset, excluding E. coli sequences By leveraging transfer learning, ProkBERT PhaStyle is optimized for handling fragmented sequences, commonly encountered in metagenomic and metavirome datasets. The model provides a fast, efficient alternative to traditional methods without requiring complex preprocessing pipelines or curated databases.

Key Points:

Trained on BACPHLIP dataset excluding E. coli sequences.
Segment Length for training: 512 base pairs.
Output: Binary classification (virulent or temperate).
Model Parameters: ~25 million parameters.

Intended Use

ProkBERT PhaStyle is designed for phage lifestyle prediction tasks, suitable for:

Phage Therapy: Identifying virulent phages for bacterial infection treatment.
Microbiome Engineering: Understanding the interaction between temperate and virulent phages in various microbiomes.
Metagenomic Studies: Classifying fragmented phage sequences from environmental or clinical samples.

Inference Code

ProkBERT PhaStyle requires the ProkBERT tokenizer and a custom classification model (BertForBinaryClassificationWithPooling). Below is a high-level overview of how to use the model in inference mode:

aaa

python bin/PhaStyle.py \
    --fastain data/EXTREMOPHILE/extremophiles.fasta \
    --out output_predictions.tsv \
    --ftmodel neuralbioinfo/PhaStyle-mini \
    --modelclass BertForBinaryClassificationWithPooling \
    --per_device_eval_batch_size 196

Datasets Used:

BACPHLIP (without E. coli): 1,868 training sequences and 246 validation sequences.
Guelin Collection: 394 Escherichia phages (temperate and virulent types).
EXTREMOPHILE Phages: 16 phages isolated from extreme environments, including deep-sea, acidic, and arsenic-rich habitats.

Each dataset was processed using 512bp segment lengths to simulate fragmented metagenomic assemblies.

Performance Results

The performance of ProkBERT PhaStyle was evaluated on various datasets, including Escherichia and EXTREMOPHILE phages, using segment lengths of 512bp and 1022bp. The results are summarized below:

Performance on Escherichia Test Set (512bp segments)

Method	Balanced Accuracy	MCC	Sensitivity	Specificity
ProkBERT-mini	0.91	0.83	0.94	0.89
ProkBERT-mini-long	0.90	0.82	0.96	0.85
ProkBERT-mini-c	0.89	0.80	0.95	0.84
DNABERT-2-117M	0.84	0.72	0.95	0.74
Nucleotide Transformer-50m	0.85	0.72	0.92	0.78
Nucleotide Transformer-100m	0.87	0.75	0.93	0.82
Nucleotide Transformer-500m	0.88	0.78	0.96	0.80
DeePhage	0.86	0.71	0.84	0.88
PhaTYP	0.91	0.83	0.94	0.88

Performance on Escherichia Test Set (1022bp segments)

Method	Balanced Accuracy	MCC	Sensitivity	Specificity
ProkBERT-mini	0.94	0.88	0.97	0.91
ProkBERT-mini-long	0.94	0.89	0.97	0.91
ProkBERT-mini-c	0.93	0.87	0.97	0.89
DNABERT-2-117M	0.90	0.80	0.95	0.85
Nucleotide Transformer-50m	0.90	0.80	0.94	0.85
Nucleotide Transformer-100m	0.92	0.83	0.94	0.89
Nucleotide Transformer-500m	0.91	0.84	0.96	0.87
DeePhage	0.91	0.82	0.94	0.88
PhaTYP	0.92	0.84	0.96	0.87

Performance on EXTREMOPHILE Test Set (512bp segments)

Method	Balanced Accuracy	MCC	Sensitivity	Specificity
ProkBERT-mini	0.93	0.83	0.99	0.87
ProkBERT-mini-long	0.93	0.82	1.00	0.86
ProkBERT-mini-c	0.92	0.80	0.99	0.84
DNABERT-2-117M	0.89	0.74	0.99	0.79
Nucleotide Transformer-50m	0.91	0.79	0.98	0.84
Nucleotide Transformer-100m	0.90	0.76	0.97	0.82
Nucleotide Transformer-500m	0.91	0.78	0.99	0.82
DeePhage	0.87	0.75	0.84	0.91
PhaTYP	0.76	0.52	0.74	0.79

Performance on EXTREMOPHILE Test Set (1022bp segments)

Method	Balanced Accuracy	MCC	Sensitivity	Specificity
ProkBERT-mini	0.96	0.91	1.00	0.93
ProkBERT-mini-long	0.96	0.90	1.00	0.92
ProkBERT-mini-c	0.94	0.86	1.00	0.89
DNABERT-2-117M	0.94	0.85	0.98	0.90
Nucleotide Transformer-50m	0.93	0.83	0.99	0.87
Nucleotide Transformer-100m	0.95	0.88	0.98	0.91
Nucleotide Transformer-500m	0.96	0.89	1.00	0.91
DeePhage	0.92	0.80	0.96	0.87
PhaTYP	0.80	0.58	0.84	0.76

Inference Speed and Running Times

Model	Execution Time (seconds)	Inference Speed (MB/sec)
ProkBERT-mini-long	132	0.52
ProkBERT-mini	141	0.49
ProkBERT-mini-c	146	0.47
DNABERT-2-117M	284	0.23
Nucleotide Transformer-50m	292	0.21
Nucleotide Transformer-100m	313	0.20
Nucleotide Transformer-500m	500	0.15
DeePhage	159	0.43
PhaTYP	2718	0.10
BACPHLIP	7125	0.04

For more detailed results, including additional metrics, please refer to the original research paper.

Inference Speed and Running Times

The computational performance of ProkBERT PhaStyle was evaluated using 1,000 randomly selected sequences from the BACPHLIP dataset. The evaluation was performed on a consistent hardware setup with NVIDIA Tesla A100 GPUs. The execution times and inference speeds of various models are summarized below:

Execution Times (in seconds)

Model	Execution Time (seconds)	Inference Speed (MB/sec)
ProkBERT-mini-long	132	0.52
ProkBERT-mini	141	0.49
ProkBERT-mini-c	146	0.47
DNABERT-2-117M	248	0.25
Nucleotide Transformer-50m	342	0.18
Nucleotide Transformer-500m	502	0.12
DeePhage	159	0.43
PhaTYP	2,718	0.03
BACPHLIP	7,125	0.01

Limitations

ProkBERT PhaStyle is specifically designed for binary classification of phage lifestyles (virulent vs. temperate) and does not handle non-phage sequences. It is recommended to use this model in conjunction with upstream pipelines that identify phage sequences. For large-scale inference, GPU support is strongly advised.

Citing this work

If you use the data in this package, please cite:

@Article{ProkBERT2024,
  author  = {Ligeti, Balázs and Szepesi-Nagy, István and Bodnár, Babett and Ligeti-Nagy, Noémi and Juhász, János},
  journal = {Frontiers in Microbiology},
  title   = {{ProkBERT} family: genomic language models for microbiome applications},
  year    = {2024},
  volume  = {14},
  URL={https://www.frontiersin.org/articles/10.3389/fmicb.2023.1331233},       
    DOI={10.3389/fmicb.2023.1331233},      
    ISSN={1664-302X}
}