metadata

language:
  - en
metrics:
  - dice-score
  - area-under-the-receiver-operating-characteristic
pipeline_tag: video-classification
tags:
  - medical
  - deep-learning
datasets:
  - arcade-dataset
  - mhi-clinical-database
library_name: pytorch
model-index:
  - name: deepcoro-segmentation-algorithm
    results:
      - task:
          type: image-segmentation
        dataset:
          name: arcade-dataset
          type: arcade-dataset
        metrics:
          - name: dice-score
            type: area-under-the-receiver-operating-characteristic
            value: 72.93
        source:
          name: npj Digital Medicine
          url: https://www.nature.com/articles/s41746-024-01134-4
  - name: deepcoro-stenosis-prediction-algorithm
    results:
      - task:
          type: video-classification
        dataset:
          name: mhi-clinical-database
          type: mhi-clinical-database
        metrics:
          - name: area-under-the-receiver-operating-characteristic
            type: area-under-the-receiver-operating-characteristic
            value: 0.8294
        source:
          name: npj Digital Medicine
          url: https://www.nature.com/articles/s41746-024-01134-4

Model Card for DeepCoro

Model Details

Model Description

DeepCoro is a deep learning model designed to run inference on DICOM coronary angiography videos. The model identifies stenoses, their locations, associated coronary artery segments, and their associetated percentage. This model repository contains the weights for DeepCoro's segmentation algorithm (Algorithm 4) and stenosis percentage prediction algorithm (Algorithm 6). Models for the primary anatomic struction classification (Algorithm 1) and stenosis detection (Algorithm 2) could not be made public.

Developed by: HeartWise AI
Model type: Deep learning supervised model for coronary angiography analysis
Language(s) (NLP): Not applicable
License: See LICENSE agreement
Finetuned from model: Algorithm 6 was fine-tuned from Torchvision's Swin3D base model

Model Sources

Repository: DeepCoro GitHub
Paper: Evaluation of Stenoses Using AI Video Models Applied to Coronary Angiographies
Demo: Supplementary Video 1 in the Supplementary Information of the paper

Uses

Direct Use

DeepCoro is only suitable for use on DICOM format coronary angiographies to predict stenosis percentage, which meet the following requirements:

Contain information about:
- Imager Pixel Spacing
- Distance Source To Detector
- Distance Source To Patient
- Recommended Display Frame Rate
- Study Date
- Patient's Birth Date
Display the right coronary arteries (RCA) or the left coronary arteries (LCA).
Be aquired at 15 frames per seconds.

Downstream Use

DeepCoro’s adaptability was successfully demonstrated by fine-tuning the model on quantitative coronary angiography (QCA) data, which further reduced the mean absolute error in stenosis percentage prediction. This capability suggests that the model can be extended to other quantitative assessment tasks in coronary angiography imaging.

Out-of-Scope Use

The model is not suitable for use on non-DICOM formats data or videos that do not meet the specified requirements. The model is not intended for use outside of clinical settings or by individuals without appropriate medical training. It should not be used for diagnosing other conditions unrelated to coronary artery stenosis.

Bias, Risks, and Limitations

DeepCoro's primary limitation is that the stenosis percentages for training and testing are based on clinician interpretation from CAG videos, which may not always align with actual stenosis values. Additionally, DeepCoro is optimized for detecting stenoses in 11 specific coronary segments, excluding side branches and CABG patients. The current average processing time of 62.60 seconds per DICOM video may pose a challenge in clinical settings where effectiveness is crucial. Moreover, running inference with the DeepCoro repository requires manual identification of coronary arteries and stenoses in an input CSV file, introducing potential human error. The model is in its development phase, and therefore should not be used for official diagnois.

Recommendations

Users should be aware of the risks, biases, and limitations of the model. To mitigate the limitations identified in the DeepCoro model, it is recommended to exclude videos involving PCI and patients with previous CABG to enhance model precision. Users may calibrate the model using QCA assessments to further refine accuracy and reduce variability. Fine-tuning the model on diverse datasets can improve its versatility and reliability. For further recommendations, refer to the repository documentation.

How to Get Started with the Model

Use the code below to run inference with DeepCoro:

Clone the DeepCoro repository:

git clone https://github.com/HeartWise-AI/DeepCoro.git

Place your input files in the dcm_input/ folder.
Build the Docker environment:
```
docker build -t deepcoro_inference .
```

Run inference:

On CPU:

docker run -v /path/to/dcm_input:/dcm_input -v /path/to/results:/results deepcoro_inference

On GPU:

docker run --gpus all -v /path/to/dcm_input:/dcm_input -v /path/to/results:/results deepcoro_inference

Training Details

Training Data

The training data for Algorithm 4 was derived from the ARCADE dataset, which included X-ray coronary angiography images and their associated manual segmentation for each coronary artery segment. This dataset comprised 1200 images, covering a wide range of anatomical variations and stenosis conditions in the LCA and RCA. The ARCADE dataset provided a rich source of training material, with the training set making up approximately 66% of the total dataset. Data augmentation techniques were applied to enhance the model’s robustness and ability to generalize.

For Algorithm 6, the training data came from the Montreal Heart Institute (MHI) database which contained 182,418 coronary angiography videos and their associated percentage of stenosis, visually assessed by cardiologists for every coronary artery segment. From this large dataset, 44,139 stenosis videos from 8057 patients were derived for training and validation purposes. The training dataset included approximately 75% of these videos, offering a diverse range of cases, from mild to severe stenosis, ensuring a comprehensive training experience. This dataset was crucial for developing the model’s ability to accurately detect and quantify stenosis across different coronary artery segments.

Training Procedure

Algorithm 4 was developed through extensive testing over 200 epochs, involving nine state-of-the-art segmentation models: UNet, UNet++, MANet, LinkNet, FPN, PSPNet, DeepLabV3, DeepLabV3+, and PAN. The training process explored a variety of loss functions, including Jaccard loss, Dice loss, focal loss, Lovasz loss, and Tversky loss. Hyperparameter optimization was conducted using random search, considering batch sizes of 4, 8, 16, 32, and 64, and learning rates ranging from 1e−2 to 1e−5. The optimization process employed 5-fold cross-validation, ensuring consistent evaluation by using the same folds across all sets of hyperparameters.

The training procedure for Algorithm 6 involved fine-tuning the Swin3D model, originally pre-trained on Kinetics400 and ImageNetV1. The model was adapted for regression tasks by modifying the activation function to Sigmoid and adjusting the output size. The training included a learning rate schedule, starting at 0.001 and adjusting based on the validation set performance, using the mean square error loss function and stochastic gradient descent optimizer. Hyperparameter tuning for Algorithm 6 was performed using learning rates of 0.00001, 0.0001, and 0.001 to minimize the validation loss. Batch sizes were chosen to maximize memory usage on the available GPU.

Preprocessing

For preprocessing, stenosis videos are resized to 17.5mm x 17.5mm and registered. This involved aligning each frame with a reference frame to maintain the segment of interest's alignment across all frames within each stenosis box video. Before inputting the videos into Algorithm 6, they were resized to 224x224 pixels with interpolation and 24 frames centered around the reference area were used.

Training Hyperparameters

Final hyperparameters for Algorithm 6 were a learning rate of 0.001 and a batch size of 4, chosen to maximize memory usage on the GPU.

Seepds, Sizes, Times

The end-to-end processing time for DeepCoro, from inputting the video to obtaining the final analysis, is approximately 62.60 seconds in average. The sizes of the avalable model weights for DeepCoro are :

Algorithm 4 (7 models): 104, 92.8, 104, 92.8, 86.1, 90 and 86.1 MB
Algorithm 6 (1 model): 731 MB

Evaluation

Testing Data, Factors & Metrics

Testing Data

The testing dataset for Algorithm 4, also from the ARCADE dataset, consisted of 200 images, representing 17% of the total dataset. This testing set was separate from the training set and included a diverse array of images. The testing data provided a balanced representation of different anatomical structures and conditions, allowing for an accurate assessment of the model’s generalization performance.

The testing data for Algorithm 6 was obtained from the MHI database. A subset of 5904 videos (15% of the total dataset) was used for testing, ensuring a representative evaluation of the model’s performance in real-world scenarios. This testing set included a wide range of stenosis severities and anatomical variations, providing a comprehensive assessment of the model’s ability to generalize beyond the training data.

Factors

For Algorithm 4, metrics were disaggregated by coronary artery segment. Evaluation metrics for Algorithm 6 were disaggregated by coronary artery (LCA or RCA).

Metrics

For Aglorithm 4, evaluation metrics included the Dice score and the predictive positive value (PPV). For Algorithm 6, evaluation metrics included mean absolute error (MAE) for regression, and the area under the teceiver operating curve (AUROC) for classification.

Results

The results for Algorithm 4, focused on segmentation, are evaluated using the Dice score and PPV across different coronary artery segments. These metrics provide insights into the accuracy and reliability of the segmentation model in delineating the anatomical structures of interest. The segmentation algorithm achieved notable performance, with variations across different segments, reflecting the inherent challenges in segmenting complex anatomical structures.

The following table presents the Dice score and PPV for each coronary artery segment:

Coronary Artery Segment	Dice Score (%)	PPV (%)
left main artery	77.84	86.48
proximal left anterior descending artery	70.89	68.70
middle left anterior descending artery	74.20	76.53
distal left anterior descending artery	71.40	74.13
proximal left circumflex artery	75.14	74.35
distal left circumflex artery	60.39	61.15
proximal right coronary artery	80.33	80.92
middle right coronary artery	72.58	78.87
distal right coronary artery	77.26	76.92
posterolateral branch from the right coronary artery	63.08	82.44
posterior descending artery	74.11	77.02
weighted average	72.93	75.96

Algorithm 6: Stenosis Detection

For Algorithm 6, the evaluation was based on MAE and AUROC for the RCA, LCA, and combined RCA+LCA artery. These metrics highlight the algorithm's effectiveness in detecting and quantifying stenosis severity.

The MAE and AUROC values for the stenosis detection algorithm are summarized in the table below:

Coronary Artery	MAE (%)	AUROC
RCA	17.82	0.8643
LCA	22.19	0.8017
RCA+LCA	20.15	0.8294

Environmental Impact

To train and test our Algorithm 4 and 6 models, we used:

Hardware Type: RTX3090
Hours used: 200h
Cloud Provider: Private
Compute Region: Canada
Carbon Emitted: 2.1 kg of CO2 (computed with Machine Learning Impact calculator)

Technical Specifications

Model Architecture and Objective

DeepCoro is an advanced AI-based pipeline designed for automatic evaluation of coronary angiography videos, focusing on the severity assessment of stenoses in the RCA and LCA. The pipeline integrates six specialized algorithms:

Primary Anatomic Structure Detection Algorithm: Uses an Xception model to identify primary anatomic structures in CAG videos.
Stenosis Detection Algorithm: Utilizes the RetinaNet architecture to locate coronary artery segments and stenoses.
Registration Algorithm: Aligns stenosis bounding boxes across video frames, accounting for cardiac motion.
Segmentation Algorithm: Applies an ensemble of seven models to segment videos, identifying 11 epicardial coronary artery segments.
Coronary Artery Stenosis Assignment to Segment Algorithm: Assigns detected stenosis to specific coronary artery segments.
Stenosis Percentage Prediction Algorithm: Uses a modified Swin3D transformer model to predict the severity of stenoses in percentage terms.

Compute Infrastructure

Training and inference were conducted on high-performance GPUs with extensive memory and computational capabilities.

Hardware

High-performance RTX3090 GPUs with a 48GB memory.

Software

The model was developed using PyTorch and other supporting libraries for video processing and deep learning.

Citation

BibTeX:

@article{avram2023evaluation,
  title={Evaluation of Stenoses Using AI Video Models Applied to Coronary Angiographies},
  author={Labrecque Langlais, E. and Corbin, D. and Tastet, O. and Hayek, A. and Doolub, G. and Mrad, S. and Tardif, J.-C. and Tanguay, J.-F. and Marquis-Gravel, G. and Tison, G. and Kadoury, S. and Le, W. and Gallo, R. and Lesage, F. and Avram, R.},
  year={2023}
}

APA: Labrecque Langlais, E., Corbin, D., Tastet, O., Hayek, A., Doolub, G., Mrad, S., Tardif, J.-C., Tanguay, J.-F., Marquis-Gravel, G., Tison, G., Kadoury, S., Le, W., Gallo, R., Lesage, F., & Avram, R. (2023). Evaluation of Stenoses Using AI Video Models Applied to Coronary Angiographies. PREPRINT (Version 1) available at Research Square. https://doi.org/10.21203/rs.3.rs-3610879/v1

Glossary

AUROC: Area Under the Receiver Operating Characteristic Curve, a performance measurement for classification problems at various threshold settings.
CABG: Coronary Artery Bypass Grafting, a surgical procedure to restore normal blood flow to an obstructed coronary artery.
CAG: Coronary Angiography, a procedure that uses a special dye and X-rays to see how blood flows through the coronary arteries.
Dice score: A statistical tool to gauge the similarity between two sets of data, often used to measure the accuracy of image segmentation models.
DICOM: Digital Imaging and Communications in Medicine, a standard for handling, storing, printing, and transmitting information in medical imaging.
LCA: Left Coronary Artery, teh other main artery that supplies blood to the heart.
MAE: Mean Absolute Error, a measure of errors between paired continuous observations to describe regression performance.
PCI: Percutaneous Coronary Intervention, a non-surgical procedure used to treat narrowing of the coronary arteries of the heart found in coronary artery disease.
PPV: Predictive Positive Value, a measure used to assess the precision of a model, indicating the proportion of positive results that are true positives.
QCA: Quantitative Coronary Angiography, a method to quantitatively measure coronary artery dimensions and the degree of stenosis.
RCA: Right Coronary Artery, one of the two main arteries that supply blood to the heart.
Stenosis: The narrowing of blood vessels.

More Information

For more details, refer to the published paper and supplementary materials.

Model Card Authors

Élodie Labrecque Langlais, Denis Corbin, Olivier Tastet, Ahmad Hayek, Gemina Doolub, Sebastián Mrad, Jean-Claude Tardif, Jean-François Tanguay, Guillaume Marquis-Gravel, Geoffrey H. Tison, Samuel Kadoury, William Le, Richard Gallo, Frederic Lesage, Robert Avram

Model Card Contact

robert.avram.md@gmail.com

Reference

Labrecque Langlais, E., Corbin, D., Tastet, O., Hayek, A., Doolub, G., Mrad, S., Tardif, J.-C., Tanguay, J.-F., Marquis-Gravel, G., Tison, G., Kadoury, S., Le, W., Gallo, R., Lesage, F, & Avram, R. Evaluation of Stenoses Using AI Video Models Applied to Coronary Angiographies, 23 November 2023, PREPRINT (Version 1) available at Research Square [https://doi.org/10.21203/rs.3.rs-3610879/v1]

@article{avram2023evaluation,
  title={Evaluation of Stenoses Using AI Video Models Applied to Coronary Angiographies},
  author={Labrecque Langlais, E. and Corbin, D. and Tastet, O. and Hayek, A. and Doolub, G. and Mrad, S. and Tardif, J.-C. and Tanguay, J.-F. and Marquis-Gravel, G. and Tison, G. and Kadoury, S. and Le, W. and Gallo, R. and Lesage, F. and Avram, R.},
  year={2023}
}