FastGlioma: foundation models for fast, label-free detection of glioma infiltration

Paper / Interactive Demo / GitHub / MLiNS Lab

Model card for our paper 'Foundation models for fast, label-free detection of glioma infiltration.' We employ a foundational model training strategy to predict the degree of diffuse glioma infiltration intraoperatively using stimulated Raman histology and deep learning.

Model Details

• Patch Encoder: ResNet-34 (21.4M parameters). Batch size of 512 distributed over 4x Nvidia A40s for 800 epochs with the self-supervised hierarchical discriminative (HiDisc) objective, base learn rate was 0.001.
• WSI Encoder, Foundation Model: Transformer (6.6M parameters). Batch size of 256 with 1 Nvidia Titan V100 for 100 epochs with the self-supervised VICReg objective, base learn rate was 0.0003.
• WSI Encoder, Supervised Finetuning: Batch size of 16 with 1 Nvidia Titan V100 for 100 epochs with the supervised ordinal metric learning objective, base learn rate was 0.0000188.

Learn rate scheduler was cosine decay with 10% warmup for all stages. Mixed precision and data parallelism were used when applicable.

Model Checkpoints

The checkpoints provided are designed to work with our inference package on the OpenSRH dataset. Note that FastGlioma was only designed for patients with adult-type diffuse gliomas. Please refer here for an in-depth description on FastGlioma's intended use.

fastglioma_highres_model.ckpt takes in FullSRH images (3 channels, no skipped rows) and has the best performance, whereas fastglioma_lowres_model.ckpt takes in FastSRH images (1 channel, every 5th row selected; >10x faster).

License Information

FastGlioma models and the OpenSRH dataset are licensed under the CC-BY-NC-SA 4.0 License. The associated code, found in our GitHub repository, is licensed under the MIT License.