WAVE-models
Description
WAVE models are generative models designed to decode natural images from fMRI data. We utilize contrastive learning and diffusion models, trained using the PyTorch library. This suite includes both universal models trained across all subjects and subject-specific models, particularly for the CSI1 subject within the BOLD5000 dataset. Pre-trained models and additional resources are also available.
Model Structure
Universal Model
- All: A universal model trained on data from all subjects.
Subject-Specific Model
- CSI1: A model specifically trained on the CSI1 subject.
Model Folder Structure
contrastive/: Contains the contrastive learning models.decode/: Contains the diffusion models.
Model Files
*/contrastive/model_last_prompt.bin: A model integrating contrastive learning with prompt.*/contrastive/model_last_vis-mask.bin: A model integrating contrastive learning with prompt and visual network masking.*/decode/model_last_prompt_vd.bin: Versatile diffusion model fine-tuned on the contrastive prompt learning model.*/decode/model_last_prompt_vis-mask_vd.bin: Versatile diffusion model fine-tuned on the contrastive prompt and visual network masking model.
Pre-trained fMRI Model
Available at:
upstream/GPT/pytorch_model.bin: Pre-trained GPT model for fMRI data from this GitHub repository.
Additional Resources
DiFuMo Atlas
labels_1024_dictionary.csv: Contains labels for the DiFuMo atlas.
Yeo 7 Atlas
index_yeo7.json: Labels for DiFuMo atlas indices associated with visual networks.
For further details on the models and applications, refer to the GitHub repository.