---
pipeline_tag: image-feature-extraction
license: mit
tags:
- embodied-ai
- representation-learning
- spatial awareness
- spatial intelligence
---

# Model Card for SPA: 3D Spatial-Awareness Enables Effective Embodied Representation

<!-- Provide a quick summary of what the model is/does. -->

Pre-trained checkpoints of [SPA](https://haoyizhu.github.io/spa/).

SPA is a novel representation learning framework that emphasizes the importance of 3D spatial awareness in embodied AI. 
It leverages differentiable neural rendering on multi-view images to endow a vanilla Vision Transformer (ViT) with 
intrinsic spatial understanding. We also present the most comprehensive evaluation of embodied representation learning to date, 
covering 268 tasks across 8 simulators with diverse policies in both single-task and language-conditioned multi-task scenarios.

## Model Details

### Model Description

<!-- Provide a longer summary of what this model is. -->



- **Developed by:** [Haoyi Zhu](https://www.haoyizhu.site/)
- **Model type:** Embodied AI Representation Learning
- **Encoder (Backbone) type:** Vision Transformer (ViT)

### Model Sources [optional]

<!-- Provide the basic links for the model. -->

- **Repository:** [https://github.com/HaoyiZhu/SPA](https://github.com/HaoyiZhu/SPA)
- **Paper:** [Arxiv (Coming Soon)]()
- **Project Page:** [https://haoyizhu.github.io/spa/](https://haoyizhu.github.io/spa/)

## Citation
```bib
@article{zhu2024spa,
    title = {SPA: 3D Spatial-Awareness Enables Effective Embodied Representation},
    author = {Zhu, Haoyi and and Yang, Honghui and Wang, Yating and Yang, Jiange and Wang, Limin and He, Tong},
    journal = {arXiv preprint},
    year = {2024},
}
```