FanqiPu/MonoDGP

This repository hosts the official implementation of MonoDGP: Monocular 3D Object Detection with Decoupled-Query and Geometry-Error Priors based on the excellent work MonoDETR. In this work, we propose a novel transformer-based monocular method called MonoDGP, which adopts geometry errors to correct the projection formula. We also introduce a 2D visual decoder for query initialization and a region segmentation head for feature enhancement.

Installation

1. Clone this project and create a conda environment:

   git clone https://github.com/PuFanqi23/MonoDGP.git
   cd MonoDGP
   
   conda create -n monodgp python=3.8
   conda activate monodgp
   

2. Install pytorch and torchvision matching your CUDA version:

   # For example, We adopt torch 1.9.0+cu111
   pip install torch==1.9.0+cu111 torchvision==0.10.0+cu111 torchaudio==0.9.0 -f https://download.pytorch.org/whl/torch_stable.html
   

3. Install requirements and compile the deformable attention:

   pip install -r requirements.txt
   
   cd lib/models/monodgp/ops/
   bash make.sh
   
   cd ../../../..
   

4. Download KITTI datasets and prepare the directory structure as:

   │MonoDGP/
   ├──...
   │data/kitti/
   ├──ImageSets/
   ├──training/
   │   ├──image_2
   │   ├──label_2
   │   ├──calib
   ├──testing/
   │   ├──image_2
   │   ├──calib
   

   You can also change the data path at "dataset/root_dir" in configs/monodgp.yaml.

Get Started

Train

You can modify the settings of models and training in configs/monodgp.yaml and indicate the GPU in train.sh:

bash train.sh configs/monodgp.yaml > logs/monodgp.log

Test

The best checkpoint will be evaluated as default. You can change it at "tester/checkpoint" in configs/monodgp.yaml:

bash test.sh configs/monodgp.yaml

You can test the inference time on your own device:

python tools/test_runtime.py

Citation

If you find our work useful in your research, please consider giving us a star and citing:

@article{pu2024monodgp,
  title={MonoDGP: Monocular 3D Object Detection with Decoupled-Query and Geometry-Error Priors},
  author={Pu, Fanqi and Wang, Yifan and Deng, Jiru and Yang, Wenming},
  journal={arXiv preprint arXiv:2410.19590},
  year={2024}
}

Acknowlegment

This repo benefits from the excellent work MonoDETR.