File size: 2,549 Bytes
47780ca
 
 
 
 
3c7feee
 
47780ca
 
 
 
 
6d737eb
3c7feee
 
 
 
 
 
 
 
 
 
7e5ea3f
 
 
 
3c7feee
 
 
 
 
 
 
 
6d737eb
 
 
 
3c7feee
 
 
 
6d737eb
3c7feee
6d737eb
 
 
3c7feee
 
6d737eb
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
3c7feee
 
 
 
 
 
 
 
 
 
 
 
 
 
6d737eb
 
 
 
 
 
 
3c7feee
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
---
title: Opdmulti Demo
emoji: 🌍
colorFrom: gray
colorTo: red
sdk: docker
app_port: 7860
pinned: false
license: mit
---

Check out the configuration reference at https://huggingface.co/docs/hub/spaces-config-reference

## Installation

## Requirements

For the docker build, you will just need docker in order to build and run the container, else you will need

* python 3.10
* git
* libosmesa6-dev (for open3d headless rendering)

**BEFORE BUILDING** as of writing, you will need to copy the model file manually to `.data/models/motion_state_pred_opdformerp_rgb.pth`
in the repository. This step must occur before the docker build, or if building locally then before running. Future
work will make this step no longer required.

### Docker (preferred)

To build the docker container, run
```
docker build -f Dockerfile -t opdmulti-demo .
```

### Local

To setup the environment, run the following (recommended in a virtual environment):
```
# install base requirements
python3.10 -m pip install -r requirements.txt

# install detectron2 (must be done after some of the libraries in requirements.txt)
python3.10 -m pip install git+https://github.com/facebookresearch/detectron2.git@fc9c33b1f6e5d4c37bbb46dde19af41afc1ddb2a

# build library for model
cd mask2former/modeling/pixel_decoder/ops
python setup.py build install

# INSTALL OPEN3D
# --------------
# Option A: running locally only
pip install open3d==0.17.0

# Option B: running over ssh connection / headless environment
# in a separate folder
git clone https://github.com/isl-org/Open3D.git
cd Open3D/
mkdir build && cd build
cmake -DENABLE_HEADLESS_RENDERING=ON -DBUILD_GUI=OFF -DBUILD_WEBRTC=OFF -DUSE_SYSTEM_GLEW=OFF -DUSE_SYSTEM_GLFW=OFF ..
make -j$(nproc)
make install-pip-package
# to test custom build
cd ../examples/python/visualization/
python headless_rendering.py
```

## Usage

### Docker (preferred)

To run the docker container, execute
```
docker run -d --network host -t opdmulti-demo
```
If you want to see the output of the container or interact with it,

* use `-it` to run in interactive mode, and remove the `-d` option
* add `bash` to the end to open into a console rather than running the app directly

### Local

To startup the application locally, run
```
gradio app.py
```

You can view the app on the specified port (usually 7860). To run over an ssh connection, setup port forwarding using
`-L 7860:localhost:7860` when you create your ssh connection. Note that you will need to install Open3D in headless 
rendering for this to work, as described above.