Update README.md
Browse files
README.md
CHANGED
@@ -2,22 +2,44 @@
|
|
2 |
license: apache-2.0
|
3 |
---
|
4 |
# LaMa Inpainting Model
|
5 |
-
This is an alpha version of ONNX model ported from original PyTorch big-lama model.
|
6 |
|
7 |
-
|
8 |
-
1. We use a custom FourierUnit implementation, as the original cannot be directly ported to ONNX without overhead. The result is not the same, but very close
|
9 |
-
2. The input shape is fixed to 512x512 pixels.
|
10 |
-
3. The opset version is 18.
|
11 |
-
4. The model works slowly due to issues with torch.onnx.dynamo_export with optimization of the onnx model.
|
12 |
|
|
|
13 |
|
14 |
-
|
15 |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
16 |
|
17 |
## Example
|
18 |
-
|
19 |
-
|
20 |
-
|
21 |
-
|
22 |
-
|
23 |
-
![](./
|
|
|
|
|
|
|
|
2 |
license: apache-2.0
|
3 |
---
|
4 |
# LaMa Inpainting Model
|
|
|
5 |
|
6 |
+
This ONNX model is a port of the original PyTorch big-lama model.
|
|
|
|
|
|
|
|
|
7 |
|
8 |
+
## Description
|
9 |
|
10 |
+
There are two versions of the model:
|
11 |
|
12 |
+
### 1. `lama_fp32.onnx` (RECOMMENDED)
|
13 |
+
|
14 |
+
This version was exported using the old torch to ONNX converter (`torch.onnx.export`).
|
15 |
+
|
16 |
+
**Notes:**
|
17 |
+
1. **Custom FourierUnitJIT**: A custom [FourierUnitJIT](https://github.com/Carve-Photos/lama/blob/main/saicinpainting/training/modules/ffc.py) implementation is used since the original cannot be directly ported to ONNX without overhead. The result is identical to the original model.
|
18 |
+
2. **Fixed Input Shape**: The input shape is fixed at 1024x1024 pixels. Although dynamic input shapes are possible, they would require resolving issues with dynamic padding in the `irfft` and `rfftn` functions in `ffc.py`.
|
19 |
+
3. **Opset Version 17**: This model uses opset version 17.
|
20 |
+
|
21 |
+
### 2. `lama.onnx` (NOT RECOMMENDED)
|
22 |
+
|
23 |
+
This version was exported using the new torch to ONNX converter (`torch.onnx.dynamo_export`).
|
24 |
+
|
25 |
+
**Notes:**
|
26 |
+
1. **Custom DFT irfftn Logic**: Uses a custom irfftn ONNX logic (patched `onnxscript`).
|
27 |
+
2. **Fixed Input Shape**: The input shape is fixed at 512x512 pixels.
|
28 |
+
3. **Opset Version 18**: This model uses opset version 18.
|
29 |
+
4. **Performance**: The model works slowly due to issues with `torch.onnx.dynamo_export` and optimization of the ONNX model.
|
30 |
+
|
31 |
+
## Resources
|
32 |
+
|
33 |
+
- Original repository: [advimman/lama](https://github.com/advimman/lama)
|
34 |
+
- Repository with custom implementation of exportable LaMa: [Carve-Photos/lama](https://github.com/Carve-Photos/lama)
|
35 |
|
36 |
## Example
|
37 |
+
|
38 |
+
**Original image:**
|
39 |
+
![original image](./image.jpg)
|
40 |
+
|
41 |
+
**ONNX output:**
|
42 |
+
![onnx output](./output_onnx.png)
|
43 |
+
|
44 |
+
**Original model output:**
|
45 |
+
![original model output](./output_orig.png)
|