Update README.md
Browse files
README.md
CHANGED
@@ -12,9 +12,48 @@ For a detailed example of usage, refer to [this notebook](https://github.com/app
|
|
12 |
2. Pop it off the memory to gain VRAM.
|
13 |
3. Loads the InstructPix2Pix pipeline and performs editing.
|
14 |
|
15 |
-
💡 MGIE needs additional set up steps that are important to follow before running inference. Please refer to the
|
|
|
|
|
16 |
|
17 |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
18 |
## Citation
|
19 |
|
20 |
```
|
|
|
12 |
2. Pop it off the memory to gain VRAM.
|
13 |
3. Loads the InstructPix2Pix pipeline and performs editing.
|
14 |
|
15 |
+
💡 MGIE needs additional set up steps that are important to follow before running inference. Please refer to the
|
16 |
+
repository for those instructions. Importantly, it needs you to merge the LLaVA weight deltas with
|
17 |
+
the original LLaMA parameters. More details are in the repository.
|
18 |
|
19 |
|
20 |
+
## Processing ultra high-resolution images
|
21 |
+
|
22 |
+
Since the [InstructPi2xPi2x pipeline](https://huggingface.co/docs/diffusers/main/en/api/pipelines/pix2pix) doesn't do any internal processing
|
23 |
+
to resize the input images, you might get OOMs when processing ultra high-resolution images
|
24 |
+
like [this one](https://i.imgur.com/CiAbKbS.jpg).
|
25 |
+
|
26 |
+
So, it's recommended to resize them, preserving their aspect-ratio. Here's a utility function that can be leveraged here:
|
27 |
+
|
28 |
+
```python
|
29 |
+
from diffusers.utils import load_image
|
30 |
+
|
31 |
+
def resize_image_aspect_ratio(img_url, base_width=None, base_height=None):
|
32 |
+
# Load the image
|
33 |
+
img = load_image(img_url).convert("RGB")
|
34 |
+
|
35 |
+
# Get the current width and height of the image
|
36 |
+
width, height = img.size
|
37 |
+
|
38 |
+
# Calculate the new dimensions based on the aspect ratio
|
39 |
+
if base_width is not None:
|
40 |
+
# Calculate new height based on the base_width to maintain aspect ratio
|
41 |
+
w_percent = (base_width / float(width))
|
42 |
+
h_size = int((float(height) * float(w_percent)))
|
43 |
+
new_size = (base_width, h_size)
|
44 |
+
elif base_height is not None:
|
45 |
+
# Calculate new width based on the base_height to maintain aspect ratio
|
46 |
+
h_percent = (base_height / float(height))
|
47 |
+
w_size = int((float(width) * float(h_percent)))
|
48 |
+
new_size = (w_size, base_height)
|
49 |
+
else:
|
50 |
+
raise ValueError("Either base_width or base_height must be provided")
|
51 |
+
|
52 |
+
# Resize the image
|
53 |
+
resized_img = img.resize(new_size, Image.ANTIALIAS)
|
54 |
+
return resized_img
|
55 |
+
```
|
56 |
+
|
57 |
## Citation
|
58 |
|
59 |
```
|