h94

/

IP-Adapter

like 1.03k

@h94 any ideas?

@MonsterMMORPG you can make a issue at InvokeAI github repo？

@MonsterMMORPG you can make a issue at InvokeAI github repo？

i just did : https://github.com/invoke-ai/InvokeAI/issues/5012

same issue?

RuntimeError: Error(s) in loading state_dict for Resampler:
Nov 04 20:42:21 ip-172-31-7-28.us-west-1.compute.internal startup.sh[59971]: size mismatch for proj_in.weight: copying a param with shape torch.Size([1280, 1280]) from checkpoint, the shape in current model is torch.Size([1280, 1664]).

hi, you should follow https://huggingface.co/h94/IP-Adapter#ip-adapter-for-sdxl-10, and use the right image encoder model.

i tested on auto1111 controlnet the bin worked. safetensor failed

@h94 after using the right image encoder model (sd15) I am getting this error:

File "/home/ec2-user/.local/lib/python3.9/site-packages/diffusers/configuration_utils.py", line 137, in getattr
Nov 05 14:49:32 ip-172-31-14-127.us-west-1.compute.internal startup.sh[185884]: raise AttributeError(f"'{type(self).name}' object has no attribute '{name}'")
Nov 05 14:49:32 ip-172-31-14-127.us-west-1.compute.internal startup.sh[185884]: AttributeError: 'StableDiffusionXLCustomPipeline' object has no attribute 'maybe_free_model_hooks'

I am using:

from ip_adapter import IPAdapterPlusXL
from ip_adapter.custom_pipelines import StableDiffusionXLCustomPipeline

base_model_path = "stabilityai/stable-diffusion-xl-base-1.0"
ip_ckpt = "ip-adapter-plus-face_sdxl_vit-h.bin"

pipe = StableDiffusionXLCustomPipeline.from_pretrained(
base_model_path,
torch_dtype=torch.float16,
add_watermarker=False,
)
ip_model = IPAdapterPlusXL(pipe, image_encoder_path, ip_ckpt, accelerator.device, num_tokens=16)

@h94 after using the right image encoder model (sd15) I am getting this error:

File "/home/ec2-user/.local/lib/python3.9/site-packages/diffusers/configuration_utils.py", line 137, in getattr
Nov 05 14:49:32 ip-172-31-14-127.us-west-1.compute.internal startup.sh[185884]: raise AttributeError(f"'{type(self).name}' object has no attribute '{name}'")
Nov 05 14:49:32 ip-172-31-14-127.us-west-1.compute.internal startup.sh[185884]: AttributeError: 'StableDiffusionXLCustomPipeline' object has no attribute 'maybe_free_model_hooks'

I am using:

from ip_adapter import IPAdapterPlusXL
from ip_adapter.custom_pipelines import StableDiffusionXLCustomPipeline

base_model_path = "stabilityai/stable-diffusion-xl-base-1.0"
ip_ckpt = "ip-adapter-plus-face_sdxl_vit-h.bin"

pipe = StableDiffusionXLCustomPipeline.from_pretrained(
base_model_path,
torch_dtype=torch.float16,
add_watermarker=False,
)
ip_model = IPAdapterPlusXL(pipe, image_encoder_path, ip_ckpt, accelerator.device, num_tokens=16)

update the diffusers?

@h94 in this example: https://github.com/tencent-ailab/IP-Adapter/blob/main/ip_adapter_sdxl_plus-face_demo.ipynb

you have this line: from diffusers import StableDiffusionXLPipeline

but StableDiffusionXLPipeline is not used.

@h94 in this example: https://github.com/tencent-ailab/IP-Adapter/blob/main/ip_adapter_sdxl_plus-face_demo.ipynb

you have this line: from diffusers import StableDiffusionXLPipeline

but StableDiffusionXLPipeline is not used.

yes, I use a custom pipeline