I got this error message with Diffusers:: TypeError: empty(): argument 'size' failed to unpack the object at pos 2 with error "type must be tuple of ints,but got NoneType".
Hello,
I got this error message with Diffusers:: TypeError: empty(): argument 'size' failed to unpack the object at pos 2 with error "type must be tuple of ints,but got NoneType".
Is there a mistake on my side?
Thank you! Stable Diffusion and Diffusers are great!
I updated the Diffusers.
pip install -U diffusers
Requirement already satisfied: diffusers in /home/omimex/.pyenv/versions/3.11.10/lib/python3.11/site-packages (0.31.0)
Requirement already satisfied: importlib-metadata in /home/omimex/.pyenv/versions/3.11.10/lib/python3.11/site-packages (from diffusers) (8.5.0)
Requirement already satisfied: filelock in /home/omimex/.pyenv/versions/3.11.10/lib/python3.11/site-packages (from diffusers) (3.13.1)
Requirement already satisfied: huggingface-hub>=0.23.2 in /home/omimex/.pyenv/versions/3.11.10/lib/python3.11/site-packages (from diffusers) (0.23.4)
Requirement already satisfied: numpy in /home/omimex/.pyenv/versions/3.11.10/lib/python3.11/site-packages (from diffusers) (1.26.4)
Requirement already satisfied: regex!=2019.12.17 in /home/omimex/.pyenv/versions/3.11.10/lib/python3.11/site-packages (from diffusers) (2024.11.6)
Requirement already satisfied: requests in /home/omimex/.pyenv/versions/3.11.10/lib/python3.11/site-packages (from diffusers) (2.32.3)
Requirement already satisfied: safetensors>=0.3.1 in /home/omimex/.pyenv/versions/3.11.10/lib/python3.11/site-packages (from diffusers) (0.4.5)
Requirement already satisfied: Pillow in /home/omimex/.pyenv/versions/3.11.10/lib/python3.11/site-packages (from diffusers) (10.2.0)
Requirement already satisfied: fsspec>=2023.5.0 in /home/omimex/.pyenv/versions/3.11.10/lib/python3.11/site-packages (from huggingface-hub>=0.23.2->diffusers) (2024.2.0)
Requirement already satisfied: packaging>=20.9 in /home/omimex/.pyenv/versions/3.11.10/lib/python3.11/site-packages (from huggingface-hub>=0.23.2->diffusers) (24.2)
Requirement already satisfied: pyyaml>=5.1 in /home/omimex/.pyenv/versions/3.11.10/lib/python3.11/site-packages (from huggingface-hub>=0.23.2->diffusers) (6.0.2)
Requirement already satisfied: tqdm>=4.42.1 in /home/omimex/.pyenv/versions/3.11.10/lib/python3.11/site-packages (from huggingface-hub>=0.23.2->diffusers) (4.67.0)
Requirement already satisfied: typing-extensions>=3.7.4.3 in /home/omimex/.pyenv/versions/3.11.10/lib/python3.11/site-packages (from huggingface-hub>=0.23.2->diffusers) (4.9.0)
Requirement already satisfied: zipp>=3.20 in /home/omimex/.pyenv/versions/3.11.10/lib/python3.11/site-packages (from importlib-metadata->diffusers) (3.21.0)
Requirement already satisfied: charset-normalizer<4,>=2 in /home/omimex/.pyenv/versions/3.11.10/lib/python3.11/site-packages (from requests->diffusers) (3.4.0)
Requirement already satisfied: idna<4,>=2.5 in /home/omimex/.pyenv/versions/3.11.10/lib/python3.11/site-packages (from requests->diffusers) (3.10)
Requirement already satisfied: urllib3<3,>=1.21.1 in /home/omimex/.pyenv/versions/3.11.10/lib/python3.11/site-packages (from requests->diffusers) (2.2.3)
Requirement already satisfied: certifi>=2017.4.17 in /home/omimex/.pyenv/versions/3.11.10/lib/python3.11/site-packages (from requests->diffusers) (2024.8.30)
Detailed error message:
The config attributes {'dual_attention_layers': [], 'force_zeros_for_pooled_projection': False, 'pos_embed_type': None, 'qk_norm': None, 'use_pos_embed': False} were passed to SD3ControlNetModel, but are not expected and will be ignored. Please verify your config.json configuration file.
Traceback (most recent call last):
File "/home/omimex/ai_asset_factory/ctrl_imagegen.py", line 18, in
controlnet = SD3ControlNetModel.from_pretrained("stabilityai/stable-diffusion-3.5-large-controlnet-canny", torch_dtype=torch.float16)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/omimex/.pyenv/versions/3.11.10/lib/python3.11/site-packages/huggingface_hub/utils/_validators.py", line 114, in _inner_fn
return fn(*args, **kwargs)
^^^^^^^^^^^^^^^^^^^
File "/home/omimex/.pyenv/versions/3.11.10/lib/python3.11/site-packages/diffusers/models/modeling_utils.py", line 821, in from_pretrained
model = cls.from_config(config, **unused_kwargs)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/omimex/.pyenv/versions/3.11.10/lib/python3.11/site-packages/diffusers/configuration_utils.py", line 260, in from_config
model = cls(**init_dict)
^^^^^^^^^^^^^^^^
File "/home/omimex/.pyenv/versions/3.11.10/lib/python3.11/site-packages/diffusers/configuration_utils.py", line 665, in inner_init
init(self, *args, **init_kwargs)
File "/home/omimex/.pyenv/versions/3.11.10/lib/python3.11/site-packages/diffusers/models/controlnet_sd3.py", line 76, in init
self.context_embedder = nn.Linear(joint_attention_dim, caption_projection_dim)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/omimex/.pyenv/versions/3.11.10/lib/python3.11/site-packages/torch/nn/modules/linear.py", line 106, in init
torch.empty((out_features, in_features), **factory_kwargs)
TypeError: empty(): argument 'size' failed to unpack the object at pos 2 with error "type must be tuple of ints,but got NoneType"
I think pip install -U diffusers is not enough because the Diffusers stable version does not include the new feature... pip install git+https://github.com/huggingface/diffusers helped.
But I have new error, OutOfMemory :( CUDA out of memory. Tried to allocate 68.00 MiB. GPU 0 has a total capacity of 8.00 GiB of which 0 bytes is free. Including non-PyTorch memory, this process has 17179869184.00 GiB memory in use. Of the allocated memory 22.58 GiB is allocated by PyTorch, and 12.61 MiB is reserved by PyTorch but unallocated. If reserved but unallocated memory is large try setting PYTORCH_CUDA_ALLOC_CONF=expandable_segments:True to avoid fragmentation. See documentation for Memory Management (https://pytorch.org/docs/stable/notes/cuda.html#environment-variables)
I tried Quantizing, which I saw here https://huggingface.co/stabilityai/stable-diffusion-3.5-large:
nf4_config = BitsAndBytesConfig(
load_in_4bit=True,
bnb_4bit_quant_type="nf4",
bnb_4bit_compute_dtype=torch.bfloat16
)
model_nf4 = SD3Transformer2DModel.from_pretrained(
"stabilityai/stable-diffusion-3.5-large",
subfolder="transformer",
quantization_config=nf4_config,
torch_dtype=torch.bfloat16
)
controlnet = SD3ControlNetModel.from_pretrained("stabilityai/stable-diffusion-3.5-large-controlnet-canny", transformer=model_nf4, torch_dtype=torch.bfloat16)
pipe = StableDiffusion3ControlNetPipeline.from_pretrained(
"stabilityai/stable-diffusion-3.5-large",
controlnet=controlnet,
transformer=model_nf4,
torch_dtype=torch.bfloat16
).to("cuda")
pipe.image_processor = SD3CannyImageProcessor()
It continued with the steps and completed all the steps, but I got an OutOfMemory :(.
Without Controlnet, stable-diffusion-3.5-large works on my computer. But it might not be enough for controlnet.
Finally I needed to add bnb_4bit_use_double_quant=True in BitsAndBytesConfig and it worked. :)
Summary:
I'm not an expert, I'm just experimenting...
The solution for me: pip install git+https://github.com/huggingface/diffusers, because when I tried the pip install -U diffusers (the stable version) does not include the new feature and needed the main version (maybe not stable...)
And because I dont have enough VRAM, needed for me the Quantizing from here https://huggingface.co/stabilityai/stable-diffusion-3.5-large and the bnb_4bit_use_double_quant=True in the BitsAndBytesConfig.
I read some new tipps for memory usage reduction: https://huggingface.co/docs/diffusers/en/optimization/memory
Maybe this is better than what I found before.