Using this in the diffusers library
#12
by
kchoi
- opened
Is there a way to use this text encoder as part of diffusers library if we want to generate an image?
I tried passing it as a tokenizer in StableDiffusionPipeline.from_pretrained("runwayml/stable-diffusion-v1-5", tokenizer=tokenizer) but it's expecting a tokenizer of type transformers.tokenization_utils.PreTrainedTokenizer instead of open_clip.tokenizer.HFTokenizer which is what i get if i load BiomedCLIP using tokenizer = open_clip.get_tokenizer('hf-hub:microsoft/BiomedCLIP-PubMedBERT_256-vit_base_patch16_224')
kchoi
changed discussion title from
Using this clip in diffusers library
to Using this in the diffusers library