Update README.md

README.md

@@ -1,3 +1,34 @@
 ---
 license: apache-2.0
 ---
+
+#### Note: This is mainly only useful if you are writing your own fine tuning script. If you just want to run inference, please visit the PhotoMaker model. [TencentARC/PhotoMaker](https://huggingface.co/TencentARC/PhotoMaker)
+
+These are just chunks of the weights broken out of [TencentARC/PhotoMaker](https://huggingface.co/TencentARC/PhotoMaker) to allow easier fine tuning and loading of the individual pieces.
+The weights here are identical to the original in every other way.
+
+The CLIP vision model can be loaded with 
+
+```python
+image_preprocessor = CLIPImageProcessor.from_pretrained("ostris/photo-maker-face-sdxl")
+clip_vision = CLIPVisionModelWithProjection.from_pretrained(
+    "ostris/photo-maker-face-sdxl",
+    ignore_mismatched_sizes=True
+)
+```
+It will warn about additional weights because the `fuse_model` and `visual_projection_2` are included in the file but not needed for CLIP.
+
+Using the included python file (modified only to handle from pretrained for now), the PhotoMakerIDEncoder can be loaded with 
+
+```python
+id_encoder = PhotoMakerIDEncoder.from_pretrained("ostris/photo-maker-face-sdxl")
+```
+
+The fuse weights are included in the vision encoder, but are also seperated out in `pytorch_fuse_module_weights.safetensors` so they can be loaded seperatly if only fine
+tuning the fuse_module and / or LoRA. 
+
+The LoRA can also be loaded seperatly with standard Diffusers LoRA loading. 
+```python
+pipeline.load_lora_weights("ostris/photo-maker-face-sdxl", adapter_name="photomaker")
+```
+