---
base_model: stabilityai/stable-diffusion-xl-base-1.0
library_name: diffusers
license: creativeml-openrail-m
tags:
- stable-diffusion-xl
- stable-diffusion-xl-diffusers
- text-to-image
- diffusers
- diffusers-training
- lora
inference: true
datasets:
- AdamLucek/jarekl-photos
pipeline_tag: text-to-image
---
# LoRA Weights for sdxl-base-1.0 Tuned on Jarek Lucek's Photos
These are LoRA adaption weights for [stabilityai/stable-diffusion-xl-base-1.0](https://huggingface.co/stabilityai/stable-diffusion-xl-base-1.0). The weights were fine-tuned on the [AdamLucek/jarekl-photos dataset](https://huggingface.co/datasets/AdamLucek/jarekl-photos).
Explicit permissions given for training from [Jarek Lucek](https://www.instagram.com/jarekl_foto/?hl=en), more information in dataset.
**Example Images:**
*"Evening cityscape highlighting the interplay of light on traditional architecture, stairs, and fencing, with a distant human figure for scale."*
*"Night scene of a quiet urban street, featuring architectural details of a prominent building and a single person in the foreground."*
*"An old church stands tall in the darkness, its white walls and steeple glowing under streetlights, with a solitary pedestrian passing by."*
## Intended uses & limitations
#### Intended uses
These LoRA weights are designed solely for educational and research purposes, intended to replicate the aesthetic and technique characteristic of Jarek Lucek's photography style. By using this model, you acknowledge that the generated images are simulations and do not reflect the original works directly.
The images produced by this model are **NOT** authorized for commercial use. They may not be sold, licensed, or otherwise exploited for any commercial purpose without explicit permission from Jarek Lucek.
Users must attribute the generated images directly to Jarek Lucek. Users are advised to ensure their use complies with applicable laws and respects the artistic rights of the original creator.
By downloading or using this model, you agree to abide by these terms and accept full responsibility for the use of the generated images.
#### How to use
Importing SDXL with LoRA & Refiner
```python
from diffusers import DiffusionPipeline
import torch
base_sdxl = DiffusionPipeline.from_pretrained(
"stabilityai/stable-diffusion-xl-base-1.0",
torch_dtype=torch.float16,
variant="fp16",
use_safetensors=True
).to("cuda")
base_sdxl.enable_model_cpu_offload()
base_sdxl.load_lora_weights("AdamLucek/sdxl-base-1.0-jarekl-lora")
refiner = DiffusionPipeline.from_pretrained(
"stabilityai/stable-diffusion-xl-refiner-1.0",
text_encoder_2=base_sdxl.text_encoder_2,
vae=base_sdxl.vae,
torch_dtype=torch.float16,
use_safetensors=True,
variant="fp16",
).to("cuda")
refiner.enable_model_cpu_offload()
```
Generating an Image
```python
prompt = "Evening cityscape highlighting the interplay of light on traditional architecture, stairs, and fencing, with a distant human figure for scale."
image = base_sdxl(
prompt=prompt,
num_inference_steps=50,
denoising_end=0.8,
output_type="latent",
).images
image = refiner(
prompt=prompt,
num_inference_steps=50,
denoising_start=0.8,
image=image,
).images[0]
image.save("cityscape.png")
```
#### Limitations and bias
**Note:** First pass on LoRA training, not refined or tuned specifically.
**Note:** Hyperparameters are not very scientifically chosen as this is a first attempt.
## Training details
Trained using a single a10 using [diffusers package](https://github.com/huggingface/diffusers/blob/main/examples/text_to_image/train_text_to_image_lora_sdxl.py), documentation available [here](https://huggingface.co/docs/diffusers/main/en/training/lora).
Training Script:
```
accelerate launch train_text_to_image_lora_sdxl.py \
--pretrained_model_name_or_path=stabilityai/stable-diffusion-xl-base-1.0 \
--dataset_name=AdamLucek/jarekl-photos \
--output_dir=output/sdxl-base-1.0-jarekl-lora \
--resolution=1024 \
--train_batch_size=1 \
--dataloader_num_workers=8 \
--gradient_accumulation_steps=4 \
--max_train_steps=2000 \
--num_train_epochs=50 \
--learning_rate=1e-04 \
--random_flip \
--max_grad_norm=1 \
--lr_scheduler="cosine" \
--lr_warmup_steps=100 \
--use_8bit_adam \
--allow_tf32 \
--checkpointing_steps=100 \
--validation_prompt="Black and white photograph of a narrow cobblestone street at night. A lone figure stands in the foreground, looking into a shop window. The street is lined with old buildings, their textures emphasized by the monochrome image. In the background, a church spire is visible against the night sky. Street lamps cast light pools on the cobblestones, creating a moody scene." \
--num_validation_images=4 \
--validation_epochs=1 \
--mixed_precision="fp16" \
--seed=66 \
--push_to_hub \
--hub_model_id="sdxl-base-1.0-jarekl-lora"
```