metadata

license: other
base_model: black-forest-labs/FLUX.1-dev
tags:
  - flux
  - flux-diffusers
  - text-to-image
  - diffusers
  - simpletuner
  - safe-for-work
  - lora
  - template:sd-lora
  - lycoris
inference: true
widget:
  - text: unconditional (blank prompt)
    parameters:
      negative_prompt: blurry, cropped, ugly
    output:
      url: ./assets/image_0_0.png
  - text: >-
      A scene from One Piece. Nami stands at the edge of a cliff, wind blowing
      through her hair, with the ocean far below. She's wearing a purple tank
      top and white skirt, her eyes full of determination. Sanji Vinsmoke is
      behind her, looking away, leaning on a tree with a cigarette in hand. He's
      dressed in his usual black suit with a blue dress shirt and black tie. The
      setting sun casts a golden glow over the rocky terrain and crashing waves
      below.
    parameters:
      negative_prompt: blurry, cropped, ugly
    output:
      url: ./assets/image_1_0.png
  - text: >-
      A scene from Jujutsu Kaisen. Maki Zenin is standing on a beach during
      sunset holding a sign that says 'I love prompts'.
    parameters:
      negative_prompt: blurry, cropped, ugly
    output:
      url: ./assets/image_2_0.png
  - text: >-
      A scene from Demon Slayer. Muzan Kibutsuji is standing in front of a large
      chalkboard filled with cryptic equations and diagrams. He's wearing a
      white fedora, a white suit with a purple shirt underneath, and white
      gloves. He holds a piece of chalk in one hand and a test tube in the
      other, his eyes gleaming with focus. The warm light from nearby candles
      flickers against the shelves of mysterious bottles and ingredients in his
      lab, giving the scene an eerie ambiance.
    parameters:
      negative_prompt: blurry, cropped, ugly
    output:
      url: ./assets/image_3_0.png
  - text: >-
      A scene from Attack on Titan. Levi Ackerman is sitting under a tree,
      holding the piece of paper close to his face as he reads intently
    parameters:
      negative_prompt: blurry, cropped, ugly
    output:
      url: ./assets/image_4_0.png
  - text: >-
      A scene from My Hero Academia. Shoto Todoroki stands on a cliff
      overlooking a vast desert, his hand outstretched and half of the cliffside
      covered in ice. He's wearing his hero costume: a dark blue jacket with
      elbow-length sleeves, matching pants, a silver-colored combat vest, and
      white boots. His outfit contrasts against the bright, sandy terrain. The
      harsh sunlight creates long shadows across the rocky landscape, adding to
      the intensity of the moment.
    parameters:
      negative_prompt: blurry, cropped, ugly
    output:
      url: ./assets/image_5_0.png
  - text: >-
      A scene from Demon Slayer. Tanjiro Kamado stands beside him, holding his
      sword at the ready. He's wearing his Demon Slayer uniform: a dark jacket
      with gold trim over a white shirt, along with his signature green and
      black checkered haori fluttering in the wind. Shoto Todoroki stands in a
      dense forest, with a serious expression. He's wearing his hero costume: a
      dark blue jacket with elbow-length sleeves, matching pants, a
      silver-colored combat vest, and white boots. His ice powers cover part of
      the forest floor.
    parameters:
      negative_prompt: blurry, cropped, ugly
    output:
      url: ./assets/image_6_0.png
  - text: >-
      A scene from Attack on Titan. Mikasa Ackerman watches in disbelief. She's
      wearing her Survey Corps uniform: a short, light brown jacket with the
      Wings of Freedom insignia, white pants, and dark brown boots. Her military
      uniform flutters in the wind as she stands ready with her ODM gear. Monkey
      D. Luffy is climbing the massive walls of Paradis Island, laughing as he
      pulls himself higher. He's wearing his classic outfit: a red vest, blue
      shorts, and sandals, with his straw hat secured around his neck. The sky
      is clear, with just a few clouds drifting above, and the vast landscape
      stretches out below them.
    parameters:
      negative_prompt: blurry, cropped, ugly
    output:
      url: ./assets/image_7_0.png
  - text: >-
      A scene from My Hero Academia. Makima stands in the middle of a quiet
      street, holding a test tube filled with a strange glowing liquid. She's
      wearing her usual attire: a white button-up shirt, black tie, and dark
      pants, with her distinctive red hair tied back in a braid. Shoto Todoroki
      stands next to her, his hand raised as he is waving. He's in his hero
      costume: a dark blue jacket with elbow-length sleeves, matching pants, a
      silver-colored combat vest, and white boots. The street is eerily still,
      with the remains of a destroyed building in the background, and the sun is
      setting, casting long shadows across the scene.
    parameters:
      negative_prompt: blurry, cropped, ugly
    output:
      url: ./assets/image_8_0.png
  - text: >-
      A scene from One Piece. Levi Ackerman is on the deck of the Thousand
      Sunny, methodically cleaning his gear with a stern expression. He's
      wearing his Survey Corps uniform: a short, light brown jacket with the
      Wings of Freedom insignia, white pants, and dark brown boots. Nami stands
      nearby, glancing over at him with a mix of curiosity and amusement, her
      arms crossed. She's wearing a blue and white striped bikini top, low-rise
      jeans, and high-heeled sandals. The ship sails through calm waters under a
      bright blue sky, with a distant island just visible on the horizon.
    parameters:
      negative_prompt: blurry, cropped, ugly
    output:
      url: ./assets/image_9_0.png
  - text: >-
      A scene from My Hero Academia. Shoto Todoroki stands in the middle of a
      training field, facing off against an opponent. Shoto Todoroki is in his
      hero costume: a dark blue jacket with elbow-length sleeves, matching
      pants, a silver-colored combat vest, and white boots. His Opponent, Ichigo
      Kurosaki stands opposite him, he is wearing his Shinigami robes: a black
      kimono and hakama, with a white sash around his waist. He holds his sword
      with a confident expression. The sky above is cloudy, with faint rays of
      sunlight piercing through.
    parameters:
      negative_prompt: blurry, cropped, ugly
    output:
      url: ./assets/image_10_0.png

anime-chars-simpletuner-lora-1

This is a LyCORIS adapter derived from black-forest-labs/FLUX.1-dev.

No validation prompt was used during training.

None

Validation settings

CFG: 3.5
CFG Rescale: 0.0
Steps: 20
Sampler: None
Seed: 42
Resolution: 1536x864

Note: The validation settings are not necessarily the same as the training settings.

You can find some example images in the following gallery:

Prompt
unconditional (blank prompt)

Negative Prompt
blurry, cropped, ugly

Prompt
A scene from One Piece. Nami stands at the edge of a cliff, wind blowing through her hair, with the ocean far below. She's wearing a purple tank top and white skirt, her eyes full of determination. Sanji Vinsmoke is behind her, looking away, leaning on a tree with a cigarette in hand. He's dressed in his usual black suit with a blue dress shirt and black tie. The setting sun casts a golden glow over the rocky terrain and crashing waves below.

Negative Prompt
blurry, cropped, ugly

Prompt
A scene from Jujutsu Kaisen. Maki Zenin is standing on a beach during sunset holding a sign that says 'I love prompts'.

Negative Prompt
blurry, cropped, ugly

Prompt
A scene from Demon Slayer. Muzan Kibutsuji is standing in front of a large chalkboard filled with cryptic equations and diagrams. He's wearing a white fedora, a white suit with a purple shirt underneath, and white gloves. He holds a piece of chalk in one hand and a test tube in the other, his eyes gleaming with focus. The warm light from nearby candles flickers against the shelves of mysterious bottles and ingredients in his lab, giving the scene an eerie ambiance.

Negative Prompt
blurry, cropped, ugly

Prompt
A scene from Attack on Titan. Levi Ackerman is sitting under a tree, holding the piece of paper close to his face as he reads intently

Negative Prompt
blurry, cropped, ugly

Prompt
A scene from My Hero Academia. Shoto Todoroki stands on a cliff overlooking a vast desert, his hand outstretched and half of the cliffside covered in ice. He's wearing his hero costume: a dark blue jacket with elbow-length sleeves, matching pants, a silver-colored combat vest, and white boots. His outfit contrasts against the bright, sandy terrain. The harsh sunlight creates long shadows across the rocky landscape, adding to the intensity of the moment.

Negative Prompt
blurry, cropped, ugly

Prompt
A scene from Demon Slayer. Tanjiro Kamado stands beside him, holding his sword at the ready. He's wearing his Demon Slayer uniform: a dark jacket with gold trim over a white shirt, along with his signature green and black checkered haori fluttering in the wind. Shoto Todoroki stands in a dense forest, with a serious expression. He's wearing his hero costume: a dark blue jacket with elbow-length sleeves, matching pants, a silver-colored combat vest, and white boots. His ice powers cover part of the forest floor.

Negative Prompt
blurry, cropped, ugly

Prompt
A scene from Attack on Titan. Mikasa Ackerman watches in disbelief. She's wearing her Survey Corps uniform: a short, light brown jacket with the Wings of Freedom insignia, white pants, and dark brown boots. Her military uniform flutters in the wind as she stands ready with her ODM gear. Monkey D. Luffy is climbing the massive walls of Paradis Island, laughing as he pulls himself higher. He's wearing his classic outfit: a red vest, blue shorts, and sandals, with his straw hat secured around his neck. The sky is clear, with just a few clouds drifting above, and the vast landscape stretches out below them.

Negative Prompt
blurry, cropped, ugly

Prompt
A scene from My Hero Academia. Makima stands in the middle of a quiet street, holding a test tube filled with a strange glowing liquid. She's wearing her usual attire: a white button-up shirt, black tie, and dark pants, with her distinctive red hair tied back in a braid. Shoto Todoroki stands next to her, his hand raised as he is waving. He's in his hero costume: a dark blue jacket with elbow-length sleeves, matching pants, a silver-colored combat vest, and white boots. The street is eerily still, with the remains of a destroyed building in the background, and the sun is setting, casting long shadows across the scene.

Negative Prompt
blurry, cropped, ugly

Prompt
A scene from One Piece. Levi Ackerman is on the deck of the Thousand Sunny, methodically cleaning his gear with a stern expression. He's wearing his Survey Corps uniform: a short, light brown jacket with the Wings of Freedom insignia, white pants, and dark brown boots. Nami stands nearby, glancing over at him with a mix of curiosity and amusement, her arms crossed. She's wearing a blue and white striped bikini top, low-rise jeans, and high-heeled sandals. The ship sails through calm waters under a bright blue sky, with a distant island just visible on the horizon.

Negative Prompt
blurry, cropped, ugly

Prompt
A scene from My Hero Academia. Shoto Todoroki stands in the middle of a training field, facing off against an opponent. Shoto Todoroki is in his hero costume: a dark blue jacket with elbow-length sleeves, matching pants, a silver-colored combat vest, and white boots. His Opponent, Ichigo Kurosaki stands opposite him, he is wearing his Shinigami robes: a black kimono and hakama, with a white sash around his waist. He holds his sword with a confident expression. The sky above is cloudy, with faint rays of sunlight piercing through.

Negative Prompt
blurry, cropped, ugly

The text encoder was not trained. You may reuse the base model text encoder for inference.

Training settings

Training epochs: 382
Training steps: 25200
Learning rate: 0.001
Effective batch size: 160
- Micro-batch size: 40
- Gradient accumulation steps: 1
- Number of GPUs: 4
Prediction type: flow-matching
Rescaled betas zero SNR: False
Optimizer: adamw_bf16
Precision: Pure BF16
Quantised: Yes: int8-quanto
Xformers: Not used
LyCORIS Config:

{
    "algo": "lokr",
    "multiplier": 1.0,
    "linear_dim": 10000,
    "linear_alpha": 1,
    "factor": 12,
    "apply_preset": {
        "target_module": [
            "Attention",
            "FeedForward"
        ],
        "module_algo_map": {
            "Attention": {
                "factor": 12
            },
            "FeedForward": {
                "factor": 6
            }
        }
    }
}

Datasets

anime_char-512

Repeats: 2
Total number of images: ~3476
Total number of aspect buckets: 1
Resolution: 0.262144 megapixels
Cropped: False
Crop style: None
Crop aspect: None

Inference

import torch
from diffusers import DiffusionPipeline
from lycoris import create_lycoris_from_weights

model_id = 'black-forest-labs/FLUX.1-dev'
adapter_id = 'pytorch_lora_weights.safetensors' # you will have to download this manually
lora_scale = 1.0
wrapper, _ = create_lycoris_from_weights(lora_scale, adapter_id, pipeline.transformer)
wrapper.merge_to()

prompt = "An astronaut is riding a horse through the jungles of Thailand."

pipeline.to('cuda' if torch.cuda.is_available() else 'mps' if torch.backends.mps.is_available() else 'cpu')
image = pipeline(
    prompt=prompt,
    num_inference_steps=20,
    generator=torch.Generator(device='cuda' if torch.cuda.is_available() else 'mps' if torch.backends.mps.is_available() else 'cpu').manual_seed(1641421826),
    width=1536,
    height=864,
    guidance_scale=3.5,
).images[0]
image.save("output.png", format="PNG")