---
license: creativeml-openrail-m
base_model: stabilityai/stable-diffusion-xl-base-1.0
dataset: None
tags:
  - stable-diffusion-xl
  - stable-diffusion-xl-diffusers
  - text-to-image
  - diffusers
  - lora
inference: true
---

# LoRA text2image fine-tuning - ra100/sdxl-lora-lower-decks-aesthetic

These are LoRA adaption weights for stabilityai/stable-diffusion-xl-base-1.0. The weights were fine-tuned on the None dataset.

Tokens to use:

- `stld aesthetic` - Star Trek Lower Decks style
- `blue and gray stldunf uniform` - to see uniform
- `nacelles`, `deflector`, `saucer` - to have more Federation style ships

Images below generated with prompt:

> a female in a blue and gray stldunf uniform standing in woods next to a lake, stld aesthetic

You can find some example images in the following.

    "a female in a blue and gray stldunf uniform standing in woods next to a lake, stld aesthetic"

![img_0](./image_0.png)

    "man in yellow and gray stldunf uniform sitting, on spaceship, stld aesthetic"

![img_1](./image_1.png)

    "a cat, stld aesthetic"

![img_2](./image_2.png)

    "red planet with green moon, space scenery, stars in background, nebulae, stld aesthetic"

![img_3](./image_3.png)

    "Luna Lovegood in a red and gray stldunf uniform standing in scifi sickbay, stld aesthetic"

![img_4](./image_4.png)

    "space ship with nacelles, saucer, deflector, floating in space above planet, stars, stld aestehtic"

![img_5](./image_5.png)

LoRA for the text encoder was enabled: True.

Special VAE used for training: madebyollin/sdxl-vae-fp16-fix.

Training command:

```shell
export MODEL_NAME="stabilityai/stable-diffusion-xl-base-1.0"
export OUTPUT_DIR="stld-aesthetic/model47-grd16-1e4-fp16"
export MODEL_VAE="madebyollin/sdxl-vae-fp16-fix"
export DATASET_DIR="stld-aesthetic/img47"

export PYTORCH_CUDA_ALLOC_CONF=max_split_size_mb:512

accelerate launch diffusers/examples/text_to_image/train_text_to_image_lora_sdxl.py \
  --output_dir=$OUTPUT_DIR \
  --pretrained_model_name_or_path=$MODEL_NAME  \
  --pretrained_vae_model_name_or_path=$MODEL_VAE \
  --train_data_dir=$DATASET_DIR \
  --hub_model_id="ra100/sdxl-lora-lower-decks-aesthetic" \
  --checkpointing_steps=1000 \
  --checkpoints_total_limit=10 \
  --learning_rate=1e-4 \
  --lr_scheduler="constant" \
  --mixed_precision="fp16" \
  --lr_warmup_steps=0 \
  --max_train_steps=4000 \
  --num_validation_images=4 \
  --resolution=1024 \
  --seed="46158136" \
  --train_batch_size=1 \
  --train_text_encoder \
  --report_to="wandb" \
  --validation_epochs=30 \
  --gradient_accumulation_steps=16 \
  --gradient_checkpointing \
  --allow_tf32 \
  --validation_prompt="a female in a blue and gray stldunf uniform standing in woods next to a lake, stld aesthetic"
```