# ControlLoRA text2image fine-tuning - Official Model Repository
These are ControlLoRA adaption weights for [runwayml/stable-diffusion-v1-5](https://huggingface.co/runwayml/stable-diffusion-v1-5). You can find the code repository in [HighCWu/ControlLoRA](https://github.com/HighCWu/ControlLoRA).

With ControlLoRA, a simple and small (~7M parameters, ~25M storage space) network, you could control the spatial information of stable diffusion.

Each of the weights is fine-tuned on the [diffusiondb_2m_first_5k_canny](https://huggingface.co/datasets/HighCWu/diffusiondb_2m_first_5k_canny) and [mpii_100_openpose](https://huggingface.co/datasets/HighCWu/mpii_100_openpose) datasets with 30k steps on RTX4080 in 3 hours. 

You can find some example images in the following. 

![img_{i}](./docs/imgs/validation_25100_fe0b49075b45efdc311c.png)
*boys are playing with a frisbee in a field, 
2009 cinematography, trending on artforum, running pose, 
bruce springsteen, connected to heart machines, with tattoos, 
beautiful - n 9, by Eric Dinyer, young child, midlands*

![img_{i}](./docs/imgs/validation_25000_6b3f8b37f2268ec0c09e.png)
*portrait of a dancing eagle woman, "
"beautiful blonde haired lakota sioux goddess, "
"intricate, highly detailed art by james jean, "
"ray tracing, digital painting, artstation, "
"concept art, smooth, sharp focus, illustration, "
"artgerm and greg rutkowski and alphonse mucha, "
"vladimir kush, giger, roger dean, 8 k*

I also uploaded a lora model fine-tuned on my selfies with 2k steps which could be use in the experiment of mixing LoRA and ControlLoRA.

![img_{i}](./docs/imgs/7.png)
*portrait of female HighCWu as a cute pink hair girl*

{"license": "creativeml-openrail-m", "tags": ["stable-diffusion", "stable-diffusion-diffusers", "text-to-image", "diffusers", "lora", "controlnet", "control-lora"], "base_model": "runwayml/stable-diffusion-v1-5", "inference": true}
diffusers safetensors stable-diffusion stable-diffusion-diffusers text-to-image lora controlnet control-lora base_model:runwayml/stable-diffusion-v1-5 license:creativeml-openrail-m has_space region:us
TAGS
#diffusers #safetensors #stable-diffusion #stable-diffusion-diffusers #text-to-image #lora #controlnet #control-lora #base_model-runwayml/stable-diffusion-v1-5 #license-creativeml-openrail-m #has_space #region-us 

# ControlLoRA text2image fine-tuning - Official Model Repository
These are ControlLoRA adaption weights for runwayml/stable-diffusion-v1-5. You can find the code repository in HighCWu/ControlLoRA.

With ControlLoRA, a simple and small (~7M parameters, ~25M storage space) network, you could control the spatial information of stable diffusion.

Each of the weights is fine-tuned on the diffusiondb_2m_first_5k_canny and mpii_100_openpose datasets with 30k steps on RTX4080 in 3 hours. 

You can find some example images in the following. 

!img_{i}
*boys are playing with a frisbee in a field, 
2009 cinematography, trending on artforum, running pose, 
bruce springsteen, connected to heart machines, with tattoos, 
beautiful - n 9, by Eric Dinyer, young child, midlands*

!img_{i}
*portrait of a dancing eagle woman, "
"beautiful blonde haired lakota sioux goddess, "
"intricate, highly detailed art by james jean, "
"ray tracing, digital painting, artstation, "
"concept art, smooth, sharp focus, illustration, "
"artgerm and greg rutkowski and alphonse mucha, "
"vladimir kush, giger, roger dean, 8 k*

I also uploaded a lora model fine-tuned on my selfies with 2k steps which could be use in the experiment of mixing LoRA and ControlLoRA.

!img_{i}
*portrait of female HighCWu as a cute pink hair girl*

# ControlLoRA text2image fine-tuning - Official Model Repository
These are ControlLoRA adaption weights for runwayml/stable-diffusion-v1-5. You can find the code repository in HighCWu/ControlLoRA.

With ControlLoRA, a simple and small (~7M parameters, ~25M storage space) network, you could control the spatial information of stable diffusion.

Each of the weights is fine-tuned on the diffusiondb_2m_first_5k_canny and mpii_100_openpose datasets with 30k steps on RTX4080 in 3 hours. 

You can find some example images in the following. 

!img_{i}
*boys are playing with a frisbee in a field, 
2009 cinematography, trending on artforum, running pose, 
bruce springsteen, connected to heart machines, with tattoos, 
beautiful - n 9, by Eric Dinyer, young child, midlands*

!img_{i}
*portrait of a dancing eagle woman, "
"beautiful blonde haired lakota sioux goddess, "
"intricate, highly detailed art by james jean, "
"ray tracing, digital painting, artstation, "
"concept art, smooth, sharp focus, illustration, "
"artgerm and greg rutkowski and alphonse mucha, "
"vladimir kush, giger, roger dean, 8 k*

I also uploaded a lora model fine-tuned on my selfies with 2k steps which could be use in the experiment of mixing LoRA and ControlLoRA.

!img_{i}
*portrait of female HighCWu as a cute pink hair girl*
TAGS
#diffusers #safetensors #stable-diffusion #stable-diffusion-diffusers #text-to-image #lora #controlnet #control-lora #base_model-runwayml/stable-diffusion-v1-5 #license-creativeml-openrail-m #has_space #region-us 
 # ControlLoRA text2image fine-tuning - Official Model Repository
These are ControlLoRA adaption weights for runwayml/stable-diffusion-v1-5. You can find the code repository in HighCWu/ControlLoRA.

With ControlLoRA, a simple and small (~7M parameters, ~25M storage space) network, you could control the spatial information of stable diffusion.

Each of the weights is fine-tuned on the diffusiondb_2m_first_5k_canny and mpii_100_openpose datasets with 30k steps on RTX4080 in 3 hours. 

You can find some example images in the following. 

!img_{i}
*boys are playing with a frisbee in a field, 
2009 cinematography, trending on artforum, running pose, 
bruce springsteen, connected to heart machines, with tattoos, 
beautiful - n 9, by Eric Dinyer, young child, midlands*

!img_{i}
*portrait of a dancing eagle woman, "
"beautiful blonde haired lakota sioux goddess, "
"intricate, highly detailed art by james jean, "
"ray tracing, digital painting, artstation, "
"concept art, smooth, sharp focus, illustration, "
"artgerm and greg rutkowski and alphonse mucha, "
"vladimir kush, giger, roger dean, 8 k*

I also uploaded a lora model fine-tuned on my selfies with 2k steps which could be use in the experiment of mixing LoRA and ControlLoRA.

!img_{i}
*portrait of female HighCWu as a cute pink hair girl*
TAGS
#diffusers #safetensors #stable-diffusion #stable-diffusion-diffusers #text-to-image #lora #controlnet #control-lora #base_model-runwayml/stable-diffusion-v1-5 #license-creativeml-openrail-m #has_space #region-us 
# ControlLoRA text2image fine-tuning - Official Model Repository
These are ControlLoRA adaption weights for runwayml/stable-diffusion-v1-5. You can find the code repository in HighCWu/ControlLoRA.

With ControlLoRA, a simple and small (~7M parameters, ~25M storage space) network, you could control the spatial information of stable diffusion.

Each of the weights is fine-tuned on the diffusiondb_2m_first_5k_canny and mpii_100_openpose datasets with 30k steps on RTX4080 in 3 hours. 

You can find some example images in the following. 

!img_{i}
*boys are playing with a frisbee in a field, 
2009 cinematography, trending on artforum, running pose, 
bruce springsteen, connected to heart machines, with tattoos, 
beautiful - n 9, by Eric Dinyer, young child, midlands*

!img_{i}
*portrait of a dancing eagle woman, "
"beautiful blonde haired lakota sioux goddess, "
"intricate, highly detailed art by james jean, "
"ray tracing, digital painting, artstation, "
"concept art, smooth, sharp focus, illustration, "
"artgerm and greg rutkowski and alphonse mucha, "
"vladimir kush, giger, roger dean, 8 k*

I also uploaded a lora model fine-tuned on my selfies with 2k steps which could be use in the experiment of mixing LoRA and ControlLoRA.

!img_{i}
*portrait of female HighCWu as a cute pink hair girl*