
# Color-Canny CantrolNet

These are ControlNet checkpoints trained on runwayml/stable-diffusion-v1-5, using fused color and canny edge as conditioning. 

You can find some example images in the following. 

## Examples

#### Color examples

**prompt**: a concept art of by Makoto Shinkai, a girl is standing in the middle of the sea

**negative prompt**: text, bad anatomy, blurry, (low quality, blurry)
![images_1)](./1.png)

**prompt**: a concept art of by Makoto Shinkai, a girl is standing in the middle of the sea

**negative prompt**: text, bad anatomy, blurry, (low quality, blurry)
![images_2)](./2.png)

**prompt**: a concept art of by Makoto Shinkai, a girl is standing in the middle of the grass

**negative prompt**: text, bad anatomy, blurry, (low quality, blurry)
![images_3)](./3.png)


#### Brightness examples
This model also can be used to control image brightness. The following images are generated with different brightness conditioning image and controlnet strength(0.5 ~ 0.7).
![images_4)](./4.jpg)


## Limitations and Bias

- No strict control by input color
- Sometimes generate image with confusion When color description in prompt

## Training

**Dataset**
We train this model on [laion-art](https://huggingface.co/datasets/laion/laion-art) dataset with 2.6m images, the processed dataset can be found in [ghoskno/laion-art-en-colorcanny](https://huggingface.co/datasets/ghoskno/laion-art-en-colorcanny).


**Training Details**

- **Hardware**: Google Cloud TPUv4-8 VM

- **Optimizer**: AdamW

- **Train Batch Size**: 4 x 4 = 16

- **Learning rate**: 0.00001 constant

- **Gradient Accumulation Steps**: 4

- **Resolution**: 512

- **Train Steps**: 36000
{"language": ["en"], "license": "apache-2.0", "library_name": "diffusers", "tags": ["jax-diffusers-event"], "datasets": ["laion/laion-art"], "pipeline_tag": "image-to-image", "base_model": "runwayml/stable-diffusion-v1-5"}
diffusers jax-diffusers-event image-to-image en dataset:laion/laion-art base_model:runwayml/stable-diffusion-v1-5 license:apache-2.0 has_space diffusers:ControlNetModel region:us
TAGS
#diffusers #jax-diffusers-event #image-to-image #en #dataset-laion/laion-art #base_model-runwayml/stable-diffusion-v1-5 #license-apache-2.0 #has_space #diffusers-ControlNetModel #region-us 


# Color-Canny CantrolNet

These are ControlNet checkpoints trained on runwayml/stable-diffusion-v1-5, using fused color and canny edge as conditioning. 

You can find some example images in the following. 

## Examples

#### Color examples

prompt: a concept art of by Makoto Shinkai, a girl is standing in the middle of the sea

negative prompt: text, bad anatomy, blurry, (low quality, blurry)
!images_1)

prompt: a concept art of by Makoto Shinkai, a girl is standing in the middle of the sea

negative prompt: text, bad anatomy, blurry, (low quality, blurry)
!images_2)

prompt: a concept art of by Makoto Shinkai, a girl is standing in the middle of the grass

negative prompt: text, bad anatomy, blurry, (low quality, blurry)
!images_3)


#### Brightness examples
This model also can be used to control image brightness. The following images are generated with different brightness conditioning image and controlnet strength(0.5 ~ 0.7).
!images_4)


## Limitations and Bias

- No strict control by input color
- Sometimes generate image with confusion When color description in prompt

## Training

Dataset
We train this model on laion-art dataset with 2.6m images, the processed dataset can be found in ghoskno/laion-art-en-colorcanny.


Training Details

- Hardware: Google Cloud TPUv4-8 VM

- Optimizer: AdamW

- Train Batch Size: 4 x 4 = 16

- Learning rate: 0.00001 constant

- Gradient Accumulation Steps: 4

- Resolution: 512

- Train Steps: 36000
# Color-Canny CantrolNet

These are ControlNet checkpoints trained on runwayml/stable-diffusion-v1-5, using fused color and canny edge as conditioning. 

You can find some example images in the following. ## Examples #### Color examples

prompt: a concept art of by Makoto Shinkai, a girl is standing in the middle of the sea

negative prompt: text, bad anatomy, blurry, (low quality, blurry)
!images_1)

prompt: a concept art of by Makoto Shinkai, a girl is standing in the middle of the sea

negative prompt: text, bad anatomy, blurry, (low quality, blurry)
!images_2)

prompt: a concept art of by Makoto Shinkai, a girl is standing in the middle of the grass

negative prompt: text, bad anatomy, blurry, (low quality, blurry)
!images_3) #### Brightness examples
This model also can be used to control image brightness. The following images are generated with different brightness conditioning image and controlnet strength(0.5 ~ 0.7).
!images_4) ## Limitations and Bias

- No strict control by input color
- Sometimes generate image with confusion When color description in prompt ## Training

Dataset
We train this model on laion-art dataset with 2.6m images, the processed dataset can be found in ghoskno/laion-art-en-colorcanny.


Training Details

- Hardware: Google Cloud TPUv4-8 VM

- Optimizer: AdamW

- Train Batch Size: 4 x 4 = 16

- Learning rate: 0.00001 constant

- Gradient Accumulation Steps: 4

- Resolution: 512

- Train Steps: 36000
TAGS
#diffusers #jax-diffusers-event #image-to-image #en #dataset-laion/laion-art #base_model-runwayml/stable-diffusion-v1-5 #license-apache-2.0 #has_space #diffusers-ControlNetModel #region-us 
 # Color-Canny CantrolNet

These are ControlNet checkpoints trained on runwayml/stable-diffusion-v1-5, using fused color and canny edge as conditioning. 

You can find some example images in the following. ## Examples #### Color examples

prompt: a concept art of by Makoto Shinkai, a girl is standing in the middle of the sea

negative prompt: text, bad anatomy, blurry, (low quality, blurry)
!images_1)

prompt: a concept art of by Makoto Shinkai, a girl is standing in the middle of the sea

negative prompt: text, bad anatomy, blurry, (low quality, blurry)
!images_2)

prompt: a concept art of by Makoto Shinkai, a girl is standing in the middle of the grass

negative prompt: text, bad anatomy, blurry, (low quality, blurry)
!images_3) #### Brightness examples
This model also can be used to control image brightness. The following images are generated with different brightness conditioning image and controlnet strength(0.5 ~ 0.7).
!images_4) ## Limitations and Bias

- No strict control by input color
- Sometimes generate image with confusion When color description in prompt ## Training

Dataset
We train this model on laion-art dataset with 2.6m images, the processed dataset can be found in ghoskno/laion-art-en-colorcanny.


Training Details

- Hardware: Google Cloud TPUv4-8 VM

- Optimizer: AdamW

- Train Batch Size: 4 x 4 = 16

- Learning rate: 0.00001 constant

- Gradient Accumulation Steps: 4

- Resolution: 512

- Train Steps: 36000
TAGS
#diffusers #jax-diffusers-event #image-to-image #en #dataset-laion/laion-art #base_model-runwayml/stable-diffusion-v1-5 #license-apache-2.0 #has_space #diffusers-ControlNetModel #region-us 
# Color-Canny CantrolNet

These are ControlNet checkpoints trained on runwayml/stable-diffusion-v1-5, using fused color and canny edge as conditioning. 

You can find some example images in the following.## Examples#### Color examples

prompt: a concept art of by Makoto Shinkai, a girl is standing in the middle of the sea

negative prompt: text, bad anatomy, blurry, (low quality, blurry)
!images_1)

prompt: a concept art of by Makoto Shinkai, a girl is standing in the middle of the sea

negative prompt: text, bad anatomy, blurry, (low quality, blurry)
!images_2)

prompt: a concept art of by Makoto Shinkai, a girl is standing in the middle of the grass

negative prompt: text, bad anatomy, blurry, (low quality, blurry)
!images_3)#### Brightness examples
This model also can be used to control image brightness. The following images are generated with different brightness conditioning image and controlnet strength(0.5 ~ 0.7).
!images_4)## Limitations and Bias

- No strict control by input color
- Sometimes generate image with confusion When color description in prompt## Training

Dataset
We train this model on laion-art dataset with 2.6m images, the processed dataset can be found in ghoskno/laion-art-en-colorcanny.


Training Details

- Hardware: Google Cloud TPUv4-8 VM

- Optimizer: AdamW

- Train Batch Size: 4 x 4 = 16

- Learning rate: 0.00001 constant

- Gradient Accumulation Steps: 4

- Resolution: 512

- Train Steps: 36000