    
# controlnet- JFoz/dog-cat-pose

Simple controlnet model made as part of the HF JaX/Diffusers community sprint.

These are controlnet weights trained on runwayml/stable-diffusion-v1-5 with pose conditioning generated using the animalpose model of OpenPifPaf. 

Some example images can be found in the following

prompt: a tortoiseshell cat is sitting on a cushion
![images_0)](./images_0.png)
prompt: a yellow dog standing on a lawn
![images_1)](./images_1.png)

Whilst not the dataset used for this model, a smaller dataset with the same
format for conditioning images can be found at https://huggingface.co/datasets/JFoz/dog-poses-controlnet-dataset

The dataset was generated using the code at https://github.com/jfozard/animalpose/tree/f1be80ed29886a1314054b87f2a8944ea98997ac


# Model Card for dog-cat-pose

This is an ControlNet model which allows users to control the pose of a dog or cat. Poses were extracted from images using the animalpose model of OpenPifPaf https://openpifpaf.github.io/intro.html . Skeleton colouring is as shown in the dataset. See also https://huggingface.co/JFoz/dog-pose



# Model Details

## Model Description

<!-- Provide a longer summary of what this model is/does. -->
This is an ControlNet model which allows users to control the pose of a dog or cat. Poses were extracted from images using the animalpose model of OpenPifPaf https://openpifpaf.github.io/intro.html. Skeleton colouring is as shown in the dataset. See also https://huggingface.co/JFoz/dog-pose

- **Developed by:** John Fozard
- **Model type:** Conditional image generation
- **Language(s) (NLP):** en
- **License:** openrail
- **Parent Model:** https://huggingface.co/runwayml/stable-diffusion-v1-5
- **Resources for more information:** 
    - [GitHub Repo](https://github.com/jfozard/animalpose/tree/f1be80ed29886a1314054b87f2a8944ea98997ac)


# Uses

<!-- Address questions around how the model is intended to be used, including the foreseeable users of the model and those affected by the model. -->

## Direct Use

<!-- This section is for the model use without fine-tuning or plugging into a larger ecosystem/app. -->
<!-- If the user enters content, print that. If not, but they enter a task in the list, use that. If neither, say "more info needed." -->

Supply a suitable, potentially incomplete pose along with  a relevant text prompt


## Out-of-Scope Use

<!-- This section addresses misuse, malicious use, and uses that the model will not work well for. -->
<!-- If the user enters content, print that. If not, but they enter a task in the list, use that. If neither, say "more info needed." -->

Generating images of non-animals. We advise retaining the stable diffusion safety filter when using this model.


# Bias, Risks, and Limitations

<!-- This section is meant to convey both technical and sociotechnical limitations. -->

The model is trained on a relatively small dataset, and may be overfit to those images.


## Recommendations

<!-- This section is meant to convey recommendations with respect to the bias, risk, and technical limitations. -->


Maintain careful supervision of model inputs and outputs.


# Training Details

## Training Data

<!-- This should link to a Data Card, perhaps with a short stub of information on what the training data is all about as well as documentation related to data pre-processing or additional filtering. -->

Trained on a subset of Laion-5B using clip retrieval with the prompts &#34;a photo of a (dog/cat) (standing/walking)&#34;

## Training Procedure

<!-- This relates heavily to the Technical Specifications. Content here should link to that section when it is relevant to the training procedure. -->

### Preprocessing

Images were rescaled to 512 along their short edge and centrally cropped. The OpenPifPaf pose-detection model was used to extract poses, which were used to generate conditioning images.




## Compute Infrastructure

TPUv4i



### Software

Flax stable diffusion controlnet pipeline



# Model Card Authors [optional]

<!-- This section provides another layer of transparency and accountability. Whose views is this model card representing? How many voices were included in its construction? Etc. -->

John Fozard


{"license": "creativeml-openrail-m", "library_name": "diffusers", "tags": ["stable-diffusion", "stable-diffusion-diffusers", "image-to-image", "diffusers", "controlnet", "jax-diffusers-event"], "base_model": "runwayml/stable-diffusion-v1-5", "inference": true}
diffusers safetensors stable-diffusion stable-diffusion-diffusers image-to-image controlnet jax-diffusers-event base_model:runwayml/stable-diffusion-v1-5 license:creativeml-openrail-m has_space diffusers:ControlNetModel region:us
TAGS
#diffusers #safetensors #stable-diffusion #stable-diffusion-diffusers #image-to-image #controlnet #jax-diffusers-event #base_model-runwayml/stable-diffusion-v1-5 #license-creativeml-openrail-m #has_space #diffusers-ControlNetModel #region-us 

    
# controlnet- JFoz/dog-cat-pose

Simple controlnet model made as part of the HF JaX/Diffusers community sprint.

These are controlnet weights trained on runwayml/stable-diffusion-v1-5 with pose conditioning generated using the animalpose model of OpenPifPaf. 

Some example images can be found in the following

prompt: a tortoiseshell cat is sitting on a cushion
!images_0)
prompt: a yellow dog standing on a lawn
!images_1)

Whilst not the dataset used for this model, a smaller dataset with the same
format for conditioning images can be found at URL

The dataset was generated using the code at URL


# Model Card for dog-cat-pose

This is an ControlNet model which allows users to control the pose of a dog or cat. Poses were extracted from images using the animalpose model of OpenPifPaf URL . Skeleton colouring is as shown in the dataset. See also URL



# Model Details

## Model Description


This is an ControlNet model which allows users to control the pose of a dog or cat. Poses were extracted from images using the animalpose model of OpenPifPaf URL Skeleton colouring is as shown in the dataset. See also URL

- Developed by: John Fozard
- Model type: Conditional image generation
- Language(s) (NLP): en
- License: openrail
- Parent Model: URL
- Resources for more information: 
    - GitHub Repo


# Uses



## Direct Use




Supply a suitable, potentially incomplete pose along with  a relevant text prompt


## Out-of-Scope Use




Generating images of non-animals. We advise retaining the stable diffusion safety filter when using this model.


# Bias, Risks, and Limitations



The model is trained on a relatively small dataset, and may be overfit to those images.


## Recommendations




Maintain careful supervision of model inputs and outputs.


# Training Details

## Training Data



Trained on a subset of Laion-5B using clip retrieval with the prompts &#34;a photo of a (dog/cat) (standing/walking)&#34;

## Training Procedure



### Preprocessing

Images were rescaled to 512 along their short edge and centrally cropped. The OpenPifPaf pose-detection model was used to extract poses, which were used to generate conditioning images.




## Compute Infrastructure

TPUv4i



### Software

Flax stable diffusion controlnet pipeline



# Model Card Authors [optional]



John Fozard


# controlnet- JFoz/dog-cat-pose

Simple controlnet model made as part of the HF JaX/Diffusers community sprint.

These are controlnet weights trained on runwayml/stable-diffusion-v1-5 with pose conditioning generated using the animalpose model of OpenPifPaf. 

Some example images can be found in the following

prompt: a tortoiseshell cat is sitting on a cushion
!images_0)
prompt: a yellow dog standing on a lawn
!images_1)

Whilst not the dataset used for this model, a smaller dataset with the same
format for conditioning images can be found at URL

The dataset was generated using the code at URL # Model Card for dog-cat-pose

This is an ControlNet model which allows users to control the pose of a dog or cat. Poses were extracted from images using the animalpose model of OpenPifPaf URL . Skeleton colouring is as shown in the dataset. See also URL # Model Details ## Model Description


This is an ControlNet model which allows users to control the pose of a dog or cat. Poses were extracted from images using the animalpose model of OpenPifPaf URL Skeleton colouring is as shown in the dataset. See also URL

- Developed by: John Fozard
- Model type: Conditional image generation
- Language(s) (NLP): en
- License: openrail
- Parent Model: URL
- Resources for more information: 
    - GitHub Repo # Uses ## Direct Use




Supply a suitable, potentially incomplete pose along with  a relevant text prompt ## Out-of-Scope Use




Generating images of non-animals. We advise retaining the stable diffusion safety filter when using this model. # Bias, Risks, and Limitations



The model is trained on a relatively small dataset, and may be overfit to those images. ## Recommendations




Maintain careful supervision of model inputs and outputs. # Training Details ## Training Data



Trained on a subset of Laion-5B using clip retrieval with the prompts &#34;a photo of a (dog/cat) (standing/walking)&#34; ## Training Procedure ### Preprocessing

Images were rescaled to 512 along their short edge and centrally cropped. The OpenPifPaf pose-detection model was used to extract poses, which were used to generate conditioning images. ## Compute Infrastructure

TPUv4i ### Software

Flax stable diffusion controlnet pipeline # Model Card Authors [optional]



John Fozard
TAGS
#diffusers #safetensors #stable-diffusion #stable-diffusion-diffusers #image-to-image #controlnet #jax-diffusers-event #base_model-runwayml/stable-diffusion-v1-5 #license-creativeml-openrail-m #has_space #diffusers-ControlNetModel #region-us 
 # controlnet- JFoz/dog-cat-pose

Simple controlnet model made as part of the HF JaX/Diffusers community sprint.

These are controlnet weights trained on runwayml/stable-diffusion-v1-5 with pose conditioning generated using the animalpose model of OpenPifPaf. 

Some example images can be found in the following

prompt: a tortoiseshell cat is sitting on a cushion
!images_0)
prompt: a yellow dog standing on a lawn
!images_1)

Whilst not the dataset used for this model, a smaller dataset with the same
format for conditioning images can be found at URL

The dataset was generated using the code at URL # Model Card for dog-cat-pose

This is an ControlNet model which allows users to control the pose of a dog or cat. Poses were extracted from images using the animalpose model of OpenPifPaf URL . Skeleton colouring is as shown in the dataset. See also URL # Model Details ## Model Description


This is an ControlNet model which allows users to control the pose of a dog or cat. Poses were extracted from images using the animalpose model of OpenPifPaf URL Skeleton colouring is as shown in the dataset. See also URL

- Developed by: John Fozard
- Model type: Conditional image generation
- Language(s) (NLP): en
- License: openrail
- Parent Model: URL
- Resources for more information: 
    - GitHub Repo # Uses ## Direct Use




Supply a suitable, potentially incomplete pose along with  a relevant text prompt ## Out-of-Scope Use




Generating images of non-animals. We advise retaining the stable diffusion safety filter when using this model. # Bias, Risks, and Limitations



The model is trained on a relatively small dataset, and may be overfit to those images. ## Recommendations




Maintain careful supervision of model inputs and outputs. # Training Details ## Training Data



Trained on a subset of Laion-5B using clip retrieval with the prompts &#34;a photo of a (dog/cat) (standing/walking)&#34; ## Training Procedure ### Preprocessing

Images were rescaled to 512 along their short edge and centrally cropped. The OpenPifPaf pose-detection model was used to extract poses, which were used to generate conditioning images. ## Compute Infrastructure

TPUv4i ### Software

Flax stable diffusion controlnet pipeline # Model Card Authors [optional]



John Fozard
TAGS
#diffusers #safetensors #stable-diffusion #stable-diffusion-diffusers #image-to-image #controlnet #jax-diffusers-event #base_model-runwayml/stable-diffusion-v1-5 #license-creativeml-openrail-m #has_space #diffusers-ControlNetModel #region-us 
# controlnet- JFoz/dog-cat-pose

Simple controlnet model made as part of the HF JaX/Diffusers community sprint.

These are controlnet weights trained on runwayml/stable-diffusion-v1-5 with pose conditioning generated using the animalpose model of OpenPifPaf. 

Some example images can be found in the following

prompt: a tortoiseshell cat is sitting on a cushion
!images_0)
prompt: a yellow dog standing on a lawn
!images_1)

Whilst not the dataset used for this model, a smaller dataset with the same
format for conditioning images can be found at URL

The dataset was generated using the code at URL# Model Card for dog-cat-pose

This is an ControlNet model which allows users to control the pose of a dog or cat. Poses were extracted from images using the animalpose model of OpenPifPaf URL . Skeleton colouring is as shown in the dataset. See also URL# Model Details## Model Description


This is an ControlNet model which allows users to control the pose of a dog or cat. Poses were extracted from images using the animalpose model of OpenPifPaf URL Skeleton colouring is as shown in the dataset. See also URL

- Developed by: John Fozard
- Model type: Conditional image generation
- Language(s) (NLP): en
- License: openrail
- Parent Model: URL
- Resources for more information: 
    - GitHub Repo# Uses## Direct Use




Supply a suitable, potentially incomplete pose along with  a relevant text prompt## Out-of-Scope Use




Generating images of non-animals. We advise retaining the stable diffusion safety filter when using this model.# Bias, Risks, and Limitations



The model is trained on a relatively small dataset, and may be overfit to those images.## Recommendations




Maintain careful supervision of model inputs and outputs.# Training Details## Training Data



Trained on a subset of Laion-5B using clip retrieval with the prompts &#34;a photo of a (dog/cat) (standing/walking)&#34;## Training Procedure### Preprocessing

Images were rescaled to 512 along their short edge and centrally cropped. The OpenPifPaf pose-detection model was used to extract poses, which were used to generate conditioning images.## Compute Infrastructure

TPUv4i### Software

Flax stable diffusion controlnet pipeline# Model Card Authors [optional]



John Fozard