transformers text-to-image en arxiv:2305.15194 license:apache-2.0 endpoints_compatible region:us
---
license: apache-2.0
language:
- en
library_name: transformers
pipeline_tag: text-to-image
---

<br>

# DiffBlender Model Card

This repo contains the models from our paper [**DiffBlender: Scalable and Composable Multimodal Text-to-Image Diffusion Models**](https://arxiv.org/abs/2305.15194).


## Model details

**Model type:**
DiffBlender successfully synthesizes complex combinations of input modalities. 
It enables flexible manipulation of conditions, providing the customized generation aligned with user preferences.
We designed its structure to intuitively extend to additional modalities while achieving a low training cost through a partial update of hypernetworks.

We provide its model checkpoint, trained with six modalities: sketch, depth map, grounding box, keypoints, color palette, and style embedding. >> `./checkpoint_latest.pth`

**License:**
Apache 2.0 License

**Where to send questions or comments about the model:**
https://github.com/sungnyun/diffblender/issues


## Training dataset
[Microsoft COCO 2017 dataset](https://cocodataset.org/#home)


<br>

More detials are in our project page, https://sungnyun.github.io/diffblender/.