Importing community made models and pipeline elements that don't have the diffusers format
ObjectiveI am currently following the using diffusers section on the documentation page and have come to a point where I can swap out pipeline elements from valid diffusers libraries hosted at hugging face. I can either load the model from the diffusers pipeline and get the components I want, or download and replace the relevant folder manually from the repository. Expanding this, I wanted to try to use community-made stable diffusion models on websites like Civitai, as well as custom community components.As an example, I would like to use this model checkpoint that is available in a safetensor format:https://civitai.com/models/7371/rev-animatedWith the following vae(s):https://huggingface.co/WarriorMama777/OrangeMixs/tree/main/VAEsProblemI have found that cloning thestable diffusion-v1-5repository and replacing the.safetensorsand.ckptswith the revAnimated '.safetensor` works to run the model.However, the community vae(s) come either in:.safetensor.vae.pt.ckptwhich is not compatible with thediffusers.models.AutoencoderKLas it expects aconfig.yamlanddiffusion_pytorch_model.binfile. I would like to try to convert these files into a format usable by the diffusers library.Attempted SolutionsFor the conversion I have tried following the instructions on this issue:https://github.com/huggingface/diffusers/issues/1928This leads me to the following spaces:https://huggingface.co/spaces/camenduru/converterhttps://huggingface.co/spaces/diffusers/sd-to-diffusersThe first page gets stuck in a preparing space loop:image1473×641 21.4 KBThe other page has a runtime error on load:Runtime error
Traceback (most recent call last):
  File "app.py", line 3, in <module>
    from convert import convert
  File "/home/user/app/convert.py", line 74, in <module>
    def convert(token: str, model_id: str, filename: str, model_type: str, sample_size: int = 512, scheduler_type: str = "pndm", extract_ema: bool = True, progress=gr.Progress()):
AttributeError: module 'gradio' has no attribute 'Progress'QueriesIs there a way I can do the conversion locally?If yes, does the resulting conversion give me a stable diffusion model or just the vae components?Is there a way to use these components with the diffuser library without the need for conversions?I am aware that these work out of the box with a custom web UI like Automatic 1111, however, my objective is to better learn the diffusers library.Thank you for taking the time to go through my topic
2023-05-22T18:21:24Z
hey@Dragonjinxwe have conversion scripts for stable diffusion herediffusers/convert_original_stable_diffusion_to_diffusers.py at main · huggingface/diffusers · GitHubNote that we can’t support arbitrary conversions for arbitrary models and I don’t think we have any recommended vae conversion scripts. You might have to write the code yourself