How do you set up a VAE in Diffusers?
Hey guyss, I’m trying to use the diffusers library withandite/pastel-mix=>https://huggingface.co/andite/pastel-mixHow do you set up the VAE here?
2023-03-27T07:01:08Z
Hi@AliceM!, I’m not sure I fully understand your question, looking at the model repo it looks like it’s a valid diffusers checkpoint. To use it you’d use code similar to the one that appears in the model card:from diffusers import StableDiffusionPipeline
import torch

model_id = "andite/pastel-mix"
pipe = StableDiffusionPipeline.from_pretrained(model_id, torch_dtype=torch.float16)
pipe = pipe.to("cuda")

prompt = "hatsune_miku"
image = pipe(prompt).images[0]

image.save("./hatsune_miku.png")Is that not working for you?
2023-03-27T17:17:29Z
I’m only getting dull colors with that snippet…After experimenting a lot, it seems like it only works when I put another vae on it:vae = AutoencoderKL.from_pretrained("stabilityai/sd-vae-ft-mse").to("cuda")
pipeline.vae = vaeThis feels very dirty though so I wonder why it doesn’t work…
2023-03-27T17:37:57Z
Oh, I see. It would appear that the model was trained with a different vae but the model card was not updated to reflect that fact. In addition, I see in the root of the repo a few differentvaemodels, I’m not sure how they are supposed to be used.I’d recommend you open a discussion in the repo itself asking for clarification. I’d ask the author if it is possible to update the model card with instructions, and ideally update thevaefolder of the repo so that it contains the recommendedvaeinstead of the default one. To open a discussion, please use theCommunitytab in the repo.Hope that helps
2023-09-15T19:05:49Z
Hello, I have something very similar, I come from the implementation of automatic1111, where I have generated checkpoints with dreambooth of which in automatic1111 they work correctly, but now that I implement my model in diffusers library and in different formats such as .safetensors, .ckpt, .bin, I have not achieved the quality in the images that I used to generate in automatic1111, I have implemented and added “VAE”, “scheduler”, “clip skip”, and I have not been able to generate the same consistency in the images, I do not know if there are more configuration in diffusers to be able to achieve a good quality of my images, does anyone have an idea on how to improve the generation
2023-09-15T19:08:14Z
EXamples Difussers: the colors come out very opaque and not saturated