Smaller pretrained models for Stable Diffusion?
As I restart my program for the hundredth time, I think:maybe if I used a smaller model while developing, I wouldn’t spend so long waiting for it to load.Are there smaller pretrained models available that work with the samediffusers StableDiffusionPipeline?I looked at a few of the options atModels - Hugging Facebut they seem to be mostly fine-tunings of the SD models and they’re not any smaller, and/or they don’t have data published for fp16.
2022-09-25T20:10:56Z
Hi@keturn!If you just want fast loading and don’t care about generation you could try to usethis tiny modelfor development. It’s not trained or anything so predictions will be useless, but it could be helpful when you are working on new features.You can use it like this:from diffusers import DiffusionPipeline
pipe = DiffusionPipeline.from_pretrained("hf-internal-testing/tiny-stable-diffusion-pipe")If you do need some reasonable outputs, then I’m not sure what would be the best option.
2022-09-26T00:00:12Z
Have you tried passinglocal_files_only=Truetofrom_pretrained? I think the slow part is the fact that is checks for updates online. I’ve also serialized the Pipeline to disk and then load time is just deserialization, which is pretty fast.
2022-09-26T01:13:29Z
dkackman:I think the slow part is the fact that is checks for updates online.Hmm, what makes you think that? I never suspected that might be a problem, since these files aren’t getting updates often.I triedlocal_files_onlyand it saves about a second (less than 10% of the total time).dkackman:I’ve also serialized the Pipeline to disk and then load time is just deserialization, which is pretty fast.How is that different than the default implementation? Do you have a serialization format that’s lots faster to load than pytorch’s pickle-based format?pcuenq:If you do need some reasonable outputs, then I’m not sure what would be the best option.I don’t need anythingimpressive,but yes, something likereasonableoutputs would be nice so I can tell whether or not I’ve completely biffed up the inputs or processing.Thank you both for the suggestions!
2022-09-26T13:59:55Z
Hmm, what makes you think that?Just an assumption. Mostly wrong it looks like.How is that different than the default implementation? Do you have a serialization format that’s lots faster > to load than pytorch’s pickle-based format?Plain old pickle binary format. But I’m serializing the output offrom_pretrainedthereby paying that cost only once. Seems about 5 times faster give or take (this is on an NVME drive mind you)