How to have one pipeline to perform text2img, img2img with shared Stable Diffusion model?
Hi,I would like to deploy SD on the server and accomplish text2img and img2img tasks.If the image is present, then it would perform img2img, if not, it would do the text2img.What I would like to do is to have only one model in the memory, so it will not consumemore than is really needed. Is it possible with some easy hack?
2022-09-08T18:29:46Z
Pipe could use a more global manager, with caching to dramatically reduce footprint when not in use after setup, or after a run. But what you should do in the meantime if you’re switching a pipe, is justdel pipeand flush torch memory and do a garbage cleanup. Then you can load up the img2img pipe if a init is available, keeping only one in memory.Another option is to, also dodel pipemethod, but to employ joblib and dump the pipe to disk, and load it on demand. So you could load up all your pipes, and cache them to disk, and load them just before diffusion, then either dump again (in case of param changes) or just del and load from the same cache before another run. Switch to a img2img cache when init is detected.The pipe will take same amount of time to set up initially before it’s dumped, but when it loads a pipe, it will take under a second and be ready to feed prompts, etc, and start a run.
2022-09-16T01:58:58Z
A version of what’s described above in case it helps anyone:github.comdkackman/fing/blob/main/src/generator/pipelines.pyimport torch
import logging
from diffusers import StableDiffusionPipeline, StableDiffusionImg2ImgPipeline, StableDiffusionInpaintPipeline
import pickle


def preload_pipelines(model_name, auth_token, pipeline_names = ["txt2img", "img2img", "imginpaint"]):
    # this will preload all the pipelines and serialize them to disk
    # get_pipeline will then retrive from disk, accomplishing two things:
    # 1 - pay the startup cost to get the model form hugging face only 1 time per process
    # 2 - keep them out of RAM (main and GPU) until actually needed
    # on demand they get deserialized and pushed to the gpu
    #
    # TODO #8 model the GPU as a class; including what pipeline is loaded and if it has a workload or not
    if "txt2img" in pipeline_names:
        logging.debug("Loading txt2img")
        pipeline = StableDiffusionPipeline.from_pretrained(
            model_name,
            revision="fp16",
            torch_dtype=torch.float16,This file has been truncated.show original
2022-09-23T12:13:59Z
You may also find a related GitHub discussion interesting, in particular the code snippet inthis comment.