Problem with XL ControlNet Inpaint
I don’t know if anyone has the same problem: when I use the controlnet inpainting model via diffusers StableDiffusionXLControlNetInpaintPipeline, the result didn’t follow the prompt, but the same model works well on the webui.This is only happens with XL version(no matter the mask area of control image is set to 0 or -1, the original controlnet uses -1, so does the v1.x diffusers piepeline, xl diffusers piepeline change it to 0.Would that be more reasonable? Why change?)Besides, I found the results with/without cotrolnet are weird，can’t explain why it happened.1.base model: dreamshaper-xl-1-0 (I downloaded the model from huggingface for diffusers and civitai  for webui, I guess there’s no difference.)2. controlnet trained with diffusers3. prompt: a blue suv is driving down the roadThe result of diffusers is completely unexplained: the trained controlnet makes the result worse.
2023-11-23T09:00:24Z
raw image:background1024×1024 214 KB
2023-11-23T09:00:54Z
mask:mask1024×1024 3.55 KB
2023-11-23T09:01:36Z
diffusers use controlnet-inpainting :  StableDiffusionXLControlNetInpaintPipeline & seed 2037764651compare—_0_01024×1024 138 KB
2023-11-23T09:02:08Z
diffusers inapinting without controlnet :StableDiffusionXLInpaintPipeline & seed 2037764651no_controlnet_0_01024×1024 140 KB
2023-11-23T09:02:39Z
webui img2img inpainting with controlnet:seed 203776465100004-20377646511024×1024 180 KB
2023-11-23T09:03:17Z
webui img2img inpainting without controlnet:seed 203776465100005-20377646511024×1024 171 KB