Confusing diffusers documentation on usage of kohya _ss Lora + SDXL
In diffusers documentation there are several ways to load and use Lora models with SDXL but it starts to get quite confusing what is the correct way to do itI have trained a lora model on SDXL model using Kohya_ssNow I want to load it form the .safetensors file in the pipeline and generate an image with lora scale=1.There are these 2 linksLoad LoRAs for inferenceLow-Rank Adaptation of Large Language Models (LoRA)What is confusing is when we load lora in the pipeline is :do I need to use PEFT and set the adapters?do I need to fuse_lora() in the pipeline and specify the lora_scale?are these mutually exclusive or doing both won’t affect the result?What is confusing when generating the image is:do I need to format the prompt as in auto1111 adding<lora:model_name-step:lora_scale>to it or just use the token+class ?do I need to passcross_attention_kwargs={"scale": lora_scale}when generating the image?Anyone can clarify what are the correct steps?thank you