OOM
How i can fix Oout of memory error .Device : rtx 4090 , VRAM : 24 GB
self.model = load_flow_model(self.name, device="cpu" if offload else self.torch_device)
File "/home/taras/flux/flux/util.py", line 344, in load_flow_model
sd = load_sft(ckpt_path, device=str(device))
File "/home/taras/flux/.venv/lib/python3.10/site-packages/safetensors/torch.py", line 315, in load_file
result[k] = f.get_tensor(k)
torch.OutOfMemoryError: CUDA out of memory. Tried to allocate 90.00 MiB. GPU 0 has a total capacity of 23.63 GiB of which 96.25 MiB is free. Including non-PyTorch memory, this process has 23.35 GiB memory in use. Of the allocated memory 22.97 GiB is allocated by PyTorch, and 7.31 MiB is reserved by PyTorch but unallocated. If reserved but unallocated memory is large try setting PYTORCH_CUDA_ALLOC_CONF=expandable_segments:True to avoid fragmentation.
Outpainting + 400px on an image with a resolution of 2336x1568px works well for me in ComfyUI
(RTX 3090 24GB, 32GB RAM) Workflow: https://pastebin.com/7fxwn3Am
same here, I've got 16GB Vram and 32GB ram and even after offload parts of the pipeline to CPU and utilizing everything to it's max still get the same error.
I have Comfy UI on my machine too and I can run the simple workflow with flux and get the result without any problem.
My question is why it needs so much ram if we use code to run it instead of comfy ui