Deepspeed mii library issues
i tried the Deepspeed mii library for create a pipeline with jupiter cuda compatibility score 8.0+ but it’s give me error :OutOfMemoryError: CUDA out of memory. Tried to allocate 19.75 GiB. GPU 0 has a total capacity of 22.03 GiB of which 19.51 GiB is free. Including non-PyTorch memory, this process has 2.52 GiB memory in use. Of the allocated memory 1.35 GiB is allocated by PyTorch, and 5.34 MiB is reserved by PyTorch but unallocated. If reserved but unallocated memory is large try settingPYTORCH_CUDA_ALLOC_CONF=expandable_segments:True to avoid fragmentation.  See documentation for Memory Management  (CUDA semantics — PyTorch 2.4 documentation)if know any one how to solve this error please help me.
2024-09-29T07:21:53Z
There are ways to kill the process frequently or manually manipulate only a part of the model without relying on libraries, but the easiest way seems to be to reduce the image size or data set size.PyTorch Forums – 29 Jul 22Memory Management using PYTORCH_CUDA_ALLOC_CONFCan I do anything about this, while training a model I am getting this cuda error:  RuntimeError: CUDA out of memory. Tried to allocate 30.00 MiB (GPU 0; 2.00 GiB total capacity; 1.72 GiB already allocated; 0 bytes free; 1.74 GiB reserved in total...Reading time: 5 mins 🕑Likes: 7 ❤stackoverflow.comRuntimeError: CUDA out of memory. How can I set max_split_size_mb?memory, pytorch, runtime-error, google-colaboratoryasked byelion04:18PM - 16 Sep 22 UTC