Handling OOMs nicely when enabling batching in general
From theoveriew page of Pipelines, we get some general recommendations on when we should enable batching. It mentions handloing OOMs nicely when using batching. Any concrete example of what it means to handle OOMs nicely? I understand it’s highly dependent on the applications but still was hoping to get a concrete example.
2023-08-17T07:54:06Z
Has there been any update on this? The closest thing I can find is thisthreadon Pytorch.