Is there a way to backpropagate through multiple steps while using Trainer API
I was wondering if there’s a way to backpropagate the accumulation of loss through multiple optimizer steps while using Trainer API? Since it’s easy to get cuda out of memory and would like to avoid it.
2021-07-09T15:06:11Z
If you are talking about gradient accumulation, you can set it withgradient_accumulation_steps=xxxin yourTrainingArguments.