| In Distributed Data Parallel (DDP) mode, `prepare_for_backward()` is called at the end of DDP forward pass. It traverses the autograd graph to find unused parameters when `find_unused_parameters` is set to True in DDP constructor. | |
| Note that traversing the autograd graph introduces extra overheads, so applications should only set to True when necessary. |