

Since the automatic detector only reports on inputs and outputs of full frames, once you know where to look, you may
want to analyse the intermediary stages of any specific forward function as well. In such a case you can use the
detect_overflow helper function to inject the detector where you want it, for example:
thon
from debug_utils import detect_overflow
class T5LayerFF(nn.Module):
    []
def forward(self, hidden_states):
    forwarded_states = self.layer_norm(hidden_states)
    detect_overflow(forwarded_states, "after layer_norm")
    forwarded_states = self.DenseReluDense(forwarded_states)
    detect_overflow(forwarded_states, "after DenseReluDense")
    return hidden_states + self.dropout(forwarded_states)