

Find the best way of debugging intermediate results. Is the original repository written in PyTorch? Then you should
  probably take the time to write a longer script that decomposes the original model into smaller sub-components to
  retrieve intermediate values. Is the original repository written in Tensorflow 1? Then you might have to rely on
  TensorFlow print operations like tf.print to output
  intermediate values. Is the original repository written in Jax? Then make sure that the model is not jitted when
  running the forward pass, e.g. check-out this link.
Use the smallest pretrained checkpoint you can find. The smaller the checkpoint, the faster your debug cycle
  becomes. It is not efficient if your pretrained model is so big that your forward pass takes more than 10 seconds.
  In case only very large checkpoints are available, it might make more sense to create a dummy model in the new
  environment with randomly initialized weights and save those weights for comparison with the 🤗 Transformers version
  of your model
Make sure you are using the easiest way of calling a forward pass in the original repository. Ideally, you want to
  find the function in the original repository that only calls a single forward pass, i.e. that is often called
  predict, evaluate, forward or __call__. You don't want to debug a function that calls forward
  multiple times, e.g. to generate text, like autoregressive_sample, generate.
Try to separate the tokenization from the model's forward pass. If the original repository shows examples where
  you have to input a string, then try to find out where in the forward call the string input is changed to input ids
  and start from this point. This might mean that you have to possibly write a small script yourself or change the
  original code so that you can directly input the ids instead of an input string.
Make sure that the model in your debugging setup is not in training mode, which often causes the model to yield
  random outputs due to multiple dropout layers in the model. Make sure that the forward pass in your debugging
  environment is deterministic so that the dropout layers are not used. Or use transformers.utils.set_seed
  if the old and new implementations are in the same framework.