If your sequence_length is super regular, then batching is more likely to be VERY interesting, measure and push
    it until you get OOMs.
The larger the GPU the more likely batching is going to be more interesting
As soon as you enable batching, make sure you can handle OOMs nicely.