GPT is a model with absolute position embeddings so it's usually advised to pad the inputs on the right rather than
  the left.
GPT was trained with a causal language modeling (CLM) objective and is therefore powerful at predicting the next
  token in a sequence. Leveraging this feature allows GPT-2 to generate syntactically coherent text as it can be
  observed in the run_generation.py example script.