Use the end-of-sequence token as the padding token and set mlm=False. This will use the inputs as labels shifted to the right by one element:

from transformers import DataCollatorForLanguageModeling
data_collator = DataCollatorForLanguageModeling(tokenizer=tokenizer, mlm=False, return_tensors="tf")

Train

If you aren't familiar with finetuning a model with the [Trainer], take a look at the basic tutorial!

You're ready to start training your model now! Load DistilGPT2 with [AutoModelForCausalLM]: