Preprocess
The next step is to load a BERT tokenizer to process the sentence starts and the four possible endings:

from transformers import AutoTokenizer
tokenizer = AutoTokenizer.from_pretrained("google-bert/bert-base-uncased")

The preprocessing function you want to create needs to: