Then you can load BERT with [TFAutoModelForMultipleChoice]:

from transformers import TFAutoModelForMultipleChoice
model = TFAutoModelForMultipleChoice.from_pretrained("google-bert/bert-base-uncased")

Convert your datasets to the tf.data.Dataset format with [~transformers.TFPreTrainedModel.prepare_tf_dataset]: