Train

If you aren't familiar with finetuning a model with the [Trainer], take a look at the basic tutorial here!

You're ready to start training your model now! Load DistilRoBERTa with [AutoModelForMaskedLM]:

from transformers import AutoModelForMaskedLM
model = AutoModelForMaskedLM.from_pretrained("distilbert/distilroberta-base")

At this point, only three steps remain: