For example, [DistilBertForSequenceClassification] is a base DistilBERT model with a sequence classification head. The sequence classification head is a linear layer on top of the pooled outputs.

from transformers import DistilBertForSequenceClassification
model = DistilBertForSequenceClassification.from_pretrained("distilbert/distilbert-base-uncased")