from transformers import AutoModelForSequenceClassification
model = AutoModelForSequenceClassification.from_pretrained("google-bert/bert-base-cased", num_labels=5)

Optimizer and learning rate scheduler
Create an optimizer and learning rate scheduler to fine-tune the model. Let's use the AdamW optimizer from PyTorch:

from torch.optim import AdamW
optimizer = AdamW(model.parameters(), lr=5e-5)

Create the default learning rate scheduler from [Trainer]: