

You can customize the training loop behavior by subclassing the methods inside [Trainer]. This allows you to customize features such as the loss function, optimizer, and scheduler. Take a look at the [Trainer] reference for which methods can be subclassed. 
The other way to customize the training loop is by using Callbacks. You can use callbacks to integrate with other libraries and inspect the training loop to report on progress or stop the training early. Callbacks do not modify anything in the training loop itself. To customize something like the loss function, you need to subclass the [Trainer] instead.
Train with TensorFlow
All models are a standard tf.keras.Model so they can be trained in TensorFlow with the Keras API. 🤗 Transformers provides the [~TFPreTrainedModel.prepare_tf_dataset] method to easily load your dataset as a tf.data.Dataset so you can start training right away with Keras' compile and fit methods.