Trainer API to log both Training and Validation Metrics
I am fine-tuning for a classification task - I am trying to replicate (and potentially replace) my native PyTorch training and evaluation loops with the Trainer API.I usually log metrics for both training and validation across each batch/epoch.Here is what I can achieve with Trainer API.The accuracy and F1 are of validation sets and I want to also see the same set of metrics at each step for training data as well. Can someone guide on how I can achieve this?Thanks.
2021-07-01T12:40:12Z
This is not implemented in theTrainer. You can manuallyevaluateyourself at the end of training on any dataset you want (training set included).
2021-07-01T18:44:34Z
Thanks for the reply@sgugger. Since loss may not be directly related to certain metrics, I was hoping to see where the metrics trend starts overfitting and make decisions based on that. Do you think this is a valid use case to think about adding this to the API in the future?