How to get predictions after evaluation phase in a callback
I want to log prediction distribution after each evaluation phase to Weights & Biases. I thought of adding a callback to do this, but couldn’t find anything which allows me to access predictions inon_evaluatemethod to do so. Is there any way to achieve this or am I missing something ?
2022-04-25T06:35:30Z
If you want to get the predictions back, you should useTrainer.predict().github.comhuggingface/transformers/blob/72728be3dbca26c70dddc8b724eb2c8d901e97dc/src/transformers/trainer.py#L2343-L2350def predict(self, test_dataset: Dataset, ignore_keys: Optional[List[str]] = None, metric_key_prefix: str = "test") -> PredictionOutput:"""Run prediction and returns predictions and potential metrics.Depending on the dataset and your use case, your test dataset may contain labels. In that case, this methodwill also return metrics, like in `evaluate()`.