How to define the compute_metrics() function in Trainer?
Hello,Coming fromtensorflowI am a bit confused as to how to properly define the compute_metrics() in Trainer. For instance, I see in the notebooks various possibilitiesdef compute_metrics(eval_pred):
    predictions, labels = eval_pred
    predictions = predictions[:, 0]
    return metric.compute(predictions=predictions, references=labels)My question may seem stupid (maybe it is) but how can I know how to compute the metrics if I cannot see whateval_predlooks like in Trainer? It is as if I had to guess what the output will be before actually training the model. Am I missing something here?Thanks!
2021-12-20T10:44:00Z
Hello, I’m a bit confused about your question. Are you trying to implement this on Trainer or TensorFlow?
2021-12-20T13:23:07Z
hello@merve, thanks. I am trying to write the correct Trainer (in pytorch). The point is that I can follow the tutorial and copy-paste the compute_metrics functions there. However, if I want to modify them I need to understand and play a little bit with theeval_predoutput. How can I geteval_predfrom atrainerso that I can see whatpredictionsandlabelslook like?
2021-12-20T13:57:10Z
Just runtrainer.predicton your eval/test dataset.