Log Perplexity using Trainer
Hi there,I am wondering, what would be the optimal solution to also report and logperplexityduring the training loop via theTrainerAPI. How would the correspondingcompute_metricsfunction look like. So far I tried without success since I am not 100% sure how theEvalPredictionoutput would look like.Thanks in advanceSimon
2021-03-27T15:55:05Z
This brings me to an adjacent question: How would acompute_metricsfunction look like that can also report the relative changeperplexityrespectivelytrain_loss? I would be super grateful if anyone could provide a little guidance!
2021-10-09T14:53:54Z
Hi, I am also looking very forward to see how it is possible to utilize perplexity during execution oc compute_metrixs() while finetuning pretrained models for MLM or CLM