Using sample weights in compute_metrics
Is it possible to include sample weights in thecompute_metricsfunction used byTrainer?For simpler cases, I can just encode the gold labels for some samples as -1 and give them a weight of 0 when computing the metrics, but in other cases it might be trickier.
2021-10-20T11:58:35Z
I don’t think it’s possible. For elaborate evaluations you can pick on your predictions after usingtrainer.predictthen do something more sophisticated on them.