Will Trainer loss functions automatically ignore -100?
I’m learning the tutorialToken classification.And I have a question:In preprocessing part, it converted the label of special tokens and the tokens that are not the first token in the word to -100.It said that this will show its usefulness in loss function, but the code hasn’t use something likeignore_indexin loss function. The -100 seems only work in metric evaluation.So I wonder if loss function in Trainer will automatically ignore -100?Or it works in num_labels, id2label and label2id parameter? Does it mean that I set these parameters, other labels will be ignored?
2023-06-29T10:54:01Z
I have the same question forGPT2LMHeadModel.Edit:-100is the defaultignore_indexinPyTorch’s CrossEntropyLoss. So, any token with a label of-100will be ignored in loss computation.
2024-05-25T06:17:14Z
This topic was automatically closed 12 hours after the last reply. New replies are no longer allowed.