For the logits from HuggingFace Transformer models, can the sum of the elements of the logit vector be greater than 1?
Hello,For the logits from HuggingFace Transformer models, can the sum of the elements of the logit vector be greater than 1?I am getting a logit vector which their element sums to a number greater than 1, and I just think this is strange.Thanks,
2020-10-16T15:00:17Z
This is normal. The logits are not normalized, and must be passed through a softmax in order to form a valid probability distribution over tokens.