Why are some NLI models giving logits in opposite positions to expected labels?
I am looking intojoeddav/xlm-roberta-large-xnlifor the purposes of additional fine-tuning on other languages.From the examples in the model card, it appears to use the following label assignment.contradition=0, neutral=1, entailment =2# we throw away "neutral" (dim 1) and take the probability of
# "entailment" (2) as the probability of the label being true 
entail_contradiction_logits = logits[:,[0,2]]However, the HF XNLI dataset (xnli · Datasets at Hugging Face) uses the following labels:entailment=0, neutral=1, contradiction=2.This seems to be the inverse of the former. This is further supported by the results that, when I evaluated the model on the English XNLI test set, I got an accuracy of 33% initially, but got 99% when I inverted the labels (e.g. swap 0 with 2) before feeding into the model.Can anyone please confirm if this is indeed the case, and what might be the reason?However, later models seem to output the logits in the positions consistent with the dataset. For example, MoritzLaurer/mDeBERTa-v3-base-mnli-xnli.