Conversion of B to I tokens, didn't update number labels for model
Inrun_ner.pyfile , when we setlabel_all_tokensinto True , it convert all B-Xxx label to its I-Xxx. Thus it reduce the unique labels . Even though unique label get reduced , models still accept the old number of unique label asnum_label.File :https://github.com/huggingface/transformers/blob/main/examples/pytorch/token-classification/run_ner.pyI believe , thenum_labelandmodel.config.label2idshould be changed . Correct me if am wrong.