Re-Training with new number of classes
Hi All.I already trained an NER (token classification) model on a custom training dataset with 19 classes. You can explore it heremarefa-nlp/marefa-nerThe base model which I used to fine-tune my model wasxlm-roberta-largeI have now a new dataset and need to use the last trained modelmarefa-nlp/marefa-neras the base model this time.The problem is that the last model was trained to predict the class out of 19 classes, while the new dataset is designed for just 6 classes.I tried to load the model and reset the configurations to thexlm-roberta-largeconfiguration like thisfrom transformers import AutoModelForTokenClassification, AutoConfig

base_model = "xlm-roberta-large"
ft_model = "marefa-nlp/marefa-ner"

config = AutoConfig.from_pretrained(base_model)

ner_model = AutoModelForTokenClassification.from_pretrained(ft_model, num_labels=19)
ner_model.config = config 

# THEN using the ner_model to train with the new datasetbut seems not working, as it still requires that the head size be 19===Does anyone know how to solve this?Thanks
2022-01-03T09:50:13Z
Hi,This can be done by passing the additional argumentignore_mismatched_sizes=Trueto thefrom_pretrainedmethod.
2022-01-03T20:07:24Z
Thank you@nielsr. It works now.I think this must be highlighted more in the new documentation.