Adding new special token to the Seq2Seq model
I am trying to add a new special token to the opus mt model (using AutoModelForSeq2SeqLM), I found these steps for doing thisspecial_tokens_dict = {'additional_special_tokens': ['<x1>','<x2>']}
num_added_toks = tokenizer.add_special_tokens(special_tokens_dict)
model.resize_token_embeddings(len(tokenizer))But I am wondering if this only the required steps since I am loading the model from config (from scratch). Shouldn’t there be other steps? such as modify the positional encoding matrix for the new special token, or any other modifications on both model and tokenizer.