Can BERT for mlm predict never seen words?
Is it possible to my BERT model predict a word seen in my training samples, but not contained in my wordpiece tokenizer vocab?
2021-12-29T19:01:23Z
I am not sure I understand your question. Any word in your training sample would be broken down into word-piece tokens, and then converted into IDs. So yes, with the MLM task BERT can predict a word from the training sample.