A blog on How to train a new language model from scratch using Transformers and Tokenizers with RoBERTa.
[RobertaForMaskedLM] is supported by this example script and notebook.
[TFRobertaForMaskedLM] is supported by this example script and notebook.
[FlaxRobertaForMaskedLM] is supported by this example script and notebook.
Masked language modeling chapter of the 🤗 Hugging Face Course.
Masked language modeling task guide