Will XLM-R-(X)XL be available in Models?
Hi guys,PyTorch FairSeq has recently published larger XLM-R models here:fairseq/examples/xlmr at master · pytorch/fairseq · GitHubThey have XLM-R-XL and XLM-R-XXL models that are much larger than the XLM-R-Base and XLM-R-Large that are currently available inModels. Is it possible to load these models inTransformers? What would it take to convert them?
2021-10-25T19:03:02Z
I know this is an old post, but in case you are still interested there is an ongoing pull request for it here:Add support for XLM-R XL and XXL models by modeling_xlm_roberta_xl.py by Soonhwan-Kwon · Pull Request #13727 · huggingface/transformers · GitHub