Accelerate: 'RobertaModel' object has no attribute 'roberta'
I am trying to load aRobertaModelusing Accelerate, following theexample in the documentation.This is my code:with init_empty_weights():
            model = RobertaModel.from_pretrained(model_name_or_path)
        model = load_checkpoint_and_dispatch(
            model, checkpoint=checkpoint, device_map="auto"
        )However, this results in an error:AttributeError: 'RobertaModel' object has no attribute 'roberta'I have tried with variousRobertaModels as well as withXmodModel, whereas doing the same with aAutoModelForMaskedLM(for instance) works. IsRobertaModelnot supported by Accelerate?Is there a list of supported model types somewhere? I could not find any in the documentation.
2023-08-29T13:44:13Z
I suppose the given example was wrong for my use case. This implementation seems to work for me with all model types:model = RobertaModel.from_pretrained(...)
accelerator = Accelerator()
accelerator.prepare(model)