

As you can see, we do make use of inheritance in 🤗 Transformers, but we keep the level of abstraction to an absolute
minimum. There are never more than two levels of abstraction for any model in the library. BrandNewBertModel
inherits from BrandNewBertPreTrainedModel which in turn inherits from [PreTrainedModel] and
that's it. As a general rule, we want to make sure that a new model only depends on
[PreTrainedModel]. The important functionalities that are automatically provided to every new
model are [~PreTrainedModel.from_pretrained] and
[~PreTrainedModel.save_pretrained], which are used for serialization and deserialization. All of the
other important functionalities, such as BrandNewBertModel.forward should be completely defined in the new
modeling_brand_new_bert.py script. Next, we want to make sure that a model with a specific head layer, such as
BrandNewBertForMaskedLM does not inherit from BrandNewBertModel, but rather uses BrandNewBertModel
as a component that can be called in its forward pass to keep the level of abstraction low. Every new model requires a
configuration class, called BrandNewBertConfig. This configuration is always stored as an attribute in
[PreTrainedModel], and thus can be accessed via the config attribute for all classes
inheriting from BrandNewBertPreTrainedModel:
python
model = BrandNewBertModel.from_pretrained("brandy/brand_new_bert")
model.config  # model has access to its config
Similar to the model, the configuration inherits basic serialization and deserialization functionalities from
[PretrainedConfig]. Note that the configuration and the model are always serialized into two
different formats - the model to a pytorch_model.bin file and the configuration to a config.json file. Calling
[~PreTrainedModel.save_pretrained] will automatically call
[~PretrainedConfig.save_pretrained], so that both model and configuration are saved.
Code style
When coding your new model, keep in mind that Transformers is an opinionated library and we have a few quirks of our
own regarding how code should be written :-)