

Custom model builds
You can modify the model's configuration class to change how a model is built. The configuration specifies a model's attributes, such as the number of hidden layers or attention heads. You start from scratch when you initialize a model from a custom configuration class. The model attributes are randomly initialized, and you'll need to train the model before you can use it to get meaningful results.
Start by importing [AutoConfig], and then load the pretrained model you want to modify. Within [AutoConfig.from_pretrained], you can specify the attribute you want to change, such as the number of attention heads: