Pretrained model attributes can be modified in the [~PretrainedConfig.from_pretrained] function:

my_config = DistilBertConfig.from_pretrained("distilbert/distilbert-base-uncased", activation="relu", attention_dropout=0.4)

Once you are satisfied with your model configuration, you can save it with [~PretrainedConfig.save_pretrained]. Your configuration file is stored as a JSON file in the specified save directory:

my_config.save_pretrained(save_directory="./your_model_save_path")