

git fetch upstream
git merge upstream/main
In general, all questions you might have regarding the model or your implementation should be asked in your PR and
discussed/solved in the PR. This way, the Hugging Face team will always be notified when you are committing new code or
if you have a question. It is often very helpful to point the Hugging Face team to your added code so that the Hugging
Face team can efficiently understand your problem or question.
To do so, you can go to the “Files changed” tab where you see all of your changes, go to a line regarding which you
want to ask a question, and click on the “+” symbol to add a comment. Whenever a question or problem has been solved,
you can click on the “Resolve” button of the created comment.
In the same way, the Hugging Face team will open comments when reviewing your code. We recommend asking most questions
on GitHub on your PR. For some very general questions that are not very useful for the public, feel free to ping the
Hugging Face team by Slack or email.
5. Adapt the generated models code for brand_new_bert
At first, we will focus only on the model itself and not care about the tokenizer. All the relevant code should be
found in the generated files src/transformers/models/brand_new_bert/modeling_brand_new_bert.py and
src/transformers/models/brand_new_bert/configuration_brand_new_bert.py.
Now you can finally start coding :). The generated code in
src/transformers/models/brand_new_bert/modeling_brand_new_bert.py will either have the same architecture as BERT if
it's an encoder-only model or BART if it's an encoder-decoder model. At this point, you should remind yourself what
you've learned in the beginning about the theoretical aspects of the model: How is the model different from BERT or
BART?". Implement those changes which often means changing the self-attention layer, the order of the normalization
layer, etc… Again, it is often useful to look at the similar architecture of already existing models in Transformers to
get a better feeling of how your model should be implemented.
Note that at this point, you don't have to be very sure that your code is fully correct or clean. Rather, it is
advised to add a first unclean, copy-pasted version of the original code to
src/transformers/models/brand_new_bert/modeling_brand_new_bert.py until you feel like all the necessary code is
added. From our experience, it is much more efficient to quickly add a first version of the required code and
improve/correct the code iteratively with the conversion script as described in the next section. The only thing that
has to work at this point is that you can instantiate the 🤗 Transformers implementation of brand_new_bert, i.e. the
following command should work:
thon
from transformers import BrandNewBertModel, BrandNewBertConfig
model = BrandNewBertModel(BrandNewBertConfig())