

Now you have set up a development environment to port BrandNewBert to TensorFlow in 🤗 Transformers.
3. (Optional) Understand theoretical aspects and the existing implementation
You should take some time to read BrandNewBert's paper, if such descriptive work exists. There might be large
sections of the paper that are difficult to understand. If this is the case, this is fine - don't worry! The goal is
not to get a deep theoretical understanding of the paper, but to extract the necessary information required to
effectively re-implement the model in 🤗 Transformers using TensorFlow. That being said, you don't have to spend too
much time on the theoretical aspects, but rather focus on the practical ones, namely the existing model documentation
page (e.g. model docs for BERT).
After you've grasped the basics of the models you are about to implement, it's important to understand the existing
implementation. This is a great chance to confirm that a working implementation matches your expectations for the
model, as well as to foresee technical challenges on the TensorFlow side.
It's perfectly natural that you feel overwhelmed with the amount of information that you've just absorbed. It is
definitely not a requirement that you understand all facets of the model at this stage. Nevertheless, we highly
encourage you to clear any pressing questions in our forum.
4. Model implementation
Now it's time to finally start coding. Our suggested starting point is the PyTorch file itself: copy the contents of
modeling_brand_new_bert.py inside src/transformers/models/brand_new_bert/ into
modeling_tf_brand_new_bert.py. The goal of this section is to modify the file and update the import structure of
🤗 Transformers such that you can import TFBrandNewBert and
TFBrandNewBert.from_pretrained(model_repo, from_pt=True) successfully loads a working TensorFlow BrandNewBert model.
Sadly, there is no prescription to convert a PyTorch model into TensorFlow. You can, however, follow our selection of
tips to make the process as smooth as possible:
- Prepend TF to the name of all classes (e.g. BrandNewBert becomes TFBrandNewBert).
- Most PyTorch operations have a direct TensorFlow replacement. For example, torch.nn.Linear corresponds to
  tf.keras.layers.Dense, torch.nn.Dropout corresponds to tf.keras.layers.Dropout, etc. If you're not sure
  about a specific operation, you can use the TensorFlow documentation
  or the PyTorch documentation.
- Look for patterns in the 🤗 Transformers codebase. If you come across a certain operation that doesn't have a direct
   replacement, the odds are that someone else already had the same problem.
- By default, keep the same variable names and structure as in PyTorch. This will make it easier to debug, track
   issues, and add fixes down the line.
- Some layers have different default values in each framework. A notable example is the batch normalization layer's
   epsilon (1e-5 in PyTorch
   and 1e-3 in TensorFlow).
   Double-check the documentation!
- PyTorch's nn.Parameter variables typically need to be initialized within TF Layer's build(). See the following
   example: PyTorch /
   TensorFlow
- If the PyTorch model has a #copied from  on top of a function, the odds are that your TensorFlow model can also
   borrow that function from the architecture it was copied from, assuming it has a TensorFlow architecture.
- Assigning the name attribute correctly in TensorFlow functions is critical to do the from_pt=True weight
   cross-loading. name is almost always the name of the corresponding variable in the PyTorch code. If name is not
   properly set, you will see it in the error message when loading the model weights.
- The logic of the base model class, BrandNewBertModel, will actually reside in TFBrandNewBertMainLayer, a Keras
   layer subclass (example).
   TFBrandNewBertModel will simply be a wrapper around this layer.
- Keras models need to be built in order to load pretrained weights. For that reason, TFBrandNewBertPreTrainedModel
   will need to hold an example of inputs to the model, the dummy_inputs
   (example).
- If you get stuck, ask for help - we're here to help you! 🤗
In addition to the model file itself, you will also need to add the pointers to the model classes and related
documentation pages. You can complete this part entirely following the patterns in other PRs
(example). Here's a list of the needed manual
changes:
- Include all public classes of BrandNewBert in src/transformers/__init__.py
- Add BrandNewBert classes to the corresponding Auto classes in src/transformers/models/auto/modeling_tf_auto.py
- Add the lazy loading classes related to BrandNewBert in src/transformers/utils/dummy_tf_objects.py
- Update the import structures for the public classes in src/transformers/models/brand_new_bert/__init__.py
- Add the documentation pointers to the public methods of BrandNewBert in docs/source/en/model_doc/brand_new_bert.md
- Add yourself to the list of contributors to BrandNewBert in docs/source/en/model_doc/brand_new_bert.md
- Finally, add a green tick ✅ to the TensorFlow column of BrandNewBert in docs/source/en/index.md
When you're happy with your implementation, run the following checklist to confirm that your model architecture is
ready:
1. All layers that behave differently at train time (e.g. Dropout) are called with a training argument, which is
propagated all the way from the top-level classes
2. You have used #copied from  whenever possible
3. TFBrandNewBertMainLayer and all classes that use it have their call function decorated with @unpack_inputs
4. TFBrandNewBertMainLayer is decorated with @keras_serializable
5. A TensorFlow model can be loaded from PyTorch weights using TFBrandNewBert.from_pretrained(model_repo, from_pt=True)
6. You can call the TensorFlow model using the expected input format
5. Add model tests
Hurray, you've implemented a TensorFlow model! Now it's time to add tests to make sure that your model behaves as
expected. As in the previous section, we suggest you start by copying the test_modeling_brand_new_bert.py file in
tests/models/brand_new_bert/ into test_modeling_tf_brand_new_bert.py, and continue by making the necessary
TensorFlow replacements. For now, in all .from_pretrained() calls, you should use the from_pt=True flag to load
the existing PyTorch weights.
After you're done, it's time for the moment of truth: run the tests! 😬