How to add additional module to BERT architecture, then load the original weight and use it
I aim toAdd an additional module to BERT architecture. In detail, each layer should cross product a similar but not identical vector, depending on the index of layer and the type of BERT modelLoad the BERT’s weight to this new BERT modelThen use BERT directly or continue train BERTI’m very confused how to do it. Since we usually only load the configuration and weights of model from huggingface directly. This might be done viamodeling_bert.py(afterhidden_statesof each layer), but now can the vector be different depending on different model types?In more detail, I’m working on bothprajjwal1/bert-tiny(now) andbert-base-uncased(next step).