A new model for the Hub depending on another Hub model
Hi,I have properly prototyped a new architectureAthat internally uses a frozen generative language model from the Hub, let’s call itB, and extensively builds on top of it. SubclassingBis not an option, asAmay use virtually any generative language model.I would like to make a few example pre-trained versions ofApushable to the Hub. For example, I would like to be able to have a pre-trained version ofAdepending on (frozen)B=OpenAI GPT, another version ofAdepending onB=Facebook OPT, etc…The simplest thing to do would be to store an identifier of the dependency modelBas a string and then load it from the Hub every timeAis being loaded from its own pre-trained weights. But what ifBis updated? My weights inArely on thehidden_statescoming fromBand thus the (now old) weights ofAwould immediately become useless.What I seem to need to be able to do is to push a particular version ofBtogether with the weights of myA. This would somehow require nested configs and nested model weight loading.Before I embark on trying to hack the heck out of the transformers library, any suggestions on how to approach this in a clean manner?