Support for Mpnet models
Hi there, wondering if it will be possible to quantize Mpnet models for SentenceTransformers soon?
2022-08-08T14:05:27Z
Hi@nickmuchi, thanks for pointing that out! We would love to add support for Mpnet and other new models, are you interested in opening a PR to add Mpnet?Basically, to add the support for a new model there are3 stepsto follow and potentially2 PRsin two repos will be needed:[transformers]Add OnnxConfig in transformers → which enables the export to ONNX format.Detailed guide on adding unsupported modelsYou can find more discussionhereRegister supported tasks of the model intotransformers.onnx.FeaturesManager, by far you will be able to export the model to ONNX / useORTModelsfor inference / useORTQuantizerfor quantization.[Optimum]If you are also interested in usingORTOptimizerto do some graph optimizations, then you shall open a PR in optimum to add the support of Mpnet inORTConfigManagerPlease feel free to reach out if you have any further questions, and I can guide you through this if you wish to contribute.
2022-08-08T14:31:11Z
Hi@JingyaI have never opened a PR before but would love to if you are able to guide me please. Thanks.