Transformers.onnx vs optimum.onnxruntime
Hello. I am interested in converting a model to ONNX to get faster inference, but I saw there are two possible approaches:Using transformers.onnx package:Export 🤗 Transformers ModelsUsing optimum.onnxruntime package:Optimum Inference with ONNX RuntimeShould I convert the model to ONNX with the first and then use it with Optimum? It looks like Optimum can convert models to ONNX by its own now, so what is the point of transformers.onnx package?
2022-09-12T10:09:21Z
Hi@Maxinho,ORTModelAPIs in Optimum manage the conversion of models from PyTorch to ONNX(we currently use the export intransformers.onnx) when it is needed, and implement the inference for different tasks so that you can use it just like usingAutoModelAPIs in Transformers.In terms of acceleration, Optimum offersORTOptimizerandORTQuantizer, with which you can optimize your computation graph and quantize your ONNX model so that you can accelerate even more the inference.