Optimizing models using ONNX
I came to know Hugging Face use optimized onnx models for inference on cpu. I tried to do something like that using keras VGGNet16 pretrained model using keras-onnx package (see this github issue) butcouldn’tsee any performance benefits. Can I know how exactly Hugging Face is optimizing models under the hood?
2020-10-21T07:42:49Z
These two blog posts might helpMedium – 17 Jun 20Accelerate your NLP pipelines using Hugging Face Transformers and ONNX RuntimeThis post was written by Morgan Funtowicz from Hugging Face and Tianlei Wu from MicrosoftReading time: 6 min readMedium – 20 Sep 21Faster and smaller quantized NLP with Hugging Face and ONNX RuntimePopular Hugging Face Transformer models (BERT, GPT-2, etc) can be shrunk and accelerated with ONNX Runtime quantization without retraining.Reading time: 6 min read