Optimum vs Accelerate
What are the key differences between HF Accelerate and HF Optimum?  Can they be used together?
2023-02-27T10:36:43Z
Hi@jeromeku,Accelerate is mainly used for distributed training on a variety of hardware, with techniques as offloading and data parallelism.Optimum is more a toolkit, giving access to several tools either geared towards speeding up training or inference, or towards deployment:ONNX, TFLite export of Transformers and Diffusers models.ONNX Runtime integration (including quantization, ORT optimizations)PyTorch’s BetterTransformerintegration (this one is compatible with accelerate!)Integration with OpenVINO, NNCF, Intel Neural Compressor (including quantization & pruning)Integration with Habana Gaudi hardware for trainingIntegration with Graphcore IPUs for trainingFeel free to refer to the documentations:🤗 Optimum&Accelerate
2023-02-27T11:02:28Z
Just to extend the previous answer, Optimum.onnxruntime module also supports training acceleration withORTTrainerclass.
2023-03-02T04:20:54Z
Thanks for the responses!  SoAccelerateis a general framework for distributed training / inference whileOptimumis oriented more towards architecture-specific optimizations which can’t be easily abstracted under a single framework.  Would be interesting to compare / learn from other optimization frameworks (e.g., ColossalAI, Alpa) – let me know how I can contribute!
2023-03-02T13:53:44Z
For sure comparing with what other framework propose is interesting! There are so many out there that it is easy to get lost. On the inference side, I think a comprehensive benchmark across the most-popular deployment tools is still something to be done.
2023-03-02T23:56:11Z
Great! Is there a working group around this currently? Perhaps a good starting point is to compile a list of frameworks bucketed by training and inference as well as currently available evaluation datasets / leaderboards by domain / dataset.