Tensorflow Models are way slower than Pytorch models, for autoregressive generation?
According to this post:Why Tensorflow Models are way slower than Pytorch models, for autoregressive modeling?, TF models are not optimized for NLG. Is it still the case?model.generatefor TF models is upto 80x slower for me compared to PyTorch models.
2022-07-26T08:31:15Z
cc@joaogante
2022-07-26T09:06:33Z
Hi@saurabh3949Your question comes with great timing! Tomorrow we will release a new version of transformers and one of its highlights is that TF NLG can now be massively accelerated, ever beyond PyTorch’s speed. There will be a blog post (also to be released tomorrow) explaining how, stay tuned!
2022-07-26T15:51:27Z
Thank you@joaogante! This is very encouraging. You have put me in eager mode now