text
I have just published my first blog post.
I have just published my first blog post.
new_line
new_line
text
While FlashAttention has been readily integrated into HuggingFace transformers, there are much higher gains to be had (at least theoretically) for finetuning models with examples of variable sequence lengths in a batch.
While FlashAttention has been readily integrated into HuggingFace transformers, there are much higher gains to be had (at least theoretically) for finetuning models with examples of variable sequence lengths in a batch.
new_line
new_line
text
For a deeper dive, please go through my blog at 
For a deeper dive, please go through my blog at 
link
https://huggingface.co/blog/mayank-mishra/padding-free-transformer
https://huggingface.co/blog/mayank-mishra/padding-free-transformer
text
.
.
I have just published my first blog post.

While FlashAttention has been readily integrated into HuggingFace transformers, there are much higher gains to be had (at least theoretically) for finetuning models with examples of variable sequence lengths in a batch.

For a deeper dive, please go through my blog at https://huggingface.co/blog/mayank-mishra/padding-free-transformer.
❤️
Kukedlc osanseviero tomaarsen samusenps eramax mayank-mishra julien-c
🤝
damerajee julien-c mayank-mishra jeffboudier
https://cdn-avatars.huggingface.co/v1/production/uploads/6032802e1f993496bc14d9e3/w6hr-DEQot4VVkoyRIBiy.png
Omar Sanseviero
osanseviero
user
https://cdn-avatars.huggingface.co/v1/production/uploads/62a093d63e7d1dda047039fc/QGpVSKuJLwl2EsiffCYML.jpeg
Olivier Dehaene
olivierdehaene
user
https://cdn-avatars.huggingface.co/v1/production/uploads/62cd5057674cdb524450093d/f67rlrdsKPRTLdXCXoa_X.jpeg
Mayank Mishra
mayank-mishra
user
https://cdn-avatars.huggingface.co/v1/production/uploads/6487239cca30096ea9f52115/HMte9wjKJgfcxsO-5vb_Q.jpeg
dame rajee
damerajee
user
https://cdn-avatars.huggingface.co/v1/production/uploads/1641203017724-noauth.png
Joao Gante
joaogante
user
https://cdn-avatars.huggingface.co/v1/production/uploads/5dd96eb166059660ed1ee413/NQtzmrDdbG0H8qkZvRyGk.jpeg
Julien Chaumond
julien-c
user