[Nov 16th Event] Matthew Carrigan: New TensorFlow Features for 🤗 Transformers and Datasets
Use this topic to ask your questions to Matthew Carrigan during his talk:New TensorFlow Features forTransformers andDatasetsYou can watch it onYouTubeor onTwitchat 8:30am PST
2021-11-16T16:40:42Z
Could the notebook shown in the video be linked?
2021-11-16T16:43:25Z
Sure thing, here’s aColab link!
2021-11-16T16:49:28Z
For something like a zero-shot model, how does TFAutomodelforSequenceClassification change?
2021-11-16T16:50:51Z
For something like a zero-shot model, how does TFAutomodelforSequenceClassification change?This is answered at 50:06 in the main stream.
2021-11-16T16:53:19Z
I understand that padding enables batching data points together, but too many padding tokens make the computations very slow (?). What is the attention_mask for? If I understand correctly, it masks the padding tokens so that the model does not pay attention to it - but do padding tokens then still slow down the training if they are masked? I haven’t fully understood the purpose of padding and attention_masks and the impact on speed.
2021-11-16T16:58:31Z
MoritzLaurer:I understand that padding enables batching data points together, but too many padding tokens make the computations very slow (?). What is the attention_mask for? If I understand correctly, it masks the padding tokens so that the model does not pay attention to it - but do padding tokens then still slow down the training if they are masked? I haven’t fully understood the purpose of padding and attention_masks and the impact on speed.This is answered at 55:00 on the main stream.
2021-11-16T19:33:17Z
Hi all, just noticed the Colab notebook didn’t have permissions set. It should be accessible now!