Adding attention mask into MLM
Hi all, I’m using this .py file (sentence-transformers/examples/unsupervised_learning/MLM at master · UKPLab/sentence-transformers · GitHub) to carry out some unsupervised training.It’s returning a message saying: ‘We strongly recommend passing in anattention_masksince your input_ids may be padded. SeeTroubleshoot’I’m a little unfamiliar with this process, but my guess is that it may be due to the fact that my training data text file has every row varying in length.How would I adapt this to add in an attention mask? Thanks!
2024-05-30T12:47:20Z
To add to this, my code is killed following the message above, and no config.json file is created in the sentence transformer