How to fine-tune a model from the Hub on a large dataset?
I’m trying to fine-tunethis Sentence Transformers model from the Hubon the Portuguese subset ofthis dataset.The dataset is fairly large (1 million triplets) and I’m running into memory issues in Google Colab. What is the best alternative here?So far I can think of these three, but I’m not sure which is the best:Fine-tune using a streaming dataset.Is this possible?Fine-tune using a smaller subset of the dataset.But I’m running into the same memory issues using even just 1% of the dataset…Pay for Google Colab Pro. But I’m not sure this will be enough.Any suggestions?
2022-11-03T08:20:32Z
Regards 1, didn’t try it myself but from reading in theload_datasetdocumentationit does seems like streaming dataset is possible by passingstreaming=True.Maybe you can read about it and try that.
2022-11-03T12:31:28Z
Yes, it is possible to read a dataset in streaming mode, what I don’t know if it’s possible to fine-tune a model using a streaming dataset