Custom training - tokenization via collate fn or __getitem__?
hello,I am finetuning clip with 1.1TB of image and text pairs using pytorch lighting fabric and mosaic-ml streaming datasets to load the data from multiple shards. When loading I have the option to apply the tokenizer and processor for text and images respectively in__getitem__or using them via colllate_fn to do it batch-wise staking them afterwards.My question is, what is more recommended, apply transformations/tokenizers incollate_fnor__getitem__? I have seen very few examples online of people applying tokenizers/processors viacollate_fn(theoretically it should be faster than in__getitem__)I tried to look theViT exampleand it seems the transformations are done on the fly and then collate fn to stack, should I follow the same to gain training speed and reduce memory footprint?thank you in advancebest,