Transformers Tokenizer on GPU?
Hi, I am finding the tokenizing takes long time when I have large text data. There may be some documentation about this somewhere, but I could not find any that address how to use multiple GPUs to process the tokenization. Any help will be much appreciated. Thanks!
2020-12-17T03:29:49Z
@Narsilmight help be able to help here
2020-12-17T08:39:49Z
Hi@jaecha.Tokenization does not happen on GPU (and won’t anytime soon). If you can show your tokenizer config that could help understand why it takes a long time ? tokenizers version, what kind of model and maybe how large your data ?If you see a warning about TOKENIZERS_PARALLELISM in your console:You use multiple threads (like withDataLoader) then it’s better to create a tokenizer instance on each thread rather than before the fork otherwise we can’t use multiple cores (because of GIL)Having a good pre_tokenizer is important (usuallyWhitespacesplitting for languages that allow it) at least.You can find more information about your options here:https://huggingface.co/docs/tokenizers/python/latest/pipeline.html#pre-tokenizationDoes that help?CheersNicolas
2020-12-17T14:28:16Z
Thank you for your replies! I think I know what to do next. Thanks!