How do tokenizer(text_target=text) work
Hello everyone, I’m doing on translating task and I don’t understand how tokenizer switches specific-tokenizer for source language and target language (eg: ‘en’ and ‘fr’). Because I see they read in the same vocabulary file. Thank you for any help.