Tokenizer for Translation Pipeline with Bert2Bert EncoderDecoder
Hello,I would like to build a translation pipeline with a Bert2Bert EncoderDecoder model.Calling an input tokenizer, then the custom Bert2Bert model, and finally the output tokenizer goes fine. However building a pipeline is very hard because of the tokenizer: the pipeline function accepts only 1 tokenizer:generator = pipeline(task=“text-generation”, model=model, tokenizer=tokenizer)How can I build a “bilingual” BertTokenizerFast which encodes french inputs in regular context, and decodes english targets under the as_target_tokenizer() context ? (just like the MarianMT tokenizer)Any help would be esteemed deeply