Chatbot using Reformer model. Not able to find "token_type_ids" like BERT
I want to use Reformer model as Chatbot.So in order to train it along with the input_ids and lm_labels. I am looking for token_type_ids to encode the context, speaker1 and speaker2 information.But from the documentation of Reformer*class* `transformers.` `ReformerForMaskedLM` 
`forward` ( *input_ids=None* ,  *position_ids=None* ,  *attention_mask=None* ,  *head_mask=None* ,  *inputs_embeds=None* ,  *num_hashes=None* ,  *labels=None* ,  *output_hidden_states=None* ,  *output_attentions=None* ,  *return_dict=None* )I do not see “token_type_ids”. How do I fine-tune this model to work as a chatbot?Also, How do I train this model from scratch in some other language for example German.
2020-09-24T15:34:42Z
there are notoken_type_idsfor reformer, you could some special delimiter tokens to separate context and speakers.Thisarticlecould help