Is there a way to return the "decoder_input_ids" from "tokenizer.prepare_seq2seq_batch"?
Using BART as an example … this:tokenizer.prepare_seq2seq_batch(
    src_texts=['This is a very short text', 'This is shorter'],
    tgt_texts=['very short', 'much shorter than very short'])returns …{'input_ids': [[100, 19, 3, 9, 182, 710, 1499, 1], [947, 19, 10951, 1, 0, 0, 0, 0]], 
'attention_mask': [[1, 1, 1, 1, 1, 1, 1, 1], [1, 1, 1, 1, 0, 0, 0, 0]], 
'labels': [[182, 710, 1, 0, 0, 0], [231, 10951, 145, 182, 710, 1]]}For fine-tuning, how should we build thedecoder_input_ids? And do we also need to shift thelabelsto the right so that they look like this?[[710, 1, 0, 0, 0, 0], [10951, 145, 182, 710, 1, 0]]
2020-12-23T01:57:17Z
Or do we even have to passdecoder_input_idsanymore???Looking at this example for MT5, it looks like hte answer is “no” …from transformers import MT5ForConditionalGeneration, T5Tokenizer
model = MT5ForConditionalGeneration.from_pretrained("google/mt5-small")
tokenizer = T5Tokenizer.from_pretrained("google/mt5-small")
article = "UN Offizier sagt, dass weiter verhandelt werden muss in Syrien."
summary = "Weiter Verhandlung in Syrien."
batch = tokenizer.prepare_seq2seq_batch(src_texts=[article], tgt_texts=[summary], return_tensors="pt")
outputs = model(**batch)
loss = outputs.lossThis sure would make it easier if all we have to pass in are the “labels” and not have to deal with thedecoder_input_idsourselves when working within ConditionalGeneration models.  Please lmk either way.Thanks
2020-12-23T06:11:16Z
Hi@wgpubs,if you just passlabelsthedecoder_input_idsare prepared inside the model by shifting thelabels. Seegithub.comhuggingface/transformers/blob/cbe63949d76efd153a1f389f38fe9ce1287e06b0/src/transformers/models/bart/modeling_bart.py#L1229-L1231if decoder_input_ids is None:decoder_input_ids = shift_tokens_right(labels, self.config.pad_token_id)
2020-12-23T23:08:55Z
Ah cool … thanks for confirming.
2020-12-26T19:42:19Z
Are the target tokens (the labels) replaced with the ignore token id somewhere as well?  Doesn’t look like it from what I can see … so I’m assuming we need to do that ourselves, and pass the label ids with padding tokens set to -100.Also, thedecoder_input_idscome back in the form of<eos> <bos> X ..., but my understanding was always that it should start with<bos>and thelabelsshifted so that<bos>attempts to predictX[0]and so forth.
2020-12-29T06:47:44Z
Yes, we should manually replace pad token with -100 inlabels.Ideally yes, it should start withbostoken, but in the originalfairseqimplementation the models are trained with<eos> <bos> X ...., so we have kept it like that for reproducibility.