BartForConditionalGeneration "logits" shape is wrong/unexpected
UsingBartForConditionalGenerationwith a batch_size = 2 … all the inputs look right but when I examine thelogitsthe shape istorch.Size([2, 1, 50264])The inputs are:x['input_ids'].shape => torch.Size([2, 256])
x['attention_mask'].shape => torch.Size([2, 256])
x['decoder_input_ids'].shape => torch.Size([2, 68])What may I be doing wrong? Or is there a bug in the model?
2020-11-11T20:52:29Z
Could you give us the command you produce the logits? Did you just call the model or using model.generate?Btw, 50264 is Bart vocab_size
2020-11-11T20:55:48Z
It seems the problem occurs when passingdecoder_input_ids.  Here’s my code:logits = self.REDACTED_TOKEN(x['input_ids'], x['attention_mask'], x['decoder_input_ids'], labels=None, return_dict=True).logitswhereself.REDACTED_TOKEN an instance ofBartForConditionalGeneration.returns a tensor with shapetorch.Size([2, 1, 50264])when the expected istorch.Size([2, 68, 50264])
2020-11-11T21:33:50Z
SOLVEDApparently, if even you are calculating the loss on your own, you have to pass inlabelsif you pass ingdecoder_input_ids.Ya’ll may want to add this to the docs or update the behavior to not require thelabelsargument.Thanks - wg
2020-11-11T21:57:15Z
Great that you solved it!If you want to compute loss on your own, perhaps you can useBartModel(does not require labels) instead ofBartForConditionalGeneration=======================About the original question, to me, the following command gave the correct outputs & shape (I used TF). But it seems we need ‘max_length’ paddingmodel = TFBartForConditionalGeneration.from_pretrained('facebook/bart-large-cnn')
tokenizer = BartTokenizer.from_pretrained('facebook/bart-large-cnn')

src_texts = ['My friends are cool but they eat too many carbs. I really want them to be healthy, so I buy them vegetable.']
tgt_texts = ['I buy them vegetable.']
x = tokenizer.prepare_seq2seq_batch(src_texts, tgt_texts, return_tensors='tf',padding='max_length')

out = model(x)Note In my case, padding has to be specified as ‘max_length’ , where in the other case, model calling failed.