Transformer loss
Hi!I am now doing a bi-level training, which means that there are two models, one for generating data and the other one is trained on the genrated data.In that case, my generated data are one-hot encoded, and I want to use that one-hot encoded label to train my second model.However, the forward function in BART and T5 are likeforward(input,...label,....  decoder_inputs_embeds....decoder_input...), and the ‘label’ only accept index instead of one-hot encoded label. Therefore, I manually embed the label and pass them todecoder_inputs_embeds. Is that the same as pass these one-hot encoded value to ‘label’?