Confusion About T5LM Properties
Hello,I’m working on using T5LM (small) for semantic parsing tasks. However, there seems to be discrepancies between the model properties as listed on the T5 documentation page:https://huggingface.co/docs/transformers/model_doc/t5versus the error outputs I get from trying to run the Supervised Training example. More specifically, when running the last line:loss = model(input_ids=input_ids, labels=labels).lossI get the error saying the model needs eitherdecoder_inputs_embedsordecoder_input_idsin place of thelabelsparameter. Additionally, the tutorial text states:The model will automatically create thedecoder_input_idsbased on thelabels,…Is there source code for this model implementation that I can reference? I don’t understand the difference between the embeds vs decoder input ids for a start. And I can’t tell if the model I imported usingmodel = transformers.AutoModel.from_pretrained(“google/t5-small-lm-adapt”)is correct because in addition to the above discrepancy, when I run the original loss command, I got the error thatSeq2SeqModelOutputdoes not have a.lossfunction.I thought the LM adaptation of T5 would be 1-to-1 to the original T5 architecture, but that clearly doesn’t seem to be the case. Any help or direction would be greatly appreciated.Thanks,Selma
2022-01-07T18:41:44Z
I now understand the difference betweendecoder_inputs_embedsand thedecoder_input_ids; however I’m still confused about the difference between thegoogle/t5-small-lm-adaptthat is being loaded from AutoModelvsTheT5ConditionalGenerationmodel supplied by hugging face