Flan-T5 / T5: what is the difference between AutoModelForSeq2SeqLM and T5ForConditionalGeneration
I’ve been playing around with the new Flan-T5 model and there seem to be different (contradictory?) pieces of information on how to run it.Themodel carduses the following classes:from transformers import T5Tokenizer, T5ForConditionalGenerationTheFLAN-T5 docsuse these classes:from transformers import AutoModelForSeq2SeqLM, AutoTokenizerAt the same time the FLAN-T5 docs refer to the originalT5 docsfor advice on fine-tuning and there the other classes are used again:from transformers import T5Tokenizer, T5ForConditionalGenerationAt the same time, the most comprehensive guide for fine-tuning(m)T5 is the summarization tutorialin the HF course, which uses the Seq2Seq class again:from transformers import AutoModelForSeq2SeqLMI understand the difference between the Auto… classes and the model specific T5 classes, but I’m not sure what the difference between ConditionalGeneration and Seq2Seq is, in terms of practical usage. My previous understanding was that the ConditionalGeneration classes are for GPT-like autoregressive/next-token-prediction models, while the Seq2Seq classes are for T5-like models (pre-trained with a masking objective). What’s confusion to me is that, both classes seem to work with versions of T5 (but not always with all versions).=>  Should one useAutoModelForSeq2SeqLMorT5ForConditionalGenerationfor FLAN-T5?What is the main difference between the two classes? (My use-case is specifically about fine-tuning FLAN-T5).
2023-02-01T04:15:03Z
@MoritzLaurerAny updates on this particular issue? Also can you share any reference tutorials/code/doc that will be helpful regarding this issue
2023-02-01T08:16:27Z
Hi@FrozenWolf, no, I haven’t found an answer to this question unfortunately. The links in the question above are the main resources I found in this regard
2023-02-01T10:43:51Z
I’ve run into the same issue as well. Actually the most up-to-date tutorial about flan-t5 I believe is this:Fine-tune FLAN-T5 for chat & dialogue summarizationby@philschmid, maybe he can tell us the differences?
2023-02-01T12:47:36Z
Hi@MoritzLaurerThanks for the issue and for your message. If I understood correctly the problem here is whether to useT5ForConditionalGenerationorAutoModelForSeq2SeqLMfor flan-t5 or t5 in general.One simple check that you can do is to run the script below:from transformers import AutoModelForSeq2SeqLM, T5ForConditionalGeneration

model = AutoModelForSeq2SeqLM.from_pretrained("t5-small")
print(model.__class__.__name__)

model = T5ForConditionalGeneration.from_pretrained("t5-small")
print(model.__class__.__name__)And you will observe the output:T5ForConditionalGeneration
T5ForConditionalGenerationAutoModelForSeq2SeqLMenables to load the correctseq2seqclass given a checkpoint. The automapping class will retrieve the correct class fromthis list. Therefore there is no practical difference betweenxxxForConditionalGenerationclasses andAutoModelForSeq2SeqLM, since they are the same.For decoder-based models (e.g. GPT2), one should usexxxForCausalLMclasses (orAutoModelForCausalLM.Flan-t5 is not a new architecture itself, it is a series of t5 models fine-tuned in a different manner thanT5. Therefore you can useT5ForConditionalGenerationorAutoModelForSeq2SeqLM.
2023-02-02T10:47:31Z
Great, thanks for the response, good to know that it’s effectively the same (although that seems a bit confusing)