Max_new_tokens warning for Flan-T5 fine-tuning
I am fine-tuning a Google/Flan-T5-base model but while training, getting this warning:“UserWarning: Using the model-agnostic defaultmax_length(=20) to control the generation length. We recommend settingmax_new_tokensto control the maximum length of the generation”The Training code is as follows:L_RATE = 3e-4BATCH_SIZE = 32PER_DEVICE_EVAL_BATCH = 32WEIGHT_DECAY = 0.01SAVE_TOTAL_LIM = 3NUM_EPOCHS = 2training_args = Seq2SeqTrainingArguments(output_dir=“./results”,evaluation_strategy=“epoch”,learning_rate=L_RATE,per_device_train_batch_size=BATCH_SIZE,per_device_eval_batch_size=PER_DEVICE_EVAL_BATCH,weight_decay=WEIGHT_DECAY,save_total_limit=SAVE_TOTAL_LIM,num_train_epochs=NUM_EPOCHS,predict_with_generate=True,push_to_hub=False)trainer = Seq2SeqTrainer(model=model,args=training_args,train_dataset=tokenized_dataset[“train”],eval_dataset=tokenized_dataset[“test”],tokenizer=tokenizer,data_collator=data_collator,compute_metrics=compute_metrics)I cannot find where should I specify the max_new_tokens as it does not work when specified in training_args : Seq2SeqTrainingArguments.init() got an unexpected keyword argument ‘max_new_tokens’
2024-05-03T08:13:29Z
Hi! AFAIK the only way to set generation parameters for the trainer is directly through the “generation_config” of the model, before passing it to “trainer”model.generation_config.max_new_tokens = 50 # generate exactly 50 tokens when predicting
model.generation_config.min_new_tokens = 50
trainer = Seq2SeqTrainer(model=model, ...)
2024-05-06T13:26:30Z
Thank you very much for the response!