I’m using Trainer to handle finetuning a GPT2 model. I see in TrainingArguments there is a max_steps that overrides num_train_epochs. For a batch size of 32, is setting max_steps=1000000 the equivalent of setting num_train_epochs=31250? Also, what happens if I have a batch size of 6 but want to se…

Explicitly set number of training steps using Trainer

sgugger September 14, 2020, 9:00pm 2

The number of steps is the number of update steps. It’s not the number of training examples to be seen.

Questions about steps with gradient accumulation