Early_stopping_patience param in EarlyStoppingCallback
Hi there,I am quite confused about the early_stopping_patience in EarlyStoppingCallback.Is it related to theevaluation_strategyin TrainingArgs?For example, when theevaluation_strategy=‘epoch’andearly_stopping_patience=8in TrainingArgs, the training will stop if the metrics/ loss does not improve/reduce after 8 epochs? And works the same whenevaluation_strategy=‘steps’.
2022-05-10T11:13:43Z
EarlyStoppingCallback is related withevaluation_strategyandmetric_for_best_model.early_stopping_patience(int) — Use withmetric_for_best_modelto stop training when the specified metric worsens forearly_stopping_patienceevaluation calls.I was confused  too whether to use it with evaluation_strategy=steps or epochs, but after some trials, I realized that it better to use it with epochs to grantee that model is trained on the whole dataset
2024-04-15T13:58:09Z
If you useearly_stopping_patienceinEarlyStoppingCallback, must:Passa functionmake evaluaton dict tocompute_metricsparam inTrainerclassUsemetric_for_best_modelto set evaluation key in compute_metrics like: mae, mse…Usegreater_is_betterto set this evaluation object greater or lower is better. For mae or mse, lower is better.My code:**def compute_metrics(eval_pred):**
            predictions, labels = eval_pred
            predictions = predictions[:, 0]
            mse = mean_squared_error(labels, predictions)
            mae = mean_absolute_error(labels, predictions)
            **return {"mse": mse, "mae": mae}**

        training_args = TrainingArguments(
            output_dir=f"{model_path.split('/')[-1]}_regression_finetuned_{output_name}",
            evaluation_strategy="epoch",
            save_strategy="epoch",
            save_total_limit=2,
            learning_rate=3e-5,
            per_device_train_batch_size=16,
            per_device_eval_batch_size=16,
            num_train_epochs=10,
            weight_decay=0.01,
            load_best_model_at_end=True,
            **metric_for_best_model="mae",**
            **greater_is_better=False,**
            warmup_steps=warmup_steps,
            lr_scheduler_type='cosine',
            logging_dir='./logs',
            logging_steps=50,
            push_to_hub=True,
            run_name='run_cosine_decay_regression',
            fp16=False,
            report_to="none"
        )

        trainer = Trainer(
            model=model,
            args=training_args,
            train_dataset=tokenized_dataset['train'],
            eval_dataset=tokenized_dataset['test'],
            **compute_metrics=compute_metrics**,
            tokenizer=tokenizer,
            data_collator=data_collator,
            **callbacks=[EarlyStoppingCallback(early_stopping_patience=2)]**
        )