best_trials = trainer.hyperparameter_search(
     direction=["minimize", "maximize"],
     backend="optuna",
     hp_space=optuna_hp_space,
     n_trials=20,
     compute_objective=compute_objective,
 )

For raytune, see raytune object_parameter, it's like following:

def ray_hp_space(trial):
     return {
         "learning_rate": tune.loguniform(1e-6, 1e-4),
         "per_device_train_batch_size": tune.choice([16, 32, 64, 128]),
     }

For wandb, see wandb object_parameter, it's like following: