code: def adaptive_universal_transformer_base_range(rhp):
    
    # After starting from base, set intervals for some parameters.
    rhp.set_discrete("act_max_steps", [8, 16, 32])
    rhp.set_float("act_loss_weight", 0.0, 0.5)
    rhp.set_discrete("hidden_size", [1024, 2048, 4096])
    rhp.set_discrete("filter_size", [2048, 4096, 8192])
    rhp.set_discrete("num_heads", [8, 16, 32])
    rhp.set_discrete("transformer_ffn_type", ["sepconv", "fc"])
    rhp.set_float("learning_rate", 0.3, 3.0, scale=rhp.LOG_SCALE)
    rhp.set_float("weight_decay", 0.0, 2.0)
docs: Range of hyperparameters.