code: def transformer_base_v3():
    
    # Update parameters here, then occasionally cut a versioned set, e.g.
    # transformer_base_v2.
    hparams = transformer_base_v2()
    hparams.optimizer_adam_beta2 = 0.997
    # New way of specifying learning rate schedule.
    # Equivalent to previous version.
    hparams.learning_rate_schedule = (
        "constant*linear_warmup*rsqrt_decay*rsqrt_hidden_size")
    hparams.learning_rate_constant = 2.0
    return hparams
docs: Base parameters for Transformer model.