code: def attention_lm_small():
    
    hparams = attention_lm_base()
    hparams.num_hidden_layers = 4
    hparams.hidden_size = 512
    hparams.filter_size = 2048
    hparams.layer_prepostprocess_dropout = 0.5
    return hparams
docs: Cheap model.
  
    on lm1b_32k:
       45M params
       2 steps/sec on  [GeForce GTX TITAN X]
  
    Returns:
      an hparams object.