code: def transformer_lm_tpu_0():
    
    hparams = transformer_clean_big()
    update_hparams_for_tpu(hparams)
    hparams.num_heads = 4  # Heads are expensive on TPUs.
    hparams.batch_size = 4096
    hparams.shared_embedding_and_softmax_weights = False
    hparams.layer_prepostprocess_dropout = 0.1
    return hparams
docs: HParams for training languagemodel_lm1b8k on tpu.  92M Params.