code: def attention_lm_moe_memory_efficient():
    
    hparams = attention_lm_moe_large()
    hparams.diet_experts = True
    hparams.layer_preprocess_sequence = "n"
    hparams.layer_postprocess_sequence = "da"
    hparams.layer_prepostprocess_dropout = 0.0
    hparams.memory_efficient_ffn = True
    hparams.attention_type = AttentionType.MEMORY_EFFICIENT
    hparams.num_heads = 8
    hparams.factored_logits = True
    return hparams
docs: Memory-efficient version.