code: def mtf_unitransformer_all_layers_tiny():
    
    hparams = mtf_unitransformer_tiny()
    hparams.moe_num_experts = 4
    hparams.moe_expert_x = 4
    hparams.moe_expert_y = 4
    hparams.moe_hidden_size = 512
    hparams.layers = ["self_att", "local_self_att", "moe_1d", "moe_2d", "drd"]
    return hparams
docs: Test out all the layers on local CPU.