code: def mtf_image_transformer_base_cifar():
    
    hparams = mtf_image_transformer_base()
    hparams.mesh_shape = "batch:8"
    hparams.layout = "batch:batch"
    hparams.learning_rate_decay_steps = 13600  # one epoch
    hparams.batch_size = 32
    hparams.num_heads = 4
    hparams.num_decoder_layers = 12
    hparams.block_length = 256
    hparams.hidden_size = 512
    hparams.d_ff = 2048
    hparams.learning_rate = 0.5
    hparams.layer_preprocess_sequence = "none"
    hparams.layer_postprocess_sequence = "dan"
    hparams.layer_prepostprocess_dropout = 0.3
    hparams.unconditional = True
    return hparams
docs: Data parallel CIFAR parameters.