code: def attention_mask_autoregressive(query_pos, dtype=tf.float32):
    
    memory_pos = rename_length_to_memory_length(query_pos)
    return mtf.cast(mtf.less(query_pos, memory_pos), dtype) * -1e9
docs: Bias for self-attention where attention to the right is disallowed.
  
    Args:
      query_pos: a mtf.Tensor with shape [..., length_dim]
      dtype: a tf.dtype
  
    Returns:
      a mtf.Tensor with shape [..., length_dim, memory_length_dim]