Make the sentence clear: This design removes the addition over heterogeneous embeddings in the input, which may potentially bring randomness, and gives more expressiveness to characterize the relationship between words/positions by using different projection matrices.
This design removes the noisy word-position correlation and gives more expressiveness to characterize the relationship between words/positions by using different projection matrices.