Question about layer norm in T5
I notice that there is no bias and no subtraction of mean in layer norm.I’m confused about the meaning of computing variance without subtraction of mean.Normally, we compute variance, for example:image982×276 30.2 KBBut it’s different here. Why is that?
2023-03-16T05:49:57Z
arxiv.org1910.07467.pdf542.02 KBI found a paper explained about this in case anyone with the same question.