Given a matrix $A \in \mathbb R^{n \times n}$ whose entries are i.i.d. $N(0,1)$, what is the expected value of its largest singular value? Equivalently, what is the expected value of the largest eigenvalue of $A'A$?



Expected value of the largest singular value of a random matrix with entries in $N (0,1)$
If $A$ is a Gaussian random matrix as you describe, then the ensemble of matrices given by $A^TA$ is known as the Wishart ensemble, or the Laguerre ensemble. It has been extensively studied, and you can find information in standard books about random matrix theory. 


The average of the largest eigenvalue of $A^TA$ is $4n$. The distribution around the average is given by the Tracy-Widom function. Distribution of large deviations from the mean are also known.


You can start collecting references about largest eigenvalues by looking here:


*Large Deviations of the Maximum Eigenvalue in Wishart Random Matrices*, by Pierpaolo Vivo, Satya N. Majumdar, Oriol Bohigas, <https://arxiv.org/abs/cond-mat/0701371> 


