Suppose that we have a uniformly distributed $d\times d$ random orthonormal matrix $\mathbf{X}$. Here "uniform" is defined in the sense of Haar measure, i.e., the distribution does not change up to any rotation of basis (e.g., multiplication with any arbitrary orthonormal matrix).


Let $\mathbf{Z}$ be the upper-left $k\times k$ principal submatrix of $\mathbf{X}$. What is the distribution of $\| \mathbf{Z} \|\_F^2$, e.g., its probability density function and cumulative distribution function? (We may be also interested in their asymptotic versions.)


This also relates to the so-called multivariate beta distribution. Let $\mathbf{Y}\in \mathbb{R}^{d\times k}$ be a random matrix with i.i.d. standard Gaussian entires, and $\mathbf{Y} = \mathbf{Q}\mathbf{R}$ be its QR decomposition. Let $\mathbf{Q}'$ be the top $k\times k$ submatrix of $\mathbf{Q}$. Then $\mathbf{Q}'$ follows the multivariate beta distribution. An equivalent question is as follows. What is the (asymptotic) distribution of $\|\mathbf{Q}'\|\_F^2$?



Frobenius norm of the principal submatrix of a uniformly distributed random orthonormal matrix
This problem has been studied in the [physics literature](http://arxiv.org/abs/1004.2438) as the distribution of the thermal conductance of a superconducting quantum dot. Let me explain the relationship: The $d\times d$ orthogonal matrix $X$ corresponds to the scattering matrix $S$, the $k\times k$ upper-left principal submatrix $Z$ corresponds to the reflection matrix $r$, the Frobenius norm squared $||Z||\_F^2$ is related to the dimensionless thermal conductance $G$ by


$$G=k-{\rm tr}\,rr^{\rm T}=k-||Z||\_F^2.$$


(Superscript T is the transpose and tr is trace.)


The distribution of $G$, and hence of the desired Frobenius norm, follows from the distribution of the socalled transmission eigenvalues $T\_n=1-R\_n$, with $R\_n$ an eigenvalue of $ZZ^{\rm T}$. The joint probability distribution of the $R\_n$'s is known exactly for any $k$ and $d=k+k'$. In the case $k\leq k'$ it is given by


$$P(R\_1,R\_2,\ldots R\_k)=C \sqrt{\prod\_{n=1}^k\frac{(1-R\_n)^{k'-k}}{(1-R\_n)R\_n}}\;\prod\_{i=1}^j\prod\_{j=1}^k |R\_i-R\_j|,$$


with $C$ a normalization constant and all $R\_n\in(0,1)$. If $k>k'$ the above result still holds with $k$ and $k'$ interchanged, and then there are an additional set of $k-k'$ of the $R\_n$'s pinned at unity.


The desired probability distribution of
$$||Z||\_F^2=\sum\_{n=1}^k R\_n$$
now follows upon integration. If $d\gg k$ an approximate result follows by taking the matrix elements of $Z$ to be independent Gaussians, of zero mean and variance $1/d$, resulting in a [chi-squared distribution](https://en.wikipedia.org/wiki/Chi-squared_distribution) with $k^2$ degrees of freedom.


If knowledge of the first two moments is sufficient, then there are exact results for any $k,d$,


$$E(||Z||\_F^2)=k^2/d,\;\;{\rm Var}\,(||Z||\_F^2)=\frac{2k^2(d-k)^2}{d^2(d-1)(d+2)}$$


I note that there are similar results when $X$ is complex unitary, rather than real orthogonal, reviewed [here.](http://arxiv.org/abs/1407.2131)


