A smooth transition from Wishart to GOE
Miklós Z. Rácz

∗

Jacob Richey

†

arXiv:1611.05838v1 [] 17 Nov 2016

November 18, 2016

Abstract
It is well known that an n × n Wishart matrix with d degrees of freedom is close to the
appropriately centered and scaled Gaussian Orthogonal Ensemble (GOE) if d is large enough.
Recent work of Bubeck, Ding, Eldan, and Racz, and independently Jiang and Li, shows that
the transition happens when d = Θ(n3 ). Here we consider this critical window and explicitly
compute the total variation distance between the Wishart and GOE matrices when d/n3 → c ∈
(0, ∞). This shows, in particular, that the phase transition from Wishart to GOE is smooth.

1

Introduction

The Wishart distribution is a fundamental object appearing in many domains, such as statistics,
geometry, quantum physics, and wireless communications, among others. In statistics it arises as
the distribution of the sample covariance matrix of a sample from a multivariate normal distribution.
In geometry it is known as the Gram matrix of inner products of n points in Rd , and it is also the
starting point for canonical models of random geometric graphs [5, 3, 8].
It is well known that an n × n Wishart matrix with d degrees of freedom is close to the
appropriately centered and scaled Gaussian Orthogonal Ensemble (GOE) if d islarge enough (see,
e.g., [5]). Recent work [3, 9] shows that the transition happens when d = Θ n3 and in this paper
we study this critical window. In Theorem 1.2 below we explicitly compute the total variation
distance between the Wishart and GOE matrices when d/n3 → c ∈ (0, ∞), showing, in particular,
that the phase transition from Wishart to GOE is smooth.

1.1

Main result

Let X be an n × d matrix where the entries are i.i.d. standard normal random variables, and let
W ≡ W (n, d) = XX T be the corresponding n × n Wishart matrix with d degrees of freedom.1 Let
M (n) be an n × n matrix drawn from the Gaussian Orthogonal Ensemble, i.e., a symmetric n × n
random matrix where the diagonal entries are i.i.d. normal random variables with mean zero and
variance 2, and the entries above the diagonal are i.i.d. standard normal random variables, with the
entries on and above the diagonal all independent. In order to match the first moment
√ and the scale
of the Wishart matrix, we center and scale M (n) appropriately: let M (n, d) := dM (n) + dIn ,
where In is the n × n identity matrix.
∗

Microsoft Research; miracz@microsoft.com.
University of Washington; jfrichey@uw.edu.
1
In statistics the number of samples is usually denoted by n and the number of parameters is usually denoted
by p, resulting in a p × p Wishart matrix with n degrees of freedom. Here our notation is taken with the geometric
perspective in mind, following [5, 3, 4].
†

1

If d is large enough compared to n, then the Wishart matrix becomes approximately like the
GOE. Recent work of Bubeck, Ding, Eldan,and Racz [3], and independently Jiang and Li [9], shows
that the transition happens when d = Θ n3 . Specifically, they proved the following theorem, where
we write TV for total variation distance.
Theorem 1.1. Define the random matrix ensembles W (n, d) and M (n, d) as above.
(a) (Bubeck, Ding, Eldan, and Racz [3]) If d/n3 → 0 then
TV (W (n, d), M (n, d)) → 1.
(b) (Bubeck, Ding, Eldan, and Racz [3]; Jiang and Li [9]) If d/n3 → ∞ then
TV (W (n, d), M (n, d)) → 0.
Our focus is on the critical window and our main result is the explicit computation of the
limiting total variation distance between W (n, d) and M (n, d) when d/n3 → c ∈ (0, ∞).
Theorem 1.2. Define the random matrix ensembles W (n, d) and M (n, d) as above and let d = d(n)
be such that d/n3 → c ∈ (0, ∞). Then


1
√ √ ,
(1.1)
lim TV (W (n, d), M (n, d)) = Erf
n→∞
4 3 c
where recall that the error function is defined as
2
Erf (x) = √
π

Z

x

2

e−t dt.

0

From this result we can immediately read off that, as c → 0 and c → ∞, the total variation
distance goes to 1 and 0, respectively, recovering the previous results described in Theorem 1.1.
Since Erf (x) = √2π x (1 + o (1)) as x → 0, the limiting total variation distance decays as


1
1
√ √
Erf
∼ √ √
4 3 c
2 3π c
as c → ∞. The behavior of the limit when c is small is plotted in Figure 1.

Figure 1: The limiting total variation distance as a function of c, when c is close to 0.
From the proof we shall see that the limit in (1.1) is the expected value of an explicit function
of a two-dimensional Gaussian, which comes from the central limit theorem for the first and third
moments of the empirical spectral distribution of a GOE matrix.
2

1.2

Further related work and open problems

Several recent works have explored extensions of Theorem 1.1, and Theorem 1.2 raises further
questions.
Robustness. Bubeck and Ganguly [4] showed that the critical dimension is universal in the
following sense: Theorem 1.1 holds (up to logarithmic factors) if the entries of X are i.i.d. from a
sufficiently smooth distribution. What can be said about the transition in the critical regime? Are
there other distributions for which the limiting total variation distance can be computed explicitly?
If not, can one prove similar qualitative behavior?
Anisotropy. Eldan and Mikulincer [8] studied the effect of anisotropy on the power of detecting
geometry in random geometric graphs. This is directly related to studying Wishart matrices where
each row of X is a multivariate normal with a diagonal covariance matrix. The authors introduce
new notions of dimensionality and prove a theorem similar to Theorem 1.1 with appropriate upper
and lower bounds on the “effective critical dimension”. While the primary open problem is to close
the gap between these bounds, one may also ask about the nature of the transition at the effective
critical dimension: can anisotropy cause qualitatively different behavior?
Other regimes. Theorems 1.1 and 1.2 state that as d/n3 → ∞, all statistics of the Wishart
W (n, d) and the GOE M (n, d) have asymptotically the same distribution, but this is not the case
if d/n3 remains bounded. In the random matrix literature there has been lots of work showing that
particular statistics of these ensembles have asymptotically the same distribution even when d  n3 .
For instance, when d = Θ (n), then the limiting empirical spectral distribution of the Wishart is
the Marchenko-Pastur law, which shows the difference between the Wishart and GOE, but the
largest eigenvalue of the Wishart already behaves like that of the GOE [10, 6, 7]. This naturally
raises the question of whether there are other regimes of d and n where there are interesting phase
transitions.

2

Proof of Theorem 1.2

The main reason that allows for an explicit computation of the limiting total variation distance in
Theorem 1.2 is that both W (n, d) and M (n, d) have explicit densities. The proof of Theorem 1.2
is similar to the case of d/n3 → ∞ presented in [3] and proceeds by a Taylor expansion of the
ratio of the densities of the two random matrix ensembles. The difference compared to the case of
d/n3 → ∞ is that here the Taylor expansion has to be done to one degree higher. As we shall see,
taking the limit of the total variation distance as d/n3 → c ∈ (0, ∞) then requires using the central
limit theorem for the moments of the empirical spectral distribution of a GOE matrix.
n(n+1)

Step 1: Writing out the total variation distance. Let P ⊂ R 2 denote the cone of
positive semidefinite matrices. It is well known (see, e.g., [11]) that when d ≥ n, W (n, d) has the
following density with respect to the Lebesgue measure on P:

1
(det (A)) 2 (d−n−1) exp − 21 Tr (A)
fn,d (A) := 1
,
Q
1
2 2 dn π 4 n(n−1) ni=1 Γ 12 (d + 1 − i)
where Tr (A) denotes the trace of the matrix A. The density of a GOE random matrix with respect

n(n+1)
1
n
to the Lebesgue measure on R 2 is A 7→ (2π)− 4 n(n+1) 2− 2 exp − 14 Tr A2 and so the density

3

n(n+1)

of M (n, d) with respect to the Lebesgue measure on R 2 is



1
Tr (A − dIn )2
exp − 4d
.
gn,d (A) :=
1
n
(2πd) 4 n(n+1) 2 2
n(n+1)

Denote the measure given by this density by µn,d , let λ denote the Lebesgue measure on R 2
and write A  0 if A is positive semidefinite. We can then write
Z

TV (W (n, d), M (n, d)) = n(n+1) gn,d (A) − fn,d (A) 1{A0} + dλ (A)
R 2


Z
fn,d (A) 1{A0}
= n(n+1) 1 −
dµn,d (A) ,
(2.1)
gn,d (A)
R 2
+
where x+ := max {x,
Let Q denote the
set of symmetric matrices for which all of the eigenvalues
h 0}. √
√ i
are in the interval d − 3 dn, d + 3 dn . Since d/n3 → c > 0, we have that Q ⊂ P for all n large
enough. It is known (see, e.g., [1]) that, with probability 1 − o (1), all the eigenvalues of M (n)
√
√
are in the interval [−3 n, 3 n], which implies that M (n, d) ∈ Q. Since the integrand in (2.1) is
bounded, we can then write

Z 
fn,d (A)
TV (W (n, d), M (n, d)) =
1−
dµn,d (A) + o (1)
(2.2)
gn,d (A) +
Q
and so we may restrict our attention to Q.
Define αn,d (A) := log (fn,d (A) /gn,d (A)). Denote the eigenvalues of an n × n matrix A by
λ1 (A) ≤ · · · ≤ λn (A); when
P from the context, we omit the dependence on
Q the matrix is obvious
A. Recall that det (A) = ni=1 λi and Tr (A) = ni=1 λi . We then have that

n 
1X
1
2
αn,d (A) =
(λi − d)
(d − n − 1) log λi − λi +
2
2d
i=1




n
X
n (n + 3) dn
n
n (n + 1)
1
+
−
log 2 + log π +
log d −
log Γ
(d + 1 − i) .
4
2
2
4
2
i=1


By Stirling’s formula we know that log Γ (z) = z − 12 log z − z + 12 log (2π) + O z1 as z → ∞, so

n 
1X
1
2
αn,d (A) =
(d − n − 1) log λi − λi +
(λi − d)
2
2d
i=1

n

n

n
n (n + 1)
1X
1X
log d −
(d − i) log (d + 1 − i) +
(d + 1 − i) + O
.
4
2
2
d
i=1
i=1



(i−1)2
(i−1)3
i−1
Now writing log (d + 1 − i) = log d + log 1 − i−1
=
log
d
−
−
+
O
we get that
2
3
d
d
2d
d
+


n 
1X
1
n3
n
2
αn,d (A) =
(d − n − 1) log λi − λi +
(λi − d) − {(d − n − 1) log d − d} −
+ o (1) .
2
2d
2
12d
i=1
n
o
1
Defining h (x) := 21 (d − n − 1) log (x/d) − (x − d) + 2d
(x − d)2 , we have that
αn,d (A) =

n
X

h (λi ) −

i=1

4

n3
+ o (1) .
12d

(2.3)

Step 2: Taylor expansion and taking the limit. The derivatives of h at d are h (d) = 0,
n+1
00
(3) (d) = d−n−1 , h(4) (d) = − 3(d−n−1) , and also h(5) (x) = 12(d−n−1) .
= − n+1
2d , h (d) = 2d2 , h
d3
d4
x5
Approximating h with its fourth order Taylor polynomial around d we get that
h0 (d)

h(x) = −

d−n−1
d−n−1
d−n−1
n+1
n+1
(x − d)2 +
(x − d)3 −
(x − d)4 +
(x − d)5 ,
(x − d)+
2d
4d2
6d3
8d4
10ξ 5

where ξ is some real number between x and d. From 2.3 we see that to compute αn,d (A), we need
to compute the sum over the eigenvalues {λi }ni=1 of each term in the expansion.
First, we argue
the contribution
from the remainder term
Recall that A ∈ Q,
h that
h is negligible.
√
√ i
√
√ i
and hence λi ∈ d − 3 dn, d + 3 dn for every i ∈ [n]. If x ∈ d − 3 dn, d + 3 dn , then
d−n−1
(x − d)5 ≤
10ξ 5

 
 √ 5
d−n−1
1

√ 5 3 dn = O n2 ,
10 d − 3 dn


where we used that d = Θ n3 . Summing n such terms gives a term of order O(1/n), which is
negligible in the limit. Turning to the four terms that matter, defining
n

n

n+1X
S1 (n, d) := −
(λi − d) ,
2d

n+1X
S2 (n, d) :=
(λi − d)2 ,
4d2

i=1

S3 (n, d) :=

n
d−n−1X

6d3

i=1

n

d−n−1X
(λi − d)4 ,
S4 (n, d) := −
8d4

3

(λi − d) ,

i=1

i=1

and also letting S0 (n, d) := −n3 /(12d), we thus have that
αn,d (A) = S0 (n, d) + S1 (n, d) + S2 (n, d) + S3 (n, d) + S4 (n, d) + o (1) .

(2.4)

(λi − d). If {λi }ni=1 are the eigenvalues of M (n, d), then {µi }ni=1 are
P
the eigenvalues of √1n M (n). Recall that the empirical spectral distribution n1 ni=1 δµi converges
√
1
weakly to the semicircle distribution with density ρsc (x) = 2π
4 − x2 1{|x|≤2} (see, e.g., [1]). With
this notation we can rewrite the four quantities above as follows:
√
n
n
n (n + 1) X
n2 (n + 1) 1 X 2
√
S1 (n, d) = −
×
µi ,
S2 (n, d) =
×
µi ,
4d
n
2 d
i=1
i=1
r
n
n
d − n − 1 n3
1X 4
d−n−1
n3 X 3
S4 (n, d) = −
×
×
µi .
×
×
µi ,
S3 (n, d) =
8d
d
n
6d
d
For i ∈ [n] define µi :=

√1
dn

i=1

i=1

Let U be a random variable distributed according to the semicircle law. We know that for any
fixed k ∈ N, the k th moment of the empirical spectral distribution converges in probability to the
k th moment of the semicircle law (see [1, Lemmas 2.1.6 and 2.1.7]), i.e.,
n

h i
1X k
µi → E U k .
n
i=1

 
 
Since E U 2 = 1 and E U 4 = 2, we have that, as d/n3 → c ∈ (0, ∞), S2 (n, d) → 1/(4c) and
S4 (n, d) → −1/(4c), so put together we have that S2 (n, d) + S4 (n, d) → 0.
5

The central limit theorem for the moments of the empirical spectral distribution of a GOE
matrix (see [1, Theorem 2.1.31 and Exercise 2.1.35] and [2]) shows that
!
n
n
X
X
µi ,
µ3i =⇒ (N1 , N3 ) ,
i=1

i=1

6 ). The entries
where (N1 , N3 ) are jointly normal with mean zero and covariance matrix C = ( 26 24
of the covariance matrix C are special cases of the more general formulas found in [1, 2], so we do
not describe the computations here, but one can verify these numbers by using the identity
n
X

µki

= Tr



√1 M
n

n 
k  X
k 
1
√
(n)
=
M (n)
n
i=1

i=1

i,i

and computing appropriate moments of normal random variables.
Putting everything together we see that, as d/n3 → c ∈ (0, ∞), we have that


1
1
1
1
1
√
√
,−
.
N1 , ,
N3 , −
(S0 , S1 , S2 , S3 , S4 ) =⇒ −
12c 2 c
4c 6 c
4c

(2.5)

Therefore, since the function (s0 , s1 , s2 , s3 , s4 ) 7→ (1 − exp (s0 + s1 + s2 + s3 + s4 ))+ is continuous
and bounded, we have by (2.2), (2.4), and (2.5) that


 
1
1
1
TV (W (n, d), M (n, d)) → E 1 − exp −
− √ N1 + √ N3
.
(2.6)
12c 2 c
6 c
+
Step 3: Evaluating the limit. What remains is to evaluate the expectation on the right
hand side of (2.6). Let Y and Z be independent normal random variables with mean zero and
variances 2 and 6, respectively. Then (N1 , N3 ) and (Y, 3Y + Z) have the same distribution, since
both are Gaussian with the same mean vector and covariance matrix. Notice that
1
1
1
− √ Y + √ (3Y + Z) = √ Z
2 c
6 c
6 c
and hence the right hand side of (2.6) is equal to


 
1
1
E 1 − exp −
+ √ Z
.
12c 6 c
+
√
√
Since 1 − exp (−1/(12c) + z/(6 c)) ≥ 0 if and only if z ≤ 1/(2 c), we have that


  Z √1 

1
2 c
z2
1
1
1
z
− 1 + √
E 1 − exp −
+ √ Z
=
1 − e 12c 6 c · √
e− 12 dz
12c 6 c
12π
−∞
+
Z √1
Z √1
√
2
(z−1/ c)2
2 c
2 c
1
1
− z12
−
12
√
√
=
e
dz −
e
dz
12π −∞
12π −∞






1
1
1
√ √ ,
=P Z< √
−P Z <− √
= Erf
2 c
2 c
4 3 c
which concludes the proof.

6

References
[1] G. W. Anderson, A. Guionnet, and O. Zeitouni. An Introduction to Random Matrices. Cambridge University Press, 2010.
[2] G. W. Anderson and O. Zeitouni. A CLT for a band matrix model. Probability Theory and
Related Fields, 134(2):283–338, 2006.
[3] S. Bubeck, J. Ding, R. Eldan, and M. Z. Rácz. Testing for high-dimensional geometry in
random graphs. Random Structures & Algorithms, 49(3):503–532, 2016.
[4] S. Bubeck and S. Ganguly. Entropic CLT and phase transition in high-dimensional Wishart
matrices. Preprint available at http://arxiv.org/abs/1509.03258, 2015.
[5] L. Devroye, A. György, G. Lugosi, and F. Udina. High-dimensional random geometric graphs
and their clique number. Electronic Journal of Probability, 16:2481–2508, 2011.
[6] N. El Karoui. On the largest eigenvalue of Wishart matrices with identity covariance when n,
p and p/n → ∞. arXiv preprint math/0309355, 2003.
[7] N. El Karoui. Tracy-Widom limit for the largest eigenvalue of a large class of complex sample
covariance matrices. The Annals of Probability, pages 663–714, 2007.
[8] R. Eldan and D. Mikulincer. Information and dimensionality of anisotropic random geometric
graphs. Preprint available at https://arxiv.org/abs/1609.02490, 2016.
[9] T. Jiang and D. Li. Approximation of Rectangular Beta-Laguerre Ensembles and Large Deviations. Journal of Theoretical Probability, 28:804–847, 2015.
[10] I. M. Johnstone. On the distribution of the largest eigenvalue in principal components analysis.
Annals of Statistics, 29(2):295–327, 2001.
[11] J. Wishart. The Generalised Product Moment Distribution in Samples from a Normal Multivariate Population. Biometrika, 20A(1/2):32–52, 1928.

7

