Let $x\_1,\ldots,x\_n$ be drawn iid from such "nice" distribution on $\mathbb R^p$ (but possibly very general!), and let $X$ be the $n$-by-$p$ matrix formed by vertically stacking the $x\_i$'s.



> 
> **Question.** What is the limiting distribution of the "scatter matrix" $\frac{1}{n}XX^T:=\frac{1}{n}\sum\_{i=1}^nx\_ix\_i^T$ as $n \rightarrow \infty$ ?
> 
> 
> 


Observations
============


* If the $x\_i$'s are from a centered multivariate Gaussian, then $XX^T$ has a Wishart distribution.



Limiting distribution of "scatter matrix" $\frac{1}{n}XX^T:=\frac{1}{n}\sum_{i=1}^nx_ix_i^T$ for iid $x_1,\ldots,x_n \in \mathbb R^p$
$\newcommand{\tr}{\operatorname{tr}}$
Let $x=[x^1,\dots,x^p]^T:=x\_1$, $y:=xx^T$, $\mu:=Ey$, $w:=y-\mu$, and $s:=\sum\_1^n x\_ix\_i^T$.
Then, by the appropriate laws of large numbers, $s/n\to Ey$ almost surely and hence in probability and in distribution, provided that $Ey$ exists in $\mathbb R^{p\times p}$. Everywhere here, the convergence is for $n\to\infty$.


Assume now that $E|x|^4<\infty$, where $|x|$ is the Euclidean norm of $x$. Note that $y$ is a random matrix in the vector space $\mathbb R^{p\times p}$, which is naturally endowed by the inner product $A\cdot B:=\tr AB^T$ for $A$ and $B$ in $\mathbb R^{p\times p}$. With respect to this inner product, the covariance operator (say $R$) of the random matrix $y$ is given by the formula
$$(Rf)\_{k,l}=\sum\_{i,j=1}^pf\_{i,j}\operatorname{Cov}(x^ix^j,x^kx^l)
$$
for $f=(f\_{i,j})\in\mathbb R^{p\times p}$ and $k,l=1,\dots,p$. So, by the multivariate central limit theorem, $(s-n\mu)/\sqrt n$ converges in distribution to a zero-mean Gaussian random matrix in $\mathbb R^{p\times p}$ with covariance operator $R$.


