Let $d,m \to \infty$ (integers) with $m/d \to \rho \in (0, \infty)$. Let $C$ be a $d \times d$ psd matrix with $trace(C)=\mathcal O(1)$, and let $w\_1,\ldots,w\_m$ be iid uniformly distributed on the unit-sphere in $\mathbb R^d$. Consider the quartic form
$$
F := \frac{1}{m}\sum\_{j,\ell=1}^m (w\_j^\top w\_\ell)(w\_j^\top C w\_\ell).
$$



> 
> **Question.** What are good probabilistic lower and upper-bounds for $F$ only in terms of $\rho$ and the eigenvalues of $C$ ?
> 
> 
> 


For example, the solution for the case where $C$ is **diagonal** will already be very helpful.


Isotropic example
-----------------


Thanks to this post <https://mathoverflow.net/a/334219/78539>, we know that if $C = (1/d) I\_d$, then $F = m^{-1}\|WW^\top\|\_F^2 = m^{-1}\sum\_{j}\lambda\_j(W W^\top)^2\overset{a.s}{\to} \langle \lambda^2\rangle\_{\text{MP}(1/\rho)}$ (if I haven't made some scaling errors), where $\text{MP}(\gamma)$ is the Marchenko-Pastur law with parameter $\gamma$.



Probabilistic lower and upper-bounds for a certain random quartic form involving gaussian random matrices
Assume iid $N(0,1)$ entries, assume $C$ diagonal, and focus first on the non-diagonal terms:


$G=\sum\_j \sum\_{l\ne j} w\_j^Tw\_l w\_j^TCw\_l
= \sum\_{j\ne l, ik} w\_{ji}w\_{li} c\_i w\_{jk} w\_{lk}$.
Write this quantity as
$$
\begin{split}
G=\sum\_{j\ne l, i\ne k} w\_{ji}w\_{li} c\_i w\_{jk} w\_{lk}
&+
\sum\_{j\ne l, i=k} (w\_{ij}^2-1)(w\_{lj}^2 -1)c\_i
\\
&+(w\_{ij}^2-1)c\_i + 
(w\_{lj}^2 -1)c\_i
+c\_i
\end{split}
$$
This is a decomposition in uncorrelated polynomials (any two terms are uncorrelated), so that the second moment is
$$
E[(G-m(m-1)trace[C])^2]=\sum\_{j\ne l, i \ne k} c\_i^2
+
\sum\_{j\ne l, i}(E[(Z^2-1)^2]^2 + 2 E[(Z^2-1)^2])c\_i^2.
$$
$$= m(m-1)\|C\|\_F^2((d-1)+E[(Z^2-1)^2]^2 + 2E[(Z^2-1)^2]).$$
The dominant term is of order $m^2d \|C\|\_F^2$, while the mean is $m(m-1)trace[C]$. Hence $G/E[G]-1$ converges to 0 in probability (or in L2) provided that $E[G]^2 \gg Var[G]$, that is,
$$
m^2 trace[C] \gg \|C\|\_F m \sqrt{d}.
$$


For the diagonal terms, we have
$\sum\_j d w\_j^TCw\_j + \sum\_{j} (\|w\_j\|^2-d)w\_j^TCw\_j$.
The second term is negligible compared to the first one if you use $\chi^2$ concentration (e..g, Bernstein inequality) for $\|w\_j\|^2-d$, while the first term has mean $md trace[C]$ and variance $2md^2\|C\|\_F^2$.
Again, the mean dominates the standard deviation if and only if
$$
m d ~trace[C] \gg \sqrt m d \|C\|\_F.$$ This is equivalent to the condition on the non-diagonal terms if $m\asymp d$.


Edit: since $\|C\|\_F^2 \le trace[C]^2$ for $C$ psd, these conditions are always satisfied.


