Let $M=O\Lambda O^\top$ be a positive semi-definite matrix, where $\Lambda\in \mathbb{R}^{p\times p}$ is a diagonal matrix with non-negative entries and $O\in \mathbb{R}^{p\times p}$ is an orthogonal matrix. Let $S\_p$ be the set of all orthogonal matrices of size $p\times p$. What is the average of $M$ over the set $S\_p$, i.e., what is the value of the following quantity?
\begin{equation}
\underset{O\in S\_p}{\text{Average}}(M)
\end{equation}



Average the covariance matrix over all orthogonal matrices
I presume you want to average over the orthogonal matrices uniformly, so with the Haar measure. Then $\mathbb{E}[O\_{ik}O\_{jk}]=p^{-1}\delta\_{ij}$, hence
$$\mathbb{E}[M\_{ij}]=\mathbb{E}\left[\sum\_{k=1}^p O\_{ik}\lambda\_k O\_{jk}\right]=p^{-1}\delta\_{ij}\sum\_{k=1}^p\lambda\_{k},$$
with $\Lambda\_{ij}=\delta\_{ij}\lambda\_i$.


