Let $X$ be an random $n \times d$ matrix with entries drawn iid from $N(0,1/d)$ and let $w$ be a unit-vector in $\mathbb R^d$. With $\lambda>0$, and define $G:=X^\top(XX^\top + \lambda I\_n)^{-1}X$. Finally, defined $\alpha := w^\top G^2 w$.



> 
> **Question.** *In the limit $n,d \to \infty$ with $n/d \to \rho \in (0,\infty)$, what is the limitting value of $\alpha$ as a function of $\lambda$ and $\rho$ ?*
> 
> 
> 


A useful subcase is when $\lambda \to 0^+$.



> 
> **Question.** *What is the value of $\lim\_{\lambda \to 0^+}\lim\_{n,d \to \infty \\ n/d \to \rho}\alpha$ as a function of $\rho$ ?*
> 
> 
> 



Asymptotics of $w^\top G^2 w$, where $w$ is a unit-vector, $G:=X^T(XX^T+t I_n)^{-1}X$, $t > 0$, and $X$ is an $n\times d$ gaussian random matrix
Let me calculate the expectation value of $\alpha$. The probability distribution of $X$ is invariant under orthogonal transformations, so without loss of generality I can orient the unit vector $w$ along one of the axes, $w\_i=\delta\_{ip}$, $p\in\{1,2,\ldots d\}$. Then
$$\mathbb{E}[\alpha]=\mathbb{E}\left(X^T(XX^T+\lambda I)^{-1}XX^T(XX^T+\lambda I)^{-1}X\right)\_{pp}.$$
Again because of orthogonal invariance the answer cannot depend on the value of the index $p$, hence we can sum over $p$ and divide by $d$, which gives the trace,
$$\mathbb{E}[\alpha]=\frac{1}{d}\mathbb{E}\,{\rm tr}\,\left(X^T(XX^T+\lambda I)^{-1}XX^T(XX^T+\lambda I)^{-1}X\right)$$
$$\qquad=\frac{1}{d}\mathbb{E}\,{\rm tr}\,\frac{W^2}{(W+\lambda I)^2},\;\;W=XX^{T}.$$


For the subcase $\lambda\rightarrow 0$ in the OP we thus find $\mathbb{E}[\alpha]=n/d \to \rho$.


For nonzero $\lambda$ and in the large-$n$ limit the result for $\mathbb{E}[\alpha]$ is the integral of $\mu^2(\mu+\lambda)^{-2}$ weighted by the [Marchenko-Pastur distribution](https://en.wikipedia.org/wiki/Marchenko%E2%80%93Pastur_distribution) $\rho(\mu)$ for the eigenvalues $\mu$ of $W$.


