Suppose that $X\_1,..,X\_n$ are i.i.d real random variables with density $f \in L\_2(\mathbb R)$, and that $g\_i$ are function forming an orthonormal basis of $L\_2(\mathbb R)$, i.e :


$$f(x) = \sum\limits\_{i} a\_i g\_i(x) \text{ for } a\_i = \int g\_i(x) f(x) dx$$


Set the Monte-Carlo coefficients to be $\widehat{a\_i} = \frac{1}{n} \sum\limits\_{i=1}^{n} g\_i(X\_i)$, and denote $\hat{f}(x) = \sum\limits\_{i} \widehat{a\_i} g\_i(x)$.


I want to show that: $$\lim\limits\_{n \to \infty} \lVert f - \hat{f} \rVert\_{2}^2 = \lim\limits\_{n \to \infty} \sum\limits\_{i} (a\_i - \widehat{a\_i})^2 = 0$$


I am able to show that the estimators $\widehat{a\_i}$ are unbiaised and converge correctly to $a\_i$, but here I need some kind of uniform convergence, right ?



Convergence of an orthormal expansion of the density
1. The (more) correct definition of the $\widehat{a\_i}$'s should be
$$\widehat{a\_{n,i}}:=\frac1n\,\sum\_{j=1}^n g\_i(X\_j).$$
2. So,
$$\widehat{a\_{n,i}}=\int\_{\mathbb R}\mu\_n(t) g\_i(t)\,dt,$$
where
$$\mu\_n(t):=\frac1n\,\sum\_{j=1}^n \delta\_{X\_j}(t)$$
and $\delta\_x$ is the Dirac probability measure at $x$, viewed as the (say) [Schwartz distribution](https://en.wikipedia.org/wiki/Distribution_(mathematics)). So, the $\widehat{a\_{n,i}}$'s may be viewed as the "coordinates" of the Schwartz distribution $\mu\_n$ in the basis $(g\_i)$ of $L\_2(\mathbb R)$. So, if we had $\sum\_i \widehat{a\_{n,i}}^2<\infty$, we would also have $\mu\_n\in L\_2(\mathbb R)$, which is of course false. So, $\sum\_i \widehat{a\_{n,i}}^2=\infty$ for any basis $(g\_i)$ of $L\_2(\mathbb R)$, and hence $\sum\_i (a\_i-\widehat{a\_{n,i}})^2=\infty$ for any real $a\_i$'s such that $\sum\_i a\_i^2<\infty$.
3. Another view at why your idea of estimation of the density $f$ did not succeed is that the sequence $(\mu\_n)$ of the [(empirical) probability measures](https://en.wikipedia.org/wiki/Empirical_measure) converges to the probability measure (say $\mu$) with density $f$ only weakly, and the probability measure $\mu\_n$ does not even have a density, to converge to $f$ in any sense or to do anything else.
4. Generally, it appears unnatural to estimate a pdf in an $L^2$ framework. The natural framework should be $L^1$. This is the view advocated (I think persuasively) by some authors, including [Devroye and Gyorfi](https://www.google.com/books/edition/Nonparametric_Density_Estimation/ZVALbrjGpCoC?hl=en), who write that their book "develops, from first principles, the ``natural'' theory for density estimation, L1, and shows why the classical L2 theory masks some fundamental properties of density estimates".


