This is a [cross-post from stats.stackexchange.com](https://stats.stackexchange.com/q/407121/44368). No answer has appeared there. Since this is a theoretical question, mathoverflow.net seems to be a more appropriate venue for it.




---


What is the analog of the central limit theorem or concentration theorem for resampling, say, an i.i.d. samples? Are there any references for this topic?


Here is a simple example. Suppose there are $n$ i.i.d. random variables $\{x\_1,x\_2,\cdots,x\_n\}$ with mean $0$ and standard deviation $1$. We sample uniformly randomly with replacement from this set $n$ times and obtain random variables ${y\_1,y\_2,\cdots,y\_n}$. What is the distribution of the mean $\displaystyle y=\frac1{\sqrt n}\sum\_{i=1}^ny\_i$ as $n\to\infty$?



Central limit theorem for resampling
First, we need to fix the notation a bit. Let $X\_1,X\_2,\dots$ be iid zero-mean unit-variance random variables (r.v.'s). For each natural $n$, let the $n$-tuple $(J\_1,\dots,J\_n):=(J\_{n,1},\dots,J\_{n,n})$ of r.v.'s be independent of the $X\_k$'s and have the multinomial distribution with parameters $n,1/n,\dots,1/n$. 
For each $k\in[n]:=\{1,\dots,n\}$, the value of $J\_k=J\_{n,k}$ is the number of times the value $X\_k$ was selected into the "re-sample" from the "sample" $X\_1,\dots,X\_n$.  

Let 
\begin{equation\*}
 S\_n:=\frac1{\sqrt n}\,\sum\_{k=1}^n J\_k X\_k, 
\end{equation\*}
so that $S\_n$ equals $\sqrt n$ times what you denoted by $y$. 
We have to find the limit distribution of $S\_n$ (as $n\to\infty$). Let us show that this limit distribution is $N(0,2)$. 


Indeed, note first here that the characteristic function (c.f.) $g\_n$ of $S\_n$ is given by the formula 
\begin{equation\*}
 g\_n(t):=Ee^{itS\_n}=EE(e^{itS\_n}|J\_1,\dots,J\_n)=E\prod\_{k=1}^n f(J\_kt/\sqrt n)
\end{equation\*}
for real $t$, where $f$ is the c.f. of $X\_1$. Next, the joint moment generating function (mgf) $M\_n$ of $(J\_1,\dots,J\_n)$ is given by the formula 
\begin{equation\*}
 M\_n(t\_1,\dots,t\_n):=Ee^{t\_1J\_1+\cdots+t\_nJ\_n}=\Big(\frac1n\,\sum\_{k=1}^n e^{t\_k}\Big)^n
\end{equation\*}
for real $t\_1,\dots,t\_n$. This follows because (i) the random vector $(J\_1,\dots,J\_n)$ is the sum of the $n$ iid random vectors $(I\_{1,1},\dots,I\_{1,n}),\dots,(I\_{n,1},\dots,I\_{n,n})$, where $I\_{j,k}$ is the indicator that the value selected from the "sample" $X\_1,\dots,X\_n$ at the $j$th step was that of $X\_k$ and (ii) the joint mgf of the random vector $(I\_{1,1},\dots,I\_{1,n})$ is $(t\_1,\dots,t\_n)\mapsto\frac1n\,\sum\_{k=1}^n e^{t\_k}$. 


Hence, for any distinct $k$ and $l$ in $[n]$
\begin{equation\*}
 EJ\_k^2=EJ\_1^2=\frac{d^2}{dt^2}M\_n(t,0,\dots,0)\Big|\_{t=0}=2-1/n=2+O(1/n), 
\end{equation\*}
\begin{equation\*}
 EJ\_k^4=EJ\_1^4=\frac{d^4}{dt^4}M\_n(t,0,\dots,0)\Big|\_{t=0}=15+O(1/n), 
\end{equation\*}
\begin{equation\*}
 EJ\_k^2 J\_l^2=EJ\_1^2 J\_2^2=\frac{\partial^4}{\partial t^2\partial u^2}M\_n(t,u,0,\dots,0)\Big|\_{t=0,u=0}=4+O(1/n). 
\end{equation\*}
So, for 
\begin{equation\*}
 W:=J\_1^2+\cdots+J\_n^2
\end{equation\*}
we have 
\begin{equation\*}
 EW=nEJ\_1^2=2n+O(1),
\end{equation\*}
\begin{equation\*}
 EW^2=nEJ\_1^4+n(n-1)EJ\_1^2 J\_2^2=4n^2+O(n),
\end{equation\*}
and hence 
\begin{equation\*}
 Var\,W=O(n). 
\end{equation\*}
So, for any real $\epsilon>0$, 
\begin{equation\*}
 P(|W-2n|>\epsilon n)=O(1/n)\to0, 
\end{equation\*}
so that 
$$\frac Wn\to2$$ 
in probability. 
Also, for the event 
\begin{equation}
 A\_n:=\{\max\_{k\in[n]}J\_k\le n^{1/3}\}
\end{equation}
(on which all the $J\_k$'s are small enough)
and its complement $A\_n^c$ we have 
\begin{equation\*}
 P(A\_n^c)\le nP(J\_1>n^{1/3})\le n\,EJ\_1^4/n^{4/3}=O(1/n^{1/3})\to0 
\end{equation\*}
and hence $P(A\_n)\to1$ and $1\_{A\_n}\to1$ in probability. 
Moreover, 
\begin{equation\*}
 f(s)=Ee^{isX\_1}=1+is\,EX\_1+(is)^2EX\_1^2/(2+o(1))=1-s^2/(2+o(1))=e^{-s^2/(2+o(1))}
\end{equation\*}
as $\mathbb R\ni s\to0$. So, for each real $t$  

\begin{equation\*}
 1\_{A\_n}\prod\_{k=1}^n f(J\_kt/\sqrt n)=1\_{A\_n}\exp\Big(-\frac{t^2W}{(2+o(1))n}\Big)\to e^{-t^2} 
\end{equation\*}
in probability. 
On the other hand, $|f(s)|=|Ee^{isX\_1}|\le E|e^{isX\_1}|=E1=1$ for all real $s$. 
Hence, 
\begin{equation\*}
 \Big|1\_{A\_n^c}\prod\_{k=1}^n f(J\_kt/\sqrt n)\Big|\le1\_{A\_n^c}\to0
\end{equation\*}
in probability for each real $t$. 
So, by [dominated convergence](https://math.stackexchange.com/questions/751647/convergence-in-probability-implies-convergence-in-mean-under-one-additional-cond), 
\begin{equation\*}
 g\_n(t)=E\prod\_{k=1}^n f(J\_kt/\sqrt n)
 =E1\_{A\_n}\prod\_{k=1}^n f(J\_kt/\sqrt n)+E1\_{A\_n^c}\prod\_{k=1}^n f(J\_kt/\sqrt n)
\to e^{-t^2} 
\end{equation\*}
for each real $t$. 


Thus, the distribution of $S\_n$ converges to $N(0,2)$, as claimed. 




---


That the asymptotic variance of $S\_n$ is $2$ (rather than $1$, as might have been expected) stems from the fact that $EJ\_k^2=2-1/n\to2$. To have another look at this phenomenon, we can let $\vec J:=(J\_1,\dots,J\_n)$ and write 
\begin{equation}
 Var\,S\_n=E\,Var(S\_n|\vec J)+Var\,E(S\_n|\vec J)
 =E\frac1n\sum\_1^n J\_k^2+Var\,0=E J\_1^2=2-1/n\to2. 
\end{equation}


