For integer $n$, $1 \le n \le N$, consider the random variables


$X\_n = \cos[t \omega\_n]$


For any fixed $N$, we can take the mean


$Y\_N = \frac{1}{N} \sum\_{n=1}^N X\_n$


and define a (cumulative) distribution by averaging over long times:


$P(Y\_N \le y) = \lim\_{T \to \infty} \frac{1}{2 T} \lambda(\{t \in [-T,T] \mid Y\_N \le y \} )$,


where $\lambda$ is the Lebesgue measure. If the $\omega\_n$ are linearly independent over the rationals for all $n$, then the random variables $X\_n$ are independent and identically distributed over long times. Since they are i.i.d., we can apply the central limit theorem to show that $p(Y\_N = y)$, the probability density of $Y\_N$, approaches a normal distribution with zero mean and variance $\sigma\_0^2/N$, where $\sigma\_0^2=1/2$ is the variance of each $X\_n$.


Now, suppose that the $\omega\_n$ are *not* all linearly independent, but are incommensurate (i.e. pairwise linearly independent). This would mean that the $X\_n$ are not independent. In particular, suppose we consider only $N=2^M$ for integer $M$ and take


$w\_n^{(M)} = \sum\_{k=0}^{M-1} (-1)^{r\_k} h\_k$


where 


$r\_k = (n/2^k) \mod 2$ 


are the digits of $n$ in base 2. Though the $h\_k$ may be linearly independent over the rationals, there are $2^M$ frequencies $w\_n^{(M)}$ for each fixed $M$. Since $k$ only ranges from $0$ to $M-1$, these frequencies are not linearly independent.


So for fixed $M$, not only are the $2^M$ random variables $X\_n^{(M)} = \cos[t \omega\_n^{(M)}]$ not independent, their joint probability density is actually only has support on a $M$-dimensional subspace. This is not a stationary sequence (I think). I'm not sure if it is ergodic.


Now, we could express the $Y$ as a mean of just $M$ independent random variables by combining the $X\_n^{(M)}$. This would seem to guarantee a variance only as small as $\sigma\_0^2/M$ rather than $\sigma\_0^2/2^M$. 


However, numerical work confirmed my intuition that the the $Y$ actually approaches a normal distribution with the small variance $\sigma\_0^2/2^M$. This suggests there is a CLT I could apply to get this result analytically. But when I read the standard texts, I can't find much in the way of CLT's for non-stationary processes. I'm trying to read about ergodicity, but I can't even tell if this sequence fits the descriptions.


Is this sequence ergodic? Does it satisfy a CLT?



What conditions on a probability distribution defined by long-time averaging do I need to satisfy a central limit theorem?
In your example, algebraic manipulation gives
$$
2^{-M} \sum\_{n=0}^{2^M-1} X\_n^{(M)} = \prod\_{j=0}^{M-1} \cos t h\_j.
$$ 
If the $h\_j$'s are linearly independent over $\Bbb Q$, then, as you point out, the random variables $\cos t h\_j$ approach independence as $t$ is chosen over larger and larger intervals. Therefore, in the limit, 
$$
Y = Z\_1 ... Z\_M, \qquad Z\_i = \cos W\_i, \qquad W\_1,...,W\_M {\rm\ \ i.i.d.\ uniform\ on\ } [0,2\pi).
$$
By symmetry, ${\bf E}[Y]=0$, and since ${\bf E}[Z\_i^2]=1/2$ for each $i$, we have ${\bf Var}[Y]=2^{-M}$. This may explain the result of your numerical experiments. However, $Y$ does not approach normal after rescaling: if $Y$ is rescaled to unit variance by setting $Y'\_M:=2^{M/2} Y$, then $\log |Y'\_M|=\log(\sqrt{2} |Z\_1|)+...+\log(\sqrt{2} |Z\_M|)$, so, since ${\bf E}[\log(\sqrt{2} |Z\_i|)]<0$ and ${\bf Var}[\log(\sqrt{2} |Z\_i|)]<\infty$, we can apply the CLT to $\log |Y'\_M|$ to show that, as $M\to\infty$, $\log |Y'\_M|$ will converge weakly to normal after rescaling, but $Y'\_M$ converges weakly to 0.


