Suppose $( X\_{n} )$ is an ergodic binary process with
$$
\mathbb P(X\_{n}=1)= \mathbb P(X\_{n}=0)=\frac 12.
$$


Naturally the entropy (rate) $h(X)$ of $X=(X\_{n})$ satisfies
$$
h(X)=\lim\_{n\to\infty} \frac 1n H(X\_1,\ldots, X\_n)\le H(\frac 12)=\log 2.
$$
The entropy will "drop" due to dependencies in $(X\_n)$.


Suppose that
$$
\sigma^2=\sum\_{n=2}^{\infty} \sup\_{a,b\in\{0,1\} }\Bigl| \mathbb P(X\_{1}=a,X\_{n}=b)-\mathbb P(X\_{1}=a)\mathbb P(X\_{n}=b)\Bigr| 
$$
is very small: $\sigma^2\ll 1$. Small $\sigma^2$ suggests that $(X\_{n})$ are nearly independent.
Can one derive a lower bound for $h(X)$ of the form
$$
h(X)\ge \log 2- f(\sigma^2),
$$
where $f$ is a continuous function with $f(0)=0$? My feeling is that small $\sigma^2$ should imply that
the $\bar{d}$-metric between $\{X\_{n}\}$ and the $\text{Ber}(1/2)$-process should be small,
hence implying that entropies should be close as well. 


In the oposite direction, suppose $\sigma^2$ is large. Then the entropy entropy drop $\log 2- h(X)$ should be substantial as well. For example, for every $\epsilon>0$ one can find
$D>0$ such that for every binary process with $\sigma^2>D$
$$
 h(X)\le \log 2 -\epsilon.
$$


Any ideas, references?



Estimate entropy of a binary process  in terms of decay of correlations
I would not be surprised if small $\sigma^2$ does indeed imply small d-bar distance to Bernoulli, but I think that the converse is false. Rather, processes with arbitrarily small d-bar distance to Bernoulli can have $\sigma^2=+\infty$.


To see this let $(x\_n)$ be a sequence which generates the $(\frac{1}{2},\frac{1}{2})$-Bernoulli process, and let $N>0$. Define a new sequence $(z\_n)$ by taking $z\_n:=x\_n$ when $n$ is not divisible by $2^N$. Otherwise, take $z\_n:=1$ if $n=k2^N$ for even $k$, and $z\_n:=0$ if $n=k2^N$ for odd $k$.


Now consider the (ergodic) process generated by $(z\_n)$. It is not difficult to see that $0$ and $1$ have equal probability, and when $n$ is divisible by $2^{N+1}$ we have
$$\mathbb{P}(X\_1=X\_n=1)=\left(1-\frac{1}{2^N}\right)\cdot \left(\frac{1}{2}\right) + \left(\frac{1}{2^N}\right)\cdot 1 = \frac{1}{2}+\frac{1}{2^{N+1}}$$
so that
$$ \sup\_{a,b}\left|\mathbb{P}(X\_1=a,X\_n=b)-\mathbb{P}(X\_1=a)\mathbb{P}(X\_n=b)\right|\geq \frac{1}{2^{N+1}}.$$
In particular $\sigma^2=+\infty$ as claimed.


