How big is the weak-\* closure of the set of all (finite) convex combinations of Bernoulli measures among all invariant probability measures? 


I mean, we are in the symbolic space $\{1,2,\ldots,d\}^{\mathbb{N}}$ and I would like to know, how big the (weak-\*) closure of the set $C$ is, where
$$C = \left\{\sum\_{i=1}^n \alpha\_i \eta\_i: n \in \mathbb{N}; \sum\_{i=1}^n \alpha\_i=1; \eta\_i = \mathrm{Ber}(p^i\_1,\ldots,p^i\_d) \right\}.$$


Of course, any reference towards this topic may be useful.


Thanks a lot for your attention 



Convex combinations of Bernoulli Measures
The set of Bernoulli measures $B$ is closed, and the closure of $C$ is precisely the set of measures of the form $\int m\,d\mathbb{P}$ where $\mathbb{P}$ is a Borel probability measure on the set of Bernoulli measures.


Let me unpack that statement a little. Let $\mathcal{M}\_\sigma$ denote the set of shift-invariant Borel probability measures on $\Sigma\_d:=\{1,\ldots,d\}^{\mathbb{N}}$. This we equip with the weak-\* topology, which is the smallest topology on $\mathcal{M}\_\sigma$ such that for every continuous function $f \colon \Sigma\_d \to \mathbb{R}$, the map $m \mapsto \int f\,dm$ is a continuous function from $\mathcal{M}\_\sigma$ to $\mathbb{R}$.


This topology (which is metrisable) defines a Borel $\sigma$-algebra on $\mathcal{M}\_\sigma$. A linear combination of measures $\sum\_{i=1}^n\alpha\_im\_i \in \mathcal{M}\_\sigma$ satisfies
$$\int f\,d\left(\sum\_{i=1}^n \alpha\_i m\_i\right) = \sum\_{i=1}^n \alpha\_i \int f\,dm\_i$$
for every $f \in C(\Sigma\_d)$. More generally, if $\mathbb{P}$ is a Borel probability measure on the compact metrisable space $\mathcal{M}\_\sigma$, then defining
$$\int f\,d\mu:=\iint f \,dm\,d\mathbb{P}(m)$$
defines a measure $\mu$ on $\Sigma\_d$, which we abbreviate to $\mu=\int m\,d\mathbb{P}(m)$.


Now, it is easy to check that the set of Bernoulli measures is closed in the weak-\* topology: one uses the fact that a measure is $(p\_1,\ldots,p\_d)$-Bernoulli if and only if every cylinder set satisfies an equation in the weights,
$$\mu([x\_1\cdots x\_n])=p\_{x\_1}\cdots p\_{x\_n},$$
and $\mu([x\_1\cdots x\_n])$ is just $\int \chi \,d\mu$ where $\chi$ is the (continuous) indicator function of the (clopen) cylinder set $[x\_1\cdots x\_n]$. The set $C$ is thus the set of all measures of the form $\int m\,d\mathbb{P}$ where $\mathbb{P}$ is an atomic probability measure supported on the closed set $B$. Since $B$ is closed, the closure of $C$ is the set of all measures of the form $\int m\,d\mathbb{P}$ where $\mathbb{P}$ is an arbitrary Borel probability measure on $B$.


I am not quite sure how to answer the question "How big is the closure of $C$?" but I can at least say that it is an infinite-dimensional sub-simplex of $\mathcal{M}\_\sigma$, and its only ergodic elements are Bernoulli measures.


