Suppose I have three identically-distributed homogeneous continuous-time discrete state space Markov chains $X\_1(t), X\_2(t), X\_3(t)$, $t\geq 0$. They evolve independently but share a common random variable $X\_0$ as an initial condition.
I let $$X\_1=X\_1(t\_1), \ \ \ X\_2=X\_2(t\_2), \ \ \ X\_3=X\_3(t\_3)$$ for some times $t\_1, t\_2, t\_3\geq 0$.


I want to show that
$$
I(X\_1; X\_2; X\_3) \geq 0
$$
where $I$ is the [multivariate mutual information](https://en.wikipedia.org/wiki/Interaction_information) (or information interaction)
$$
I(A,B,C)= H(A,B,C) -H(A,B) - H(B,C) - H(A,C) + H(A) + H(B) + H(C)
$$
where $H$ is the usual Shannon entropy.


Background/Motivation
---------------------


There are well-known situations where $I(A;B;C)<0$, a famous one being if $A$ and $B$ are independent random variables, each $\pm 1$ with probability $1/2$, and $C=AB$. But I conjecture that in the case I have described above $I(X\_1;X\_2;X\_3)\geq 0$. I believe that the Markov chains being continuous-time and homogeneous is essential.


The more general motivation is that I want to find very general situations where multivariate mutual information is non-negative. (One well-known example is if $A,B,C$ form a Markov chain.)



Is this a situation where triple mutual information is always non-negative?
My conjecture in the original question is false.


Let $X\_i(t)$ be continuous time Markov chains with two states 0 and 1, such that the rate of transition from 0 to 1 and from 1 to 0 is $1$. Let $X\_0$ be 0 with probability $0.9$ and 1 with probability $0.1$. Choose $t\_1=t\_2=t\_3$ such that the probability that $X\_i=X\_0$ is $3/4$ for all $i$.


Then for all $i$,
$$
H(X\_i)=0.88129 \ldots,
$$
and for all $i\neq j$,
$$
H(X\_i,X\_j)= 1.75448 \ldots,
$$
and 
$$
H(X\_1,X\_2,X\_3)=2.61812 \ldots
$$
This gives
$$
I(X\_1;X\_2;X\_3) = -0.00144 \ldots < 0.
$$


I don't yet have any insight into why the triple mutual information is negative. This is about as negative as I could make it with the two state symmetric Markov chain.


