Let $A$, $B$ and $C$ be three random variables and $p\_{A,B,C}=p\_Ap\_Bp\_{C|A,B}$ and $q\_{A,B,C}=p\_Aq\_{B|A}p\_{C|A,B}$ be two distributions on them. Then, we can conclude that?
\begin{align\*}
\lvert I\_p(A,B;C)-I\_q(A,B;C)\rvert\leq\log\lvert\mathcal{B}\rvert.
\end{align\*}
where $\mathcal{B}$ is the alphabet set of $B$ and $I(.;.)$ is the mutual information.



Bounding difference of two mutual information with different distributions on random variables
The set-up is the following.



> 
> We have random variables $A,B,C$ and $A', B', C'$. We know $A$ and $A'$ have the same distribution, and for all $(a,b)$, the random variable $C | (A,B)=(a,b)$ has the same distribution as $C' | (A', B') = (a,b)$.
> 
> 
> 


Under the above hypothesis, the claim you want is in fact false. Here's a counterexample.


Let $A$ and $A'$ be uniformly distributed over $\{1, 2, \ldots, a\}$. Let $B$ and $B'$ be independently drawn from $\{0,1\}$ such that $\mathbb{P}(B=0) = \mathbb{P}(B'=1) = \varepsilon$. And let $C = AB$ and $C' = A' B'$ be the product of the other two.


(I'm allowing the $\varepsilon$ in there just so that we don't have to condition on any events having probability $0$ in the definitions of $C$ and $C'$, but you should imagine $\varepsilon \approx 0$. And every $\approx$ below becomes equality in the limit as $\varepsilon \to 0$.)


Then we have $H(A,B) = H(A', B') = H(A) + H(B) \approx H(A) = \log(a)$. On the other hand, $H(A,B | C) \approx 0$ since with probability $1-\varepsilon$, $(A,B)$ is uniquely determined by $C$. Similarly $H(A', B' | C') \approx H(A')$ since $C'$ usually tells us nothing about $A'$. This give us


$$
I(A, B; C) - I(A', B'; C') = H(A,B) - H(A,B|C) - [H(A',B') - H(A',B'|C')] = H(A',B'|C') - H(A,B|C) \approx \log(a),
$$
which can be made arbitrarily large.


