Let $I(X,Y):=H(X)+H(Y)-H(X,Y)$ be the mutual information of the joint probability distribution $p\_{XY}$ (here $H(\cdot)$ is the Shannon entropy of its argument). I know that the mutual information is invariant IF $X'=\text{invertible\_function}(X)$ and $Y'=\text{invertible\_function}(Y)$, that is, it is invertible under a re-parametrization.


Is the converse true? I.e., is it true that
$$I(X,Y) = I(X', Y')$$
implies that the marginals $X', Y'$ are invertible functions of $X$ and $Y$, respectively?


Thanks!



Invariance of mutual information
This is not true. Look at the following counterexample:


Suppose $X=X'$ and $Y'$ be a sufficient statistics of $Y$ with respect to $X$, that is we have a double Markov relation $X\to Y\to Y'$ and $X\to Y'\to Y$. The double Markov relation implies $I(X; Y')=I(X; Y)$. It also implies that either $X$ and $(Y, Y')$ are independent or $Y'$ is a deterministic function of $Y$. In non of these cases, $Y$ and $Y'$ are bijective.


There is another simple counterexample. Consider the trivial case that all $Y, Y'$ and $X$ are mutually independent; $I(X; Y)=I(X; Y')=0$, however, $Y$ and $Y'$ can be some arbitrary independent random variables.


