Let $I(Y;X)$ denote the mutual information between $Y$ and $X$. If we have $I(Y;X\_{i}) < B$ for all $i \quad (1 \leq i \leq N)$, could we also get the upper-bound of $I(Y; X\_{1}, X\_{2}, ..., X\_{N})$?



Upper bound of $I(Y; X_{1}, ..., X_{N})$ when we have $I(Y;X_{i}) < B$ for all $i$ $(1 \leq i \leq N)$
Fix any integer $N\ge2$. Let $X:=(X\_1,\dots,X\_N)$. There is no upper bound $U\_N(B)$ on $I(Y;X)$ such that $U\_N(B)\to0$ whenever $\max\_{i=1}^N I(Y;X\_i)< B\downarrow0$. That is, each of the $I(Y;X\_i)$'s may be arbitrarily small while $I(Y;X)$ is not small. Informally, each $X\_i$ may contain little or no information about the distribution of $Y$, but the information about the distribution of $Y$ contained in all the $X\_i$'s together may be essential, even for $N$ as small as $2$.


Indeed, suppose that $X\_1,\dots,X\_N$ are independent Rademacher random variables (r.v.'s), so that $P(X\_i=1)=P(X\_i=-1)=1/2$ for each $i$. Then $Y:=X\_1\cdots X\_N$ is a Rademacher r.v. independent of each of the $X\_i$'s. So, $\max\_{i=1}^N I(Y;X\_i)=0<B$ for any real $B>0$. However (assuming the natural logarithm in the definition of the mutual information), $I(Y;X)=\ln2\not\to0$ as $B\downarrow0$.


**Details on the equality $I(Y;X)=\ln2$:** Assuming the usual convention $0\ln0:=0$, we have
\begin{equation}
\begin{aligned}
 &I(Y;X) \\ 
 &=\sum\_{(y,x\_1,\dots,x\_N)\in\{-1,1\}^{N+1}}
P(Y=y,X\_1=x\_1,\dots,X\_N=x\_N) \\
&\qquad\qquad\qquad\times\ln\frac{P(Y=y,X\_1=x\_1,\dots,X\_N=x\_N)}{P(Y=y)P(X\_1=x\_1,\dots,X\_N=x\_N)} \\ 
&=\sum\_{(y,x\_1,\dots,x\_N)\in\{-1,1\}^{N+1}}
P(X\_1=x\_1,\dots,X\_N=x\_N) \\
&\qquad\qquad\qquad\times1(y=x\_1\cdots x\_N)\,
\ln\frac{P(X\_1=x\_1,\dots,X\_N=x\_N)}{\tfrac12\,P(X\_1=x\_1,\dots,X\_N=x\_N)} \\ 
&=\sum\_{(x\_1,\dots,x\_N)\in\{-1,1\}^N}
P(X\_1=x\_1,\dots,X\_N=x\_N) \\
&\qquad\qquad\qquad\times
\ln\frac{P(X\_1=x\_1,\dots,X\_N=x\_N)}{\tfrac12\,P(X\_1=x\_1,\dots,X\_N=x\_N)}=\ln2. 
\end{aligned}
\end{equation}


