Let $0<a<b<c$ be distinct positive reals.


Define four different probability distributions:


$$\mathcal{P}\_{ab}:P\_{a,ab}=\frac{a}{a+b}=1-P\_{b,ab}$$
$$\mathcal{P}\_{bc}:P\_{b,bc}=\frac{b}{b+c}=1-P\_{c,bc}$$
$$\mathcal{P}\_{ca}:P\_{c,ca}=\frac{c}{c+a}=1-P\_{a,ca}$$
$$\mathcal{P}\_{abc}:P\_{a,abc}=\frac{a}{a+b+c},\mbox{ }P\_{b,abc}=\frac{b}{a+b+c},\mbox{ }P\_{c,abc}=\frac{c}{a+b+c}$$


Does Shannon entropy of a random variable from distribution $\mathcal{P}\_{abc}$ dominate the other three for all $a,b,c\in\Bbb R^+$ such that $0<a<b<c$?


In general when does $\mathcal{P}\_{abc}$ dominate?




---



Entropy dominance

> 
> Does Shannon entropy of a random variable from distribution $\mathcal{P}\_{abc}$ dominate the other three for all $a,b,c\in\Bbb R^+$ such that $0<a<b<c$?
> 
> 
> 


No.


On a sample space $\{A,B,C\}$ we can think of $\mathcal P\_{ab}$ as just $\mathcal P\_{abc}$ conditioned on the event that $C$ did not happen.


If the probability $x$ of $C$ is very large and the probabilities of $A$ and $B$ are equal and small, then the entropy of $\mathcal P\_{ab}$ will be larger than that of $\mathcal P\_{abc}$ as you can verify by looking at the graph of
$$-2((1-x)/2)\log\_2((1-x)/2)-x\log\_2(x),\quad x\in [1/3, 1]
$$


