Sorry for the long title. What I mean is that for two vectors (a\_1,...,a\_n) and (b\_1,...,b\_n) with the property $a\_i,b\_i \geq 0 $ and $ \sum a\_i =\sum b\_i =1$. 


If $ -\sum a\_ilog(a\_i) > -\sum b\_ilog(b\_i) $ implies $\sum a\_i^2 < \sum b\_i^2$ or something similar?



Is there a relationship between Entropy of a fininte distrete probability distribution and the squre sum of the values of probability mass function of that distribution?
Of course, in general it is not true that inequality between entropies implies the same inequality between $\ell^2$ norms. Although this is true for $n=2$, it fails already for $n=3$ (as the entropy $H(p\_1,p\_2,p\_3)$ is obviously not constant on the level curve determined by conditions $\sum p\_i=1$ and $\sum p\_i^2=Const$). 


Nonetheless, there is a deep link between these two quantities. In order to explain it, it is better to somewhat change the viewpoint. Namely, given two probability distributions $P=(p\_1,\dots,p\_n)$ and $Q=(q\_1,\dots,q\_n)$ (for simplicity I assume that all $p\_i,q\_i$ are strictly positive), the corresponding **Kullback-Leibler deviation** is defined as 
$$
I(Q|P) = \sum p\_i \log \frac{p\_i}{q\_i} \;,
$$
and the so-called **information energy** as
$$
\chi^2(Q,P) = \sum \biggl(\frac{p\_i}{q\_i}-1\biggr)^2 q\_i \;.
$$
Both these quantities measure "closedness" of $P$ and $Q$ (although they are not distances) and are monotone invariants of the pair $(Q,P)$ (they do not increase under quotient maps). If $Q=U\_n$ is the uniform distribution, then these quantities are, up to linear rescaling, precisely the entropy and the $\ell^2$-norm of $P$, respectively, as
$$
I(U\_n|P) = \log n - H(P)
$$
and
$$
\chi^2(U\_n,P) = n \sum p\_i^2 - 1 \;.
$$


Now, the link between $I$ and $\chi^2$ is provided by the fact that in naturally defined infinitesimal limits the Kullback-Leibler deviation $I$ and the information energy $\chi^2$ coincide
and produce the **Fisher information metric**. One can read more about this in the corresponding wiki articles, in the old book *Statistical decision rules and optimal inference* by Čencov (AMS, 1982), or in more recent publications on **information geometry**.


