The following information theoretic inequality is needed in my work.


Let $n, m, n\_1, n\_2, \dots, n\_k \in \mathbb{Z}^+$ such that $m < n = n\_1 + n\_2 + \dots + n\_k$. I would like to prove that with condition $\sum\_{i=1}^k \min \{n\_i, m\} \geq \alpha$ we have
$$
\sum\_{i=1}^k \frac{n\_i}{n} \log \frac{n}{n\_i} \geq \frac{\alpha - m}{n - m} \log \frac{n}{m}
$$
Of course we can prove it by the method of adjustment. But it would be a little lengthy. Since the proof technique has nothing to do with my work, I am coming here and asking for help about a short proof or preferably, a proof in some other's work (so that I could simply refer to it without making a proof by myself).




---


A short proof when $m=1$.


$\sum\_{i=1}^k \min \{n\_i, m\} \geq \alpha \implies k \geq \alpha$. Let $H(p\_1,p\_2,\dots,p\_k) = -\sum\_{i=1}^k p\_i \log p\_i$. We have that point
$
(\frac{n\_1}{n}, \frac{n\_2}{n}, \dots, \frac{n\_k}{n})
$
is a weighted arithmetic average of $(\frac{n-k+1}{n}, \frac{1}{n}, \dots, \frac{1}{n}),\ (\frac{1}{n}, \frac{n-k+1}{n}, \dots, \frac{1}{n}),\ \dots,\ (\frac{1}{n}, \frac{1}{n}, \dots, \frac{n-k+1}{n})$. So by that $H$ is concave we have
$$
\sum\_{i=1}^k \frac{n\_i}{n} \log \frac{n}{n\_i} \geq (k-1) \frac{1}{n}\log n+ \frac{n-k+1}{n}\log \frac{n}{n-k+1} \geq \frac{k-1}{n-1} \log n.
$$



Finding a short proof for a certain information theoretic inequality
Assume that $n\_1,\dots,n\_t<m\leqslant n\_{t+1},\dots,n\_k$. Denote $p\_i=n\_i/n$, $m/n=a$; $H(p)=-p\log p$ is entropy function, and we want to prove $$\sum H(p\_i)\geqslant \frac{p\_1+\dots+p\_t+a(k-t-1)}{1-a}\log a^{-1}.$$
Note that $H(p)$ is concave function, thus we have $H(p\_i)\geqslant H(a)\cdot \frac{p\_i}a$ for $i\leqslant t$ and $H(p\_i)\geqslant H(a)\cdot \frac{1-p\_i}{1-a}$ for $i>t$. Substituting these estimates for $H(p\_i)$ to LHS and simplifying we get exactly RHS.


