Consider a collection of positive integers $\{a\_i\}\_{i=1}^m$ and the distribution $p\_i=\frac{a\_i}{\sum\_{i=1}^ma\_i}$.


Similarly for the collection $\{a\_i\}\_{i=1}^{m+1}$ form the distribution $q\_i=\frac{a\_i}{\sum\_{i=1}^{m+1}a\_i}$, for the collection $\{a\_i\}\_{i=1}^{n}$ form the distribution $r\_i=\frac{a\_i}{\sum\_{i=1}^{n}a\_i}$ and for the collection $\{a\_i\}\_{i=1}^{n+1}$ form the distribution $s\_i=\frac{a\_i}{\sum\_{i=1}^{n+1}a\_i}$.


Let $m+1\leq n$. Note all collections are subsets of the collection for the distribution $s$.


For the cases $(1)$ $a\_{i+1}=a\_i+\theta(\log^ka\_i)$, $(2)$ $a\_{i+1}=a\_i+\theta(a\_i)$ where $k$ is a postive constant, is it true that the difference in entropy is strictly decreasing? That is, is $H(q)-H(p)>H(s)-H(r)$ valid? When can one expect difference in Shannon entropy to be dominant in sequences of these types? Clearly in this post [Entropy dominance](https://mathoverflow.net/questions/191391/entropy-dominance), one criteria for negative result is given. Can $a\_i$ be any larger or smaller than in the cases given?




---


Can we guess $\lim\_{n\rightarrow\infty}H(s)=\infty$ in all these cases?


What spacing in $a\_i$ would guarantee $\lim\_{n\rightarrow\infty}H(s)<c\in\Bbb N$?



Entropy difference dominance of sequences
Here's some examplary evidence that maybe the answers are Yes.


* For Case 2, let $a\_{i+1}=2a\_i$ for all $i$ with $a\_1=1$.
Then the binary entropy is
$$
-\frac1{2^{n}-1} \sum\_{k=0}^{n-1} 2^k \log\_2\left(\frac{2^k}{2^{n}-1}\right)$$
$$
\approx-\frac1{2^{n}} \sum\_{k=0}^{n-1} 2^k \log\_2\left(\frac{2^k}{2^{n}}\right)$$
$$
=\sum\_{k=0}^{n-1} \frac{n-k}{2^{n-k}}=\frac{2^{n+1}-n-2}{2^n}\nearrow 2$$
which is the entropy of the geometric distribution with parameter $1/2$. So it seems that in this case yes, the difference in entropy is decreasing.
A similar calculation woks if $a\_{i+1}=u\cdot a\_i$ for a positive integer $u>2$.
* For Case 1, well let's go all the way to the extreme case $a\_i=1$ for all $i$. In this case the entropy is just $\log n$ and so by concavity of log, yes, the difference in entropy is decreasing.


