Prove that the [Normal (Gaussian) Distribution](https://en.wikipedia.org/wiki/Normal_distribution) with a given Variance $ {\sigma}^{2} $ maximizes the [Differential Entropy](https://en.wikipedia.org/wiki/Differential_entropy) among all distributions with defined and finite 1st Moment and Variance which equals $ {\sigma}^{2}
 $. 



Differential Entropy of Random Signal
Cover and Thomas's book is indeed the right place to learn about this.


The statement basically follows by convexity, in the form of Jensen's inequality. Here is the way it is usually presented:


Let $f$ be the probability density of a real random variable. Then the Shannon entropy is given by


$-\int f\log f dx$


You want to prove among all real random variables with finite Shannon entropy and variance equal to $1$, the Shannon entropy is maximized only for Gaussians.


Given two probability densities $f$ and $g$, since $\log$ is a concave function, Jensen's inequality tells us that


$\int f \log (g/f) dx < \log \int f(g/f) dx = \log \int g dx = 0$


Moreover, since $\log$ is strictly concave, equality holds if and only if $g = f$. If you now set $g$ equal to the probability density of a Gaussian with the same variance as $f$ and plug in the explicit formula for $g$, you get what you want.


