Background
----------


For a real-valued random variable $X$, define its entropy by $H(X) = E[\phi(X)] - \phi(E[X])$, where $\phi(u) = u \log u$.
It can be shown that, if the entropy satisfies the bound
$$
H(e^{\lambda X}) \leq \frac{\sigma^2 \lambda^2}{2} E[e^{\lambda X}]
$$
for all real $\lambda$, then $X$ must be a sub-Gaussian variable with parameter $\sigma$, by which I mean the following holds for all real $\lambda$:
$$
E[e^{\lambda (X - E[X])}] \leq \exp \left( \frac{\sigma^2 \lambda^2}{2} \right)
\text{.}
$$
This is known as the Herbst argument.


Question
--------


**Is the converse also true?** i.e. do any sub-Gaussian variables satisfy the following entropy bound?
$$
H(e^{\lambda X}) \leq \frac{\sigma^2 \lambda^2}{2} E[e^{\lambda X}] \quad \forall \lambda \in \mathbb{R}
$$



Converse of the Herbst argument?
As in the comment, the converse is true up to a factor of 4. The proof is given below.


Suppose $X$ is $\sigma^2/4$-subgaussian, i.e. the following holds for any $\lambda \in \mathbb{R}$:
$$
\mathbb{E}[e^{\lambda X}] \leq \exp \left( \lambda \mathbb{E}[X] + \frac{\lambda^2 \sigma^2}{8} \right) \text{.}
$$
Let $Z := e^{\lambda X} / \mathbb{E}[e^{\lambda X}]$. Then, one has
\begin{align\*}
\mathbb{E}[Z \log Z]
=&
\mathbb{E} \left[ \frac{e^{\lambda X}}{\mathbb{E}[e^{\lambda X}]} \log \frac{e^{\lambda X}}{\mathbb{E}[e^{\lambda X}]} \right]
\\=&
\frac{1}{\mathbb{E}[e^{\lambda X}]} \mathbb{E} \left[ e^{\lambda X} \left( \log e^{\lambda X} - \log \mathbb{E}[e^{\lambda X}] \right) \right]
\\=&
\frac{1}{\mathbb{E}[e^{\lambda X}]} 
\left(
\mathbb{E}[e^{\lambda X} \log e^{\lambda X}]
- \mathbb{E}[e^{\lambda X}] \log \mathbb{E}[e^{\lambda X}]
\right)
\\=&
\frac{H[e^{\lambda X}]}{\mathbb{E}[e^{\lambda X}]}
\text{.}
\end{align\*}
Moreover, note that
$$
\mathbb{E}[Z \log Z] = \frac{\mathbb{E}[e^{\lambda X} \log Z]}{\mathbb{E}[e^{\lambda X}]} = \mathbb{E}\_\lambda[\log Z]
\text{,}
$$
where $\mathbb{E}\_\lambda[f(X)] := \mathbb{E}[e^{\lambda X} f(X)] / \mathbb{E}[e^{\lambda X}]$ denotes the Gibbs expectation.
Now it suffices to show $\mathbb{E}\_\lambda[\log Z] \leq \frac{\lambda^2 \sigma^2}{2}$.


The concavity of $\log$ and the Jensen inequality gives
$$
\mathbb{E}\_\lambda[\log Z] \leq \log \mathbb{E}\_\lambda[Z] \text{.}
$$
By the definition of the Gibbs expectation, we have
\begin{align\*}
\mathbb{E}\_\lambda[Z]
=&
\frac{\mathbb{E}\_\lambda[e^{\lambda X}]}{\mathbb{E}[e^{\lambda X}]}
\\=&
\frac{\mathbb{E}[e^{2 \lambda X}] / \mathbb{E}[e^{\lambda X}]}{\mathbb{E}[e^{\lambda X}]}
\\=&
\frac{\mathbb{E}[e^{2 \lambda X}]}{\mathbb{E}[e^{\lambda X}]^2}
\end{align\*}
Now the assumption gives
$$
\mathbb{E}[e^{2 \lambda X}] \leq \exp \left(2 \lambda \mathbb{E}[X] + \frac{\lambda^2 \sigma^2}{2} \right)
\text{,}
$$
and the convexity of $\exp$ and the Jensen inequality yields $\mathbb{E}[e^{\lambda X}]^2 \geq e^{2 \lambda \mathbb{E}[X]}$.
Hence, we have
$$
\mathbb{E}\_\lambda[\log Z] \leq \log \frac{\exp \left(2 \lambda \mathbb{E}[X] + \frac{\lambda^2 \sigma^2}{2}\right)}{\exp(2 \lambda \mathbb{E}[X])} = \frac{\lambda^2 \sigma^2}{2}
$$
as desired.


