Let $Y$ denote a Gaussian random variable characterized by a mean $\mu$ and a variance $\sigma^2$. Consider $N$ independent and identically distributed (i.i.d.) copies of $Y$, denoted as $Y\_1, Y\_2, \ldots, Y\_N$. Now, let's examine the number of $N$ for which the probability satisfies the inequality:


\begin{align}
\mathbb{P}\left[\left|-\frac{1}{N}\sum\_{i=1}^{N}\log f(Y\_i)-h(Y)\right|\geq\delta\right]\leq\epsilon,
\end{align}


Here, $f(y)$ represents the probability density function (pdf) of the Gaussian distribution, and $h(Y)$ represents the differential entropy of $Y$. It is worth noting that the expected value of the expression $-\frac{1}{N}\sum\_{i=1}^{N}\log f(Y\_i)$ is equal to $h(Y)$:


\begin{align}
\mathbb{E}\left[-\frac{1}{N}\sum\_{i=1}^{N}\log f(Y\_i)\right] = h(Y).
\end{align}


To analyze the probability inequality, we can employ a suitable concentration inequality. One approach is to consider the subgaussianity of the term $\log f(Y\_i) = \frac12\log(2\pi\sigma^2) - \frac{(Y\_i-\mu)^2}{2\sigma^2}$. Is considering the subgaussianity of $\log f(Y\_i)$ a valid approach? If so, could you provide guidance on how to compute the precise value of the subgaussianity parameter for $-\frac{1}{N}\sum\_{i=1}^{N}\log f(Y\_i)$?



Is it reasonable to consider the subgaussian property of the logarithm of the Gaussian pdf?
$\newcommand{\de}{\delta}\newcommand{\ep}{\epsilon}\newcommand{\si}{\sigma}\newcommand{\R}{\mathbb R}$We have
\begin{equation\*}
 L\_i:=-\ln f(Y\_i) = c+Z\_i^2/2,
\end{equation\*}
where $c:=\frac12\ln(2\pi\si^2)$ and $Z\_i:=\frac{Y\_i-\mu}\si$, so that the $Z\_i$'s are independent standard normal random variables (r.v.'s).


Clearly, the r.v.'s $L\_i$ are not sub-Gaussian, because for any real $s>0$
\begin{equation\*}
 Ee^{L\_i^2/s^2}=\int\_\R dz\,\frac1{\sqrt{2\pi}}e^{(c+z^2/2)^2/s^2-z^2/2}=\infty,
\end{equation\*}
since $e^{(c+z^2/2)^2/s^2-z^2/2}\to\infty$ as $z\to\infty$.


However, letting
\begin{equation\*}
 X\_i:=2(L\_i-EL\_i)=Z\_i^2-1,
\end{equation\*}
for all real $h<1/2$ we have
\begin{equation\*}
 R(h):=Ee^{hX\_i}=e^{-h}\int\_\R dz\,\frac1{\sqrt{2\pi}}e^{hz^2-z^2/2}=\frac{e^{-h}}{\sqrt{1-2h}}. \tag{1}\label{1}
\end{equation\*}
Note that for $h\in[0,1/2)$
\begin{equation\*}
 R(-h)\le R(h), \tag{2}\label{2}
\end{equation\*}
since for $g(h):=\ln\dfrac{R(-h)}{R(h)}$ we have $g(0)=0$ and $g'(h)=-\dfrac{8h^2}{1-4h^2}\le0$ for $h\in[0,1/2)$.


We want to find $n$ such that
\begin{equation\*}
 p\_n:=P\Big(\Big|-\frac1n\sum\_{i=1}^n\ln f(Y\_i)-h(Y)\Big|\ge\de\Big)\le\ep. \tag{3}\label{3}
\end{equation\*}
Note that
\begin{equation\*}
 p\_n=P(|S\_n|\ge2n\de), 
\end{equation\*}
where
\begin{equation\*}
 S\_n:=\sum\_{i=1}^n X\_i.
\end{equation\*}
So, we want to find $n$ such that
\begin{equation\*}
 P(|S\_n|\ge2n\de)\le\ep, 
\end{equation\*}
where $\de\in(0,\infty)$ and $\ep\in(0,1)$.
For all real $h\in[0,1/2)$, in view of \eqref{2} and \eqref{1},
\begin{equation\*}
\begin{aligned}
 P(|S\_n|\ge2n\de)&=P(S\_n\ge2n\de)+P(-S\_n\ge2n\de) \\ 
 &\le e^{-2n\de h}Ee^{hS\_n}+e^{-2n\de h}Ee^{-hS\_n} \\ 
 &=e^{-2n\de h}R(h)^n+e^{-2n\de h}R(-h)^n \\
 &\le2e^{-2n\de h}R(h)^n =2e^{nl(h)},
\end{aligned}
\end{equation\*}
where $l(h):=-h\de-h-\frac12\ln(1-2h)$. Next, the minimum of $l(h)$ over $h\in[0,1/2)$ occurs at $h=h\_\de:=\frac\de{2(1+\de)}$, and $e^{l(h\_\de)}=q(\de):=e^{-\de/2}\sqrt{1+\de}<1$.


So, \eqref{3} will hold if $2q(\de)^n\le\ep$, that is, if
\begin{equation}
 n\ge n\_{\de,\ep},
\end{equation}
where
\begin{equation}
 n\_{\de,\ep}=\frac{\ln(\ep/2)}{\ln q(\de)},
\end{equation}
so that $n\_{\de,\ep}$ is the root $n$ of the equation $2q(\de)^n=\ep$.
One may also note that
\begin{equation}
 n\_{\de,\ep}\sim\frac{4\ln(2/\ep)}{\de^2}
\end{equation}
as $\de\downarrow0$.


