I am currently reading "Smoothing of Multivariate Data" by Klemela. It contains Lemma 11.6, which upper and lower bounds the KL-divergence of two densities in terms of the $L\_{2}$-metric. The Lemma specifically states the following 2 bounds ***without*** proof:


**Upper bound**



> 
> Let $f, f\_{0}$ be densities. In particular, if $\inf \_{x \in \mathbf{R}^{d}} f\_{0}(x)>0$, then
> $$
> D\_{K}^{2}\left(f, f\_{0}\right) \leq \frac{\left\|f-f\_{0}\right\|\_{2}^{2}}{\inf \_{x \in \mathbf{R}^{d} f\_{0}(x)}}
> $$
> 
> 
> 


**Lower bound**



> 
> Also, if $f$ and $f\_{0}$ are both bounded and bounded away from zero, then
> $$
> D\_{K}^{2}\left(f, f\_{0}\right) \geq \int\_{\left\{x \in \mathbf{R}^{d}: f\_{0}(x)>0\right\}}\left(f-f\_{0}\right)+C\left\|f-f\_{0}\right\|\_{2}^{2}
> $$
> for a positive constant $C$.
> 
> 
> 


I can prove the upper bound rigorously as follows:
\begin{align}
 D\_{K}^{2}(f, f\_{0}) 
 &\leq \chi^{2}(f || f\_{0})
 \tag{well known upper bound} \\
 &=: \int\_{\left\{x \in \mathbf{R}^{d}: f\_{0}(x)>0\right\}} \frac{\left(f-f\_{0}\right)^{2}}{f\_{0}} 
 \tag{by definition} \\
 &\leq \frac{1}{\inf \_{x \in \mathbf{R}^{d} f\_{0}(x)}} 
 \int\_{\left\{x \in \mathbf{R}^{d}: f\_{0}(x)>0\right\}} 
 \left(f-f\_{0}\right)^{2}
 \tag{since $\inf$ exists.} \\
 &=: \frac{\left\|f-f\_{0}\right\|\_{2}^{2}}{\inf \_{x \in \mathbf{R}^{d} f\_{0}(x)}}
\end{align}


Note here that the **definition** of the KL divergence used in the Lemma is as follows:



> 
> If $f$ and $g$ are densities of $P$ and $Q$ with respect to the Lebesgue measure, then we may write
> $$
> D\_{K}^{2}(f, g)=\int\_{\mathbf{R}^{d} \cap\{x: g(x)>0\}} f \log \_{e}\left(\frac{f}{g}\right)
> $$
> 
> 
> 


I'm unsure on how to prove the lower bound. Some comments (for the lower bound to hold):


* It appears that the densities here must also be bounded above (else see first counterexample [here](https://mathoverflow.net/a/393477/486152)).
* Since the densities are assumed to be bounded above and positively bounded from below over their common support, the support set in $\mathbb{R}^{d}$ must also be bounded (else see second counterexample [here](https://mathoverflow.net/a/393477/486152)).


Could anyone please show how to prove this **lower bound** rigorously (or provide a citable rigorous proof reference)? Also, if I've made a mistake in my two comments above, please also let me know.


**Aside:** I had originally posted this on math.SE. To respect math.overflow [cross-posting etiquette](https://meta.mathoverflow.net/a/1726/486152), I've deleted that post due to no responses. I realized that it had also been asked [there before](https://math.stackexchange.com/questions/2635261/kullback-leibler-divergence-is-equivalent-to-l2-norm-in-some-cases) without any suitable responses. Since it is a research related question with no well citable proof, I believe it is fair to post here on math.overflow to settle the issue.



Lower bound for KL divergence of bounded densities and $L_{2}$ metric
$\newcommand{\ep}{\varepsilon}$As in your post and comments, suppose that $f$ and $f\_0$ are supported on a compact set $S$, and
\begin{equation\*}
 a\le f\le b,\quad a\le f\_0\le b
\end{equation\*}
on $S$ for some real $a,b$ such that $0<a<b$.


Then
\begin{equation\*}
 \int\_{\left\{x \in \mathbf{R}^{d}: f\_{0}(x)>0\right\}}\left(f-f\_{0}\right)
 =\int\_S\left(f-f\_{0}\right)=0. 
\end{equation\*}
So, the inequality in question is simply is
\begin{equation\*}
 D\_{K}^{2}\left(f, f\_{0}\right) \ge C\left\|f-f\_{0}\right\|\_{2}^{2}. \tag{1}\label{1} 
\end{equation\*}
By definition,
\begin{equation\*}
 D\_K^2(f,f\_0)=\int\_S f\ln\frac f{f\_0}=-\int\_S f\ln\frac{f\_0}f. 
\end{equation\*}
We have the elementary inequality
\begin{equation\*}
 \ln x\le(x-1)-\ep\_M(x-1)^2 \tag{2}\label{2} 
\end{equation\*}
for any real $M>1$ and all $x\in(0,M]$, where
\begin{equation\*}
 \ep\_M:=\frac{M-1-\ln M}{(M-1)^2}>0. 
\end{equation\*}
Note that $0<\frac{f\_0}f\le\frac ba$ on $S$. So, using \eqref{2} with $x=\frac{f\_0}f$ and $M=\frac ba$, we get
\begin{equation\*}
\begin{aligned}
 D\_K^2(f,f\_0)&\ge-\int\_S f\Big(\frac{f\_0}f-1-\ep\_{b/a}\Big(\frac{f\_0}f-1\Big)^2\Big) \\ 
& =\ep\_{b/a}\int\_S f\Big(\frac{f\_0}f-1\Big)^2
 =\ep\_{b/a}\int\_S \frac{(f\_0-f)^2}f \\ 
 &\ge\frac{\ep\_{b/a}}b\,\int\_S (f\_0-f)^2
 =C\_{a,b}\|f-f\_0\|\_2^2,
\end{aligned}
\end{equation\*}
where
\begin{equation\*}
 C\_{a,b}:=\frac{\ep\_{b/a}}b>0, 
\end{equation\*}
as desired.


