By KL divergence I mean $D(P||Q) = \int dP \log(\frac{dP}{dQ})$. I am looking for the conditions under which this strong convexity is true and possible references. I could not find an answer for infinite dimensions. I am specifically interested in the case where the dimension is uncountable.


Note that this strong convexity is equivalent to the strong convexity of negative entropy function $H(P) = \int dP \log(dP)$. 



Is KL divergence $D(P||Q)$ strongly convex over $P$ in infinite dimension
$\newcommand{\de}{\delta}
\newcommand{\De}{\Delta}
\newcommand{\ep}{\varepsilon}$


Take any probability measures $P\_0,P\_1$ absolutely continuous with respect (w.r.) to $Q$. 
We shall prove the following:



> 
> **Theorem 1.** For any $t\in(0,1)$,
>  \begin{align\*} \De:=(1-t)H(P\_0)+tH(P\_1)-H(P\_t) 
> \ge\frac{(1-t)t}2\,\|P\_1-P\_0\|^2, 
> \end{align\*}
>  where $\|P\_1-P\_0\|:=\int|dP\_1-dP\_0|$ is the total variation norm of $P\_1-P\_0$, 
>  \begin{equation\*}
>  H(P):=D(P||Q)=\int \ln\frac{dP}{dQ}\,dP,
> \end{equation\*}
>  and, for any elements $C\_0,C\_1$ of a linear space, $C\_t:=(1-t)C\_0+tC\_1$. 
>  Thus, by ["A third definition[8] for a strongly convex function"](https://en.wikipedia.org/wiki/Convex_function#Strongly_convex_functions), indeed $D(P||Q)$ is strongly convex in $P$ w.r. to the total variation norm. 
> 
> 
> 


We see that the lower bound on $\De$ does not depend on $Q$. 


*Proof of Theorem 1.* Take indeed any $t\in(0,1)$. Let $f\_j:=\frac{dP\_j}{dQ}$ for $j=0,1$, so that $f\_t=\frac{dP\_t}{dQ}$. By [Taylor's theorem with the integral form of the remainder](https://www.math.umd.edu/~jmr/141/remainder.pdf), for $h(x):=x\ln x$ and $j=0,1$ we have
\begin{equation\*}
 h(f\_j)=h(f\_t)+h'(f\_t)(f\_j-f\_t)+(f\_j-f\_t)^2\int\_0^1 h''((1-s)f\_t+sf\_j)(1-s)\,ds,
\end{equation\*}
whence 
\begin{align\*}
 \de&:=(1-t)h(f\_0)+th(f\_1)-h(f\_t) \\ 
 &=(1-t)t\,(f\_1-f\_0)^2\,
 \int\_0^1\Big(\frac t{(1-s)f\_t+sf\_0}+\frac{1-t}{(1-s)f\_t+sf\_1}\Big)(1-s)\,ds \\ 
 &=(1-t)t\,(f\_1-f\_0)^2\,
 \int\_0^1\Big(\frac t{f\_{u\_0(t,s)}}+\frac{1-t}{f\_{u\_1(t,s)}}\Big)(1-s)\,ds, 
\end{align\*}
where 
$$u\_j(t,s):=(1-s)t+js.$$ 
So,
\begin{equation}
 \De=\int\de\,dQ=(1-t)t\,\int\_0^1(1-s)\,ds\,[tI(u\_0(t,s))+(1-t)I(u\_1(t,s))], \tag{1}
\end{equation}
where 
\begin{equation\*}
 I(u):=\int\frac{(f\_1-f\_0)^2}{f\_u}\,dQ. 
\end{equation\*}
Next, take any $u\in(0,1)$. Then $P\_1$ is absolutely continuous w.r. to $P\_u$. Introducing  

$g\_u:=\dfrac{dP\_1}{dP\_u}=\dfrac{f\_1}{f\_u}$, we have 
\begin{multline\*}
 I(u)=\frac1{(1-u)^2}\,\int\frac{(f\_1-f\_u)^2}{f\_u}\,dQ
 =\frac1{(1-u)^2}\,\int(g\_u-1)^2\,dP\_u \\ 
 \ge\frac1{(1-u)^2}\,\Big(\int|g\_u-1|\,dP\_u\Big)^2
 =\frac1{(1-u)^2}\,\|P\_1-P\_u\|^2=\|P\_1-P\_0\|^2. \tag{2}
\end{multline\*}
Note also that for any $t\in(0,1)$ and $s\in(0,1)$ we have $u\_0(t,s)\in(0,1)$ and $u\_1(t,s)\in(0,1)$ and hence, by (2), $I(u\_j(t,s))\ge\|P\_1-P\_0\|^2$ for $j=0,1$. 
Now Theorem 1 follows by (1). 


*Remark.* The constant factor $\frac12$ in the lower bound in Theorem 1 is the best possible one.
Indeed, assuming that $P\_1$ is absolutely continuous w.r. to $P\_0$ and introducing $f:=\frac{dP\_1}{dP\_0}$, after some rather straightforward manipulations we get 
\begin{equation}
 \De=\int k(t,f)\,dP\_0, \tag{\*}
\end{equation}
where $\De$ is as before and 
\begin{equation}
 k(t,f):=t f \ln f-(1-t+t f)\ln(1-t+t f). 
\end{equation}
Take now any $h\in(0,1)$ and let $f$ take values $1-h,1+h$ each on a set of $P\_0$-measure $1/2$, so that $\|P\_1-P\_0\|=h$. Then, in view of (\*), for each $t\in(0,1)$, 
\begin{equation}
\De=\frac12\,k(t,1-h)+\frac12\,k(t,1+h)\sim \frac{(1-t)t}2\,h^2=\frac{(1-t)t}2\,\|P\_1-P\_0\|^2 
\end{equation}
as $h\downarrow0$, which confirms the optimality claim. 


