We consider the two distributions
$$
p\_t = p\_0 \* N(0, tI),\quad q\_t = q\_0 \* N(0, t I),
$$
where $\*$ denotes the convolution between two densities, while $p\_0$ and $q\_0$ have the same mean and variance. In particular, we assume that $q\_0$ is $N(0, I)$. In other words, we consider two random variables
$$
X\_t = X\_0 + N(0, t I)\sim p\_t, \quad Y\_t = Y\_0 + N(0, tI)\sim q\_t,
$$
where $X\_0\sim p\_0$ and $Y\_0\sim q\_0 = N(0, I)$. 


We are interested in characterizing the evolution of 
$$
\text{KL}(p\_t, q\_t) = \int p\_t \log \left(\frac{p\_t}{q\_t}\right) dx
$$
along $t$. In particular, we aim to prove:


(i) $\text{KL}(p\_t, q\_t)$ is monotonically decreasing along $t$;


(ii) $\text{KL}(p\_t, q\_t)$ is convex along t;


(iii) $\text{KL}(p\_t, q\_t)$ is smooth along t, that is, 
$\frac{d^2\text{KL}(p\_t, q\_t)}{dt^2}
$ is upper bounded. 


(Comment: As pointed out by Jon, (i) follows directly from the data processing inequality. As pointed out by Nawaf, (ii) also holds.)



Monotonicity, Convexity, and Smoothness of the KL-Divergence between Two Brownian Motions with Different Initializers
Write the KL divergence in terms of the [differential entropy](https://en.wikipedia.org/wiki/Differential_entropy) of the random variables $X\_t$ and $Y\_t$; the result quickly follows. Indeed, since $Y\_t \sim \mathcal{N}(0,1+t)$, we have \begin{align\*}
\operatorname{KL}(p\_t, q\_t) &= - h(p\_t) + \frac{1}{2} \int \frac{x^2}{1+t} p\_t(x) dx + \frac{1}{2} \log(1+t) + \frac{1}{2} \log(2 \pi) \\
&= - h(p\_t) + \frac{1}{2} \mathbb{E}((X\_0+\sqrt{t} \mathcal{N}(0,1))^2)\frac{1}{1+t} + \frac{1}{2} \log(1+t) + \frac{1}{2} \log(2 \pi) \\
&= -h(p\_t) + h(q\_t)
\end{align\*} where $h(\cdot)$ is the differential entropy. By Lemma 2 of [Zhang, Anantharam and Geng](https://www.mdpi.com/1099-4300/20/3/182/htm), subject to $\operatorname{var}(X\_0)=1$, the minimum of $-\frac{d^2}{dt^2} h(p\_t)$ is achieved when $X\_0$ is Gaussian. Thus, $-\frac{d^2}{dt^2} h(p\_t) \ge -\frac{d^2}{dt^2} h(q\_t)$, and hence, $\frac{d^2}{dt^2}\operatorname{KL}(p\_t, q\_t) \ge 0$ which implies that $\operatorname{KL}(p\_t, q\_t)$ is convex with respect to $t$.


**ADD**


The Gaussian minimality result used above seems to go back to


McKean, H. P., [**Speed of approach to equilibrium for Kac’s caricature of a Maxwellian gas**](http://dx.doi.org/10.1007/BF00264463), Arch. Ration. Mech. Anal. 21, 343-367 (1966). [ZBL1302.60049](https://zbmath.org/?q=an:1302.60049).


