Let $\mu$ and $\nu$ be two multivariate Gaussian measures on $\mathbb{R}^d$ with non-singular covariance matrices. Can the Fisher-Rao distance $d(\mu,\nu)$ computed on the information manifold of non-generated $d$-dimensional Gaussian measures with Fisher-Rao metric, be bounded by the (symmetrized) KL divergence/relative-entropy between $\mu$ and $\nu$?



Upper-bound on the Fisher-Rao distance between multivariate Gaussian measures by the KL-divergence
Since relative entropy behaves locally like a squared distance, we might expect the *squared* Fisher-Rao metric to be comparable to the symmetrized KL divergence. This is indeed the case.


Let $d\_F$ denote the Fisher-Rao metric on the manifold of non-degenerate multivariate Gaussians, and let $D(\mu,\nu):= D\_{KL}(\mu\|\nu) + D\_{KL}(\nu\|\mu)$ denote the symmetrized KL divergence between measures $\mu,\nu$.


**Claim**: For multivariate Gaussian measures $\mu\_1,\mu\_2$ with nonsingular covariance matrices, we have
$$
d\_F(\mu\_1,\mu\_2)^2 \leq 2 D(\mu\_1,\mu\_2).
$$


*Proof*: By the triangle inequality, we have
\begin{align\*}
d\_F\big(N(\theta\_1,\Sigma\_1),N(\theta\_2,\Sigma\_2)\big)^2 
&\leq \big(d\_F(N(\theta\_1,\Sigma\_1),N(\theta\_1,\Sigma\_2) )+d\_F(N(\theta\_1,\Sigma\_2),N(\theta\_2,\Sigma\_2) ) \big)^2\\
&\leq 2 d\_F\big(N(\theta\_1,\Sigma\_1),N(\theta\_1,\Sigma\_2)\big)^2 + 2 d\_F\big(N(\theta\_1,\Sigma\_2),N(\theta\_2,\Sigma\_2)\big)^2 
\end{align\*}


On the submanifold of Gaussians with common mean, the (squared) Fisher-Rao distance is equal to
$$
d\_F\big(N(\theta\_1,\Sigma\_1),N(\theta\_1,\Sigma\_2)\big)^2 = \frac{1}{2}\sum\_{i}(\log \lambda\_i)^2,
$$  

where $(\lambda\_i)$ denote the eigenvalues of the matrix $\Sigma\_2^{-1/2}\Sigma\_1\Sigma\_2^{-1/2}$. On the submanifold of Gaussians with common covariance, the (squared) Fisher-Rao distance is equal to
$$
d\_F\big(N(\theta\_1,\Sigma\_2),N(\theta\_2,\Sigma\_2)\big)^2 = (\theta\_1-\theta\_2)^T \Sigma\_2^{-1} (\theta\_1-\theta\_2).
$$


The symmetrized KL divergence is given by
$$
D(N(\theta\_1,\Sigma\_1),N(\theta\_2,\Sigma\_2)) = \frac{1}{2}\sum\_{i}\left(\lambda\_i +\frac{1}{\lambda\_i}- 2 \right) +\frac{1}{2} (\theta\_1-\theta\_2)^T (\Sigma\_1^{-1}+\Sigma\_2^{-1}) (\theta\_1-\theta\_2).
$$


Now, the claim follows on account of the inequality
$$
(\log x)^2 \leq x + \frac{1}{x}-2, ~~~x>0.
$$


*Remark*: The closed-form expressions for the special cases of the Fisher-Rao metric used above can be found in Section 2.1 (and references therein) of:


Pinele, Julianna, João E. Strapasson, and Sueli IR Costa. "The Fisher-Rao distance between multivariate normal distributions: Special cases, bounds and applications." Entropy 22.4 (2020): 404.


