Consider two mean centered multivariate normal densities $N(0,\Sigma\_{1})$ and $N(0,\Sigma\_{2})$. Are there known expressions (as opposed to bounds provided by the Pinsker inequality) for the total variation distance between such densities? 


Let $P\_{1}=\Sigma\_{1}^{-1}$ and $P\_{2}=\Sigma\_{2}^{-1}$. I am looking for either a general expression for the total variation distance or for expressions in special cases such as when (i) $P\_{1}-P\_{2}$ is positive definite or (ii) when $P\_{1} =k P\_{2}$ for some positive constant $k>1$. In special case (ii), the distance should be expressible using multivariate generalizations of the error function but I don't have a numeric approximation for the multivariate generalization of the error function.



Are there known expressions for total variation distance between mean centered multivariate normal densities $N(0,\Sigma_{1})$ and $N(0,\Sigma_{2})$
After some work, I have come up with the following answer.


In the general case, the following holds for any two multivariate normal densities:
 \begin{array}{rcl} f(\mathbf{x}|\mathbf{0},\mathbf{\Sigma}\_{1}) & \geq & f(\mathbf{x}|\mathbf{0},\mathbf{\Sigma}\_{2}) \\
&\Updownarrow & \\
-\frac{1}{2} x^{T} \mathbf{P}\_{1}x + \frac{1}{2} log [|\mathbf{P}\_{1}|] & \geq & -\frac{1}{2} x^{T} \mathbf{P}\_{2}x + \frac{1}{2} log [|\mathbf{P}\_{2}|] \\
& \Updownarrow & \\
x^{T}[ \mathbf{P}\_{1}-\mathbf{P}\_{2} ]x & \leq & log [|\mathbf{P}\_{1}|] - log [|\mathbf{P}\_{2}|]
\end{array} 


The set $A:=\{x| f(x|\mathbf{0},\mathbf{\Sigma\_{1}})\geq f(x|\mathbf{0},\mathbf{\Sigma}\_{2})\}$ is a compact and convex set if $\mathbf{P}\_{1}-\mathbf{P}\_{2}$ is a positive definite matrix as in special cases (i). Moreover, if $\mathbf{P}\_{1} = k \times \mathbf{P}\_{2}$ for $k>1$ then 


$ \begin{array}{rcl} f(x|\mathbf{0},\mathbf{\Sigma}\_{1}) & \geq & f(x|\mathbf{0},\mathbf{\Sigma}\_{2}) \\
&\Updownarrow & \\
x^{T} \mathbf{P}\_{1} x & \leq & \frac{k}{k-1} n \times log [k ] \\
&\Updownarrow & \\
x^{T} \mathbf{P}\_{2} x & \leq & \frac{1}{k-1} n \times log [k ]. 
\end{array}
 $


In [A Generalized Error Function in n-dimensions](http://www.dtic.mil/dtic/tr/fulltext/u2/401722.pdf/), M. Brown defines an n-dimensional generalized error function $erf\_{n}(.): \mathbf{R}\_{+} \rightarrow \mathbf{R}$ by


$ erf\_{n}(x) = \frac{\int\_{0}^{x} e^{-u^{2}} u^{n-1} d u }{\int\_{0}^{\infty} e^{-u^{2}} u^{n-1} d u} $


Equation [47] in the paper concerns diagonal variance-covariance matrices and states that 


$Prob[\sum\_{i=1}^{n} \frac{x\_{i}^{2}}{\sigma\_{i}^{2}} \leq \beta^{2}] = erf\_{n}(\frac{\beta}{\sqrt{2}}). $


If this equations extends to the general case so that 


$ Prob[ x^{T} \Sigma^{-1} x \leq \beta^{2}] = erf\_{n}(\frac{\beta}{\sqrt{2}}) $


then the total variation distance under special case (ii) can be expressed as


$\begin{array}{rcl} ||f(.|\mathbf{0},\mathbf{\Sigma}\_{1}) - f(.|\mathbf{0},\mathbf{\Sigma}\_{2})||\_{TV} & = & erf\_{n}(\frac{\sqrt{ \frac{k}{k-1} n \times \ln [k ]}}{\sqrt{2}}) - erf\_{n}(\frac{\sqrt{ \frac{1}{k-1} n \times \ln [k ] }}{\sqrt{2}}). \end{array} $


As per the [Brown paper](http://www.dtic.mil/dtic/tr/fulltext/u2/401722.pdf/), the error function is given specifically by 


$ erf\_{2m}(x) = 1 -e^{-x^{2}}[1 + \frac{x^{2}}{1!} +\frac{x^{4}}{2!} + \ldots+\frac{x^{2(m-1)}}{(m-1)!}] $ if $n$ is of even dimensions and by 
$erf\_{2m+1}(x) = erf\_{1}(x) -\frac{e^{-x^{2}}}{\sqrt{\pi}}[\frac{(2x)0!}{1!} + \frac{(2x)^{3}1!}{3!} + \ldots+\frac{(2x)^{2m-1}(m-1)!}{(2m-1)!}] $


if $n$ is of odd dimension.


Considering the one-dimensional case and letting $k=(1+\epsilon)^{2}$, we get 


$ \begin{array}{rcl} ||f(.|\mathbf{0},\mathbf{\Sigma}\_{1}) - f(.|\mathbf{0},\mathbf{\Sigma}\_{2})||\_{TV} & = & erf\_{1}(\frac{\sqrt{ \frac{k}{k-1} \times \ln [k ]}}{\sqrt{2}}) - erf\_{1}(\frac{\sqrt{ \frac{1}{k-1} \times \ln [k ] }}{\sqrt{2}}) \\
& = & erf\_{1}(\frac{\sqrt{ \frac{(1+\epsilon)^{2}}{(1+\epsilon)^{2}-1} \times \ln [(1+\epsilon)^{2} ]}}{\sqrt{2}}) \\
& & - erf\_{1}(\frac{\sqrt{ \frac{1}{(1+\epsilon)^{2}-1} \times \ln [(1+\epsilon)^{2} ] }}{\sqrt{2}}) \\
& = & erf\_{1}(\frac{(1+\epsilon) \sqrt{ \frac{1}{\epsilon(2+\epsilon)} \times 2\times \ln [(1+\epsilon) ]}}{\sqrt{2}}) \\
& & - erf\_{1}(\frac{\sqrt{ \frac{1}{\epsilon(2+\epsilon)} \times 2 \times \ln [(1+\epsilon) ] }}{\sqrt{2}}) \\
& = & erf\_{1}(\frac{(1+\epsilon) \sqrt{ \ln (1+\epsilon) }}{\sqrt{\epsilon(2+\epsilon)}}) \\
& & - erf\_{1}(\frac{ \sqrt{ \ln (1+\epsilon) }}{\sqrt{\epsilon(2+\epsilon)}})
 \end{array} $


which corresponds to one of the [answers](https://mathoverflow.net/questions/156946/are-there-known-expressions-for-total-variation-distance-between-n0-sigma-12?rq=1) to a univariate version of this question.


The above can be partially extended to all of special case (i) to provide a bound on the total variation distance in terms of error functions. 


Define a function $g\_{1}(.): A \rightarrow \mathbf{R}\_{+}$ by $g\_{1}(x)=x^{T}P\_{1}x$. Let $b\_{1} =\max\_{x\in A} g\_{1}(x)$. Since $A$ is a compact convex set and the function $g(.)$ is continous, this maximum is well defined under the present assumptions. We we can hence define a set $A\_{1}^{\*}:=\{x \in \mathbf{R}^{n}| x^{T}P\_{1} x \leq b\_{1}\}$. Clearly, $A \subseteq A\_{1}^{\*}$. 


Similarly, define a function $g\_{2}(.): A \rightarrow \mathbf{R}\_{+}$ by $g\_{2}(x)=x^{T}P\_{2}x$ and define a parameterized family of sets $A\_{2}(.) : g\_{2}(A) \rightarrow \mathbf{R}\_{+}^{n}$ by $A\_{2}(b)=\{x \in \mathbf{R}^{n}| x^{T}P\_{2} x \leq b\}$. Let $A\_{2}^{\*}= \{x \in A|A\_{2}(g\_{2}(x)) \subseteq A)\}$ and $b\_{2}=\max\_{x\in A\_{2}^{\*}} g\_{2}(x)$. This maximum exists under the present assumptions since the set $A\_{2}^{\*}$ can be shown to the non-empty, convex, and compact. Clearly, $A\_{2}^{\*} \subseteq A$.


We can now provide a bound for the total variation distance in terms of error functions that generalizes the expressions for the total variation in special case (ii):


$ \begin{array}{rcl} ||f(.|\mathbf{0},\mathbf{\Sigma}\_{1}) - f(.|\mathbf{0},\mathbf{\Sigma}\_{2})||\_{TV} & = & \int\_{x \in A} f(x|\mathbf{0},\mathbf{\Sigma}\_{1}) dx -\int\_{x \in A} f(x|\mathbf{0},\mathbf{\Sigma}\_{2}) dx \\
& \leq & \int\_{x \in A\_{1}^{\*}} f(x|\mathbf{0},\mathbf{\Sigma}\_{1}) dx -\int\_{x \in A\_{2}^{\*}} f(x|\mathbf{0},\mathbf{\Sigma}\_{2}) dx \\
& = & erf\_{n}(\frac{\sqrt{ b\_{1}}}{\sqrt{2}}) - erf\_{n}(\frac{\sqrt{b\_{2} }}{\sqrt{2}}). \end{array} $


In special case (ii), $b\_{1}=\frac{k}{k-1} n \times \ln [k]$ and $b\_{2}=\frac{1}{k-1} n \times \ln [k]$. 


While not generally available in closed form solution, increasingly good estimates for $b\_{1}$ and $b\_{2}$ can be generated through repeated simulation from centered multivariate normal densities with precision matrices $P\_{1}-P\_{2}$. To generate estimates for $b\_{1}$ and $b\_{2}$, let $B=\{x^{(i)}\}\_{i=1}^{L}$ be a set of (non-zero) random draws from a mean centered multivariate normal with precision matrix $P\_{1}-P\_{2}$. For each $x^{(i)} \in B$, define 


$\begin{array}{rcccccl}
t(x^{(i)}) &= & \sqrt{\frac{log [|\mathbf{P}\_{1}|] - log [|\mathbf{P}\_{2}|]}{(x^{(i)})^{T}[ \mathbf{P}\_{1}-\mathbf{P}\_{2} ]x^{(i)} }} & , & \tilde{x}(x^{(i)}) & = & t(x^{(i)}) \times x^{(i)} \\
\tilde{b}\_{1}(x^{(i)}) &= & ( \tilde{x}(x^{(i)}))^{T} P\_{1}\tilde{x}(x^{(i)}) & , & \tilde{b}\_{2}(x^{(i)}) & = & ( \tilde{x}(x^{(i)}))^{T} P\_{2}\tilde{x}(x^{(i)}) \end{array}$ 


and estimate $b\_{1}$ and $b\_{2}$ by 
$\hat{b}\_{1} = \max\_{x^{(i)} \in B}\tilde{b}\_{1}(x^{(i)})$ and $\hat{b}\_{2} = \min\_{x^{(i)} \in B} \tilde{b}\_{2}(x^{(i)}) $ respectively.


As the number of draws from the multivariate normal density increase, $E[\hat{b}\_{1}] \rightarrow b\_{1}$ and $E[\hat{b}\_{2}] \rightarrow b\_{2}$ where the former convergence is from the below and the latter from above.


