Let $P = N(\vec{0}, I^d)$ be a standard multivariate Gaussian distribution in $d$ dimensions. Let $Q$ be distributed the same as $P$, except that samples from $Q$ have one of their coordinates, chosen uniformly at random from $1$ to $d$, distributed as $N(\mu, \sigma^2)$ instead.


That is, $Q$ is a mixture of $d$ multivariate Gaussians, each with weight $\frac1d$. The $i$th mixture component has mean $\vec{0} + \vec{e}\_i \cdot \mu$ and variance with diagonal $\vec{1} + \vec{e}\_i \cdot (\sigma^2 - 1)$.


I'd like to bound the total variation distance between $P$ and $Q$, as a function of $d$ (when $\mu$ and $\sigma$ are constants that do not depend on $d$).


My intuition is that $Q$ is very close to a multivariate Gaussian $N(\frac{\vec{\mu}}{d}, I^d \cdot (1+\frac{\sigma^2}{d}+ o(\frac1d))$, which would give a Kullback–Leibler divergence of $D\_{KL}(P || Q) = O(\frac1d)$ and thus a total variation distance of $O(\frac{1}{\sqrt d})$.


The case where $\sigma^2=1$ might be easier and sufficient for what I need. Any suggestions on how I could bound the TV distance between P and Q?



Asymptotic bound on the total variation distance between a standard multivariate normal and a simple mixture
$\newcommand{\m}{\vec{\mu}} 
\newcommand{\e}{\vec{e}}
\newcommand{\x}{\vec{x}}$
 Let $n:=d$. Assume that $\sigma=1$ and $\mu\ne0$. Then for the densities 
\begin{equation\*}
 f\_Q(\x)=(2\pi)^{-n/2}\,\frac1n\,\sum\_1^n e^{-|\x-\mu\e\_i|^2/2}
\end{equation\*}
and 
\begin{equation\*}
 f\_P(\x)=(2\pi)^{-n/2}\,e^{-|\x|^2/2}
\end{equation\*}
of $P$ and $Q$ we have 
\begin{equation\*}
 f\_Q(\x)<f\_P(\x)\iff \sum\_1^n e^{\mu x\_i}<ne^{\mu^2/2}, 
\end{equation\*}
where $\x=(x\_1,\dots,x\_n)\in\mathbb R^n$ and $|\cdot|$ denotes the Euclidean norm. Hence, 
\begin{equation\*}
 d\_{TV}(P,Q)=p\_1-p\_2, \tag{1}
\end{equation\*}
where 
\begin{equation\*}
 p\_1:=P(S\_n<ne^{\mu^2/2}),\quad p\_2:=P(T\_n<ne^{\mu^2/2}), 
\end{equation\*}
\begin{equation\*}
 S\_n:=\sum\_1^n U\_i,\quad T\_n:=S\_{n-1}+V\_n, 
\end{equation\*}
\begin{equation\*}
 U\_i:=e^{\mu Z\_i},\quad V\_n:=e^{\mu(Z\_n+\mu)}=e^{\mu^2}U\_n, 
\end{equation\*}
and the $Z\_i$'s are iid standard normal random variables. Note that 
\begin{equation\*}
 ES\_n=na,\quad ET\_n=na+O(1),\quad Var\, S\_n=nb^2,\quad Var\,T\_n=nb^2+O(1), 
\end{equation\*}
where $a:=e^{\mu^2/2}$ and $b^2:=e^{2\mu^2}-e^{\mu^2}$. Also, $E|U\_1|^3<\infty$. So, letting $\Phi$ denote the standard normal cumulative distribution function and using the Berry--Esseen inequality, we have 
\begin{equation\*}
 p\_1=P(S\_n<na)=\Phi(0)+O(1/\sqrt n)=\frac12+O(1/\sqrt n)
\end{equation\*}
and 
\begin{multline\*}
 p\_2=P(T\_n<na)=\Phi\Big(\frac{na-ET\_n}{\sqrt{Var\,T\_n}}\Big)+O(1/\sqrt n) \\ 
 =\Phi\Big(\frac{O(1)}{\sqrt{nb^2+O(1)}}\Big)+O(1/\sqrt n)
 =\frac12+O(1/\sqrt n)
\end{multline\*}
Thus, by (1), we have 
\begin{equation\*}
 d\_{TV}(P,Q)=O(1/\sqrt n), 
\end{equation\*}
as conjectured. 


