Let $\{\mathbb{P}\_{\theta}\}\_{\theta}$ be an exponential family of probability measures, all with finite mean. Under what conditions is the parameterization map $\theta\mapsto \mathbb{P}\_{\theta}$ continuous with respect to the Wasserstein-$1$ distance?



Parameterization of exponential family
$\newcommand\om\omega\newcommand\Om\Omega\newcommand\ep\epsilon\newcommand\R{\mathbb R}$Let $(P\_t)\_{t\in T}$ be an exponential family over a separable complete metric space $(X,d)$, where $T$ is an open subset of $\mathbb R^k$ and $t=(t\_1,\dots,t\_k)$ is a [natural parameter](https://en.wikipedia.org/wiki/Exponential_family#Vector_parameter). So, for all $t\in T$, the density $p\_t$ of $P\_t$ with respect to Borel measure $\mu$ on $X$ is given by the formula
\begin{equation\*}
 p\_t(x)=c(t)\exp\{t\cdot U(x)\}\tag{1}
\end{equation\*}
for all $x\in X$, where
$U\colon X\to\mathbb R^k$ is a Borel-measurable function, $\cdot$ denotes the dot product on $\mathbb R^k$,
\begin{equation\*}
 c(t):=1/I(t),\quad I(t):=\int\_X\mu(dx)\exp\{t\cdot U(x)\}\in(0,\infty). \tag{2} 
\end{equation\*}


Assume that, for some (and hence for all) $x\_0\in X$, the "first moment"
\begin{equation}
 m\_t(x\_0):=\int\_X P\_t(dx)d(x\_0,x)=\int\_X \mu(dx)p\_t(x)d(x\_0,x)<\infty\tag{2a}
\end{equation}
for all $t\in T$.


Then the parameterization map $U\ni t\mapsto P\_t$ is continuous with respect to the Wasserstein-1 distance $W\_1$.


Indeed, by the [penultimate paragraph of the introduction section](https://w3.ens-rennes.fr/math/people/francois.bolley/wasserstein.pdf), if suffices to show that for each $t\in T$
\begin{equation\*}
 \int\_X \mu(dx)p\_s(x)f(x)\to \int\_X \mu(dx)p\_t(x)f(x)\tag{3}
\end{equation\*}
as $s\to t$, where $f\colon X\to\mathbb R$ such that
\begin{equation\*}
C:=\sup\_{x\in X}|f(x)|/(1+d(x\_0,x))<\infty \tag{3a}
\end{equation\*}
(for some (and hence for all) $x\_0\in X$).


To verify (3), take any $t=(t\_1,\dots,t\_k)$ in the open set $T$. Take any real $\ep>0$ such that $t+\ep\om\in T$ for all $\om=(\om\_1,\dots,\om\_k)\in\Om:=\{-1,1\}^k$.
Take then any $s\in T$ such that $\max\_{j=1}^k|s\_j- t\_j|\le\ep$ and any $u\in\R^k$. Then
\begin{align\*}\exp\{s\cdot u\}/\exp\{t\cdot u\}&=\exp\{(s-t)\cdot u\} \\ 
&\le\max\_{\om\in\Om}\exp\{\ep\om\cdot u\} \\ 
&\le\sum\_{\om\in\Om}\exp\{\ep\om\cdot u\};
\end{align\*}
the first inequality in the above display follows because the function $\exp$ is convex and the condition $\max\_{j=1}^k|s\_j- t\_j|\le\ep$ implies that the point $s-t$ is in the convex hull of the set $\{\ep\om\colon\om\in\Om\}$. So, for all $x\in X$
\begin{equation\*}
 \exp\{s\cdot U(x)\}\le g(x):=\sum\_{\om\in\Om}\exp\{( t+\ep\om)\cdot U(x)\}. \tag{5}
\end{equation\*}
Moreover, by (2),
\begin{equation\*}
 \int\_X\mu(dx)g(x)=\sum\_{\om\in\Om}I(t+\ep\om)<\infty. 
\end{equation\*}
So, by the dominated convergence theorem, $I(s)\to I(t)$ and hence $c(s)\to c(t)$ as $s\to t$. So, choosing $\ep$ small enough, we have
\begin{equation\*}
 c(s)\le 2c(t). \tag{6}
\end{equation\*}


In view of (1) and (6), inequality (5) yields
\begin{equation\*}
 p\_s(x)\le c(s)\sum\_{\om\in\Om}\frac{p\_{t+\ep\om}(x)}{c(t+\ep\om)}
 \le K\sum\_{\om\in\Om}p\_{t+\ep\om}(x),
\end{equation\*}
where
\begin{equation\*}
 K:=2c(t)\max\_{\om\in\Om}\frac1{c(t+\ep\om)}<\infty. 
\end{equation\*}
So, by (3a),
\begin{equation\*}
 p\_s(x)|f(x)|\le h(x):=KC\sum\_{\om\in\Om}p\_{t+\ep\om}(x)(1+d(x\_0,x)). 
\end{equation\*}
Moreover, by (2a),
\begin{equation\*}
 \int\_X \mu(dx)h(x)=KC\sum\_{\om\in\Om}(1+m\_{t+\ep\om}(x\_0))<\infty. 
\end{equation\*}
So, again by the the dominated convergence theorem, (3) follows. $\Box$


