Let $X,Y$ be locally-compact Polish spaces, equip the set $\mathcal{P}(Y)$ of probability measures on $Y$ with the weak$^{\star}$ topology (topology of convergence in distribution), and equip $C(X,\mathcal{P}(Y))$ with the compact-open topology. Let $\operatorname{co}(\{\delta\_y\}\_{y \in Y})$ denote the set of finite convex combinations of point-masses on $Y$. 


(When) is the subset $C(X,co(\{\delta\_y\}\_{y \in Y}))$ dense in $C(X,\mathcal{P}(Y))$?



Density of $C(X,\operatorname{co}\{\delta_y\}_{y \in Y})$ in $C(X,\mathcal{P}(Y))$
$\newcommand{\ep}{\varepsilon}
\newcommand{\de}{\delta}
\newcommand{\B}{\mathcal B}
\newcommand{\K}{\mathcal K}
\newcommand{\NN}{\mathcal N}
\newcommand{\PP}{\mathcal P}
\newcommand{\supp}{\operatorname{supp}}$


The answer to your question is yes. 


Indeed, recall that the weak$^{\star}$ topology on $\PP(Y)$ is metrized by the [Lévy--Prokhorov metric](https://en.wikipedia.org/wiki/L%C3%A9vy%E2%80%93Prokhorov_metric) $L$ defined by the condition: for any real $h>0$ and any $P,Q$ in $\PP(Y)$, 
\begin{equation}
 L(P,Q)\le h\iff\forall B\in\B(Y)\ P(B)\le Q(B\_h)+h,
\end{equation}
where $\B(Y)$ is the Borel $\sigma$-algebra over $Y$ and $B\_h$ is the $h$-neighborhood of $B$. 


Let $\supp P$ denote the support of $P$. Let $\PP(K):=\{Q\in\PP(Y)\colon\supp Q\subseteq K\}$. 


**Lemma 1:** Take any compact $\K\subseteq\PP(Y)$ and any real $\ep>0$. Then there is a compact $K\subseteq Y$ such that for each $P\in\K$ there is some $Q\_P\in\PP(K)$ such that 
\begin{equation}
 \sup\_{P\in\K}L(P,Q\_P)\le\ep. 
\end{equation}


*Proof:* By [Prokhorov's theorem](https://en.wikipedia.org/wiki/Prokhorov%27s_theorem), for each real $\de\in(0,1)$ there is a compact $K\subseteq Y$ such that for each $P\in\K$ we have $P(K)\ge1-\de$. For all $B\in\B(Y)$, let then 
\begin{equation}
 Q\_P(B):=\frac{P(B\cap K)}{P(K)}. 
\end{equation}
Then for all $P\in\K$ and all $B\in\B(Y)$ we have
\begin{equation}
 Q\_P(B)\le\frac{P(B)}{1-\de}\le P(B)+\frac\de{1-\de}=P(B)+\ep 
\end{equation}
if $\de=\ep/(1+\ep)$. So, indeed $L(P,Q\_P)\le\ep$ for all $P\in\K$. $\Box$


Let $\PP\_{fin}(Y)$ denote the set of all $Q\in\PP(Y)$ with a finite support. 


**Lemma 2:** Take any compact $\K\subseteq\PP(Y)$ and any real $\ep>0$. Then for each $P\in\K$ there is some $R\_P\in\PP\_{fin}(Y)$ such that 
\begin{equation}
 \sup\_{P\in\K}L(P,R\_P)\le2\ep. 
\end{equation}


*Proof:* Let $K\subseteq Y$ and $Q\_P\in\PP(K)$ for $P\in\K$ be as in Lemma 1. Since $K$ is compact, there is a finite Borel-measurable partition $(C\_{\ep;j})$ of $K$ such that for each $j$ there is a point $y\_j\in Y$ such that $C\_{\ep;j}$ is contained in the ball $B\_\ep(y\_j)$ of radius $\ep$ centered at $y\_j$. 


For any $P\in\PP(Y)$ and any $B\in\B(Y)$, let 
\begin{equation}
 Q\_P^\ep(B):=\sum\_j Q\_P(C\_{\ep;j})1\_{y\_j\in B}=Q\_P(C\_{\ep;B}),
\end{equation}
where 
\begin{equation}
 C\_{\ep;B}:=\bigcup\_{j\colon y\_j\in B}C\_{\ep;j}.
\end{equation}
Then $Q\_P^\ep\in\PP\_{fin}(Y)$. Moreover, 
$$L(Q\_P^\ep,Q\_P)\le\ep$$ 
for all $P\in\PP(Y)$, because $C\_{\ep;B}\subseteq B\_\ep$ for all $B\in\B(Y)$. Letting now $R\_P:=Q\_P^\ep$, we see that Lemma 2 follows by Lemma 1. $\Box$


Finally, take any function $P\_\cdot\in C(X,\PP(Y))$, which is a continuous map $X\ni x\mapsto P\_x\in\PP(Y)$. Take also any compact $K\_X\subseteq X$. Then the image $\K:=\K\_{K\_X}:=P\_\cdot(K\_X)$ of the compact $K\_X$ under the continuous map $P\_\cdot$ is compact. So, by Lemma 2, for each real $\ep>0$ and each $x\in K\_X$ there is some $R\_{\ep;x}\in\PP\_{fin}(Y)$ such that 
\begin{equation}
 \sup\_{x\in K\_X}L(P\_x,R\_{\ep;x})\le2\ep. 
\end{equation}
This means that indeed $C(X,\PP\_{fin}(Y))$ is dense in $C(X,\PP(Y))$, in the required sense. 


