Let $\mu, \mu\_1, \mu\_2, \dots$ be random measures on
a Polish space (separable completely metrizable topological space) $(S, {\mathcal S})$.
Suppose I know that 


$$\int f d \mu\_n \to \int f d\mu$$


*in probability* for each bounded continuous real-valued function. This would be the definition of weak convergence $\mu\_n \Rightarrow \mu$, if I dropped "in probability" and the measures $\mu\_n$ were deterministic. 


Is there any standard way to extract a weakly convergent subsequence from $(\mu\_n)$ consisting of almost all members of $(\mu\_n)$? Possibly with additional assumptions? Where can I learn about such things?


Sorry if this is a trivial question.



convergence of integral for each bounded function in probability
It is interesting if you let the random index set depend on the realizations. For simplicity, restrict attention to random sequences $\{X\_1, X\_2, X\_3, \ldots\}$ that converge to 0 in probability, but not with probability 1. We say that a set $B$ contains almost all positive integers if: 
$$ \lim\_{n\rightarrow\infty} \frac{1}{n}| \{1, \ldots, n\} \cap B| = 1 $$


Claim 1: If $\{X\_i\}\_{i=1}^{\infty}$ are mutually independent, then with probability 1 there exists a random set $B$ (possibly dependent on $\{X\_i\}$) that contains almost all positive integers and such that $X\_i$ converges to $0$ over $i \in B$. 


Claim 2: There exist examples where $\{X\_i\}\_{i=1}^{\infty}$ are mutually independent, but for which there exists no *deterministic* set $B$ that contains almost all positive integers and is such that $X\_i$ converges to 0 with probability 1 over $i \in B$. 


Claim 3: There are examples where no (possibly random) set $B$ with the desired properties exists. (In view of Claim 1, all such examples must have dependencies between the $X\_i$ variables.)




---


Proof of Claim 1: Suppose $\{X\_i\}\_{i=1}^{\infty}$ are mutually independent and converge to 0 in probability. Then for all $\epsilon>0$ we have $Pr[|X\_i|>\epsilon]\rightarrow 0$. It follows that there is a deterministic sequence of positive numbers $\{\epsilon\_1, \epsilon\_2, \epsilon\_3, \ldots\}$ such that the following two things hold: 
\begin{align} 
&\lim\_{i\rightarrow\infty} \epsilon\_i = 0 \\
&\lim\_{i\rightarrow\infty} Pr[|X\_i| > \epsilon\_i] = 0
\end{align} 
(As a quick explanation of why: Choose $\epsilon\_1=1$. Find an index $n\_2>1$ such that $Pr[|X\_i|>1/2] \leq 1/2$ for all $i \geq n\_2$ and define $\epsilon\_i=\epsilon\_1$ for all $i \in \{1, \ldots, n\_2-1\}$, and define $\epsilon\_{n\_2}=1/2$. Then find an index $n\_3>n\_2$ such that $Pr[|X\_i|>1/3] \leq 1/3$ for all $i \geq n\_3$, and define $\epsilon\_i = \epsilon\_2$ for all $i \in \{n\_2, \ldots, n\_3-1\}$, and define $\epsilon\_{n\_3} = 1/3$, and so on.)


Now define the random set $B$ as follows: Include a positive integer $i$ in the set $B$ if and only if $\{|X\_i|\leq \epsilon\_i\}$. If the set $B$ has an infinite number of positive integers, then clearly $X\_i$ converges to $0$ over $i \in B$. It remains to show that $B$ has almost all positive integers. 


Define $I\_i$ as an indicator function that is $1$ if $\{|X\_i|>\epsilon\_i\}$, and $0$ else. Define $N(k) = \sum\_{i=1}^k I\_i$ as the number of integers in $\{1, \ldots, k\}$ that are *not* in the set $B$. Notice that $|I\_i-E[I\_i]|\leq 1$ for all $i$, so:
\begin{align}
&E[(I\_i-E[I\_i])^2] \leq 1\\
&E[(I\_i-E[I\_i])^4]\leq 1
\end{align} 


Define $S(k) = \sum\_{i=1}^k E[I\_i]$. Then for all $\delta>0$:  

\begin{align} 
Pr\left[\left|\frac{N(k)-S(k)}{k}\right| \geq \delta\right] &=Pr\left[ |N(k)-S(k)| \geq \delta k \right]\\
&\leq Pr[(N(k)-S(k))^4 \geq \delta^4 k^4] \\
&= Pr\left[ \left(\sum\_{i=1}^k(I\_i-E[I\_i])\right)^4 \geq \delta^4 k^4 \right]\\
&\leq \frac{E\left[\left( \sum\_{i=1}^k(I\_i-E[I\_i]) \right)^4 \right]}{\delta^4 k^4}
\end{align} 
where the final inequality holds by the Markov inequality. Because $\{I\_i-E[I\_i]\}\_{i=1}^{\infty}$ are mutually independent and zero mean with second and fourth moments bounded by 1, it holds that there is a number $D>0$ such that: 
$$ E\left[\left( \sum\_{i=1}^k(I\_i-E[I\_i]) \right)^4\right] \leq Dk + Dk^2 $$
Hence:
$$ Pr\left[\left|\frac{N(k)-S(k)}{k}\right| \geq \delta\right] \leq \frac{Dk + Dk^2}{\delta^4 k^4} $$ 
The right-hand-side is summable, and so with probability 1: 
$$ \lim\_{k\rightarrow\infty} \frac{N(k)-S(k)}{k} = 0 $$
However, $\frac{S(k)}{k} = \frac{\sum\_{i=1}^k Pr[|X\_i|>\epsilon\_i]}{k} \rightarrow 0$, since it is the average of terms that converge to 0. It follows that with probability 1: 
$$ \lim\_{k\rightarrow\infty} \frac{N(k)}{k} = 0 $$
and so (with prob 1) the random set $B$ contains almost all positive integers. $\Box$




---


Example for Claim 2: Define $\{X\_1, X\_2, X\_3, \ldots\}$ mutually independent with: 
$$ X\_i =\left\{ \begin{array}{ll}
1 &\mbox{ with probability $1/i$} \\
0 & \mbox{ otherwise} 
\end{array}\right. $$
This example is well known to converge to 0 in probability, but not with probability 1. Suppose there is a deterministic set $B$ that contains almost all positive integers and for which $X\_i$ converges to $0$ over $i \in B$ (we reach a contradiction). 


For each positive integer $i$, define $g(i)$ as the number of elements in $\{i, i+1, \ldots, 2i\}$ are are *not* in $B$. Since $B$ has almost all positive integers, it is not difficult to show that: 
$$ \lim\_{i\rightarrow\infty} \frac{g(i)}{i} = 0 $$
Now for each positive integer $i$, define $\theta\_i$ as the probability that there is at least one index $j \in \{i, i+1, \ldots, 2i\} \cap B$ for which $X\_j=1$. Then: 
\begin{align} 
\theta\_i &= 1 - \prod\_{j \in \{i, \ldots, 2i\} \cap B}\left(\frac{i-1}{i}\right)\\
&\geq 1 - \frac{\prod\_{j=i}^{2i}\left(\frac{i-1}{i}\right)}{(1-1/i)^{g(i)}}\\
&= 1 - \frac{\left(\frac{i-1}{2i}\right)}{(1-1/i)^{g(i)}}
\end{align} 
However, since $g(i)/i\rightarrow 0$ we have: 
$$ (1-1/i)^{g(i)} = \left((1-1/i)^{i}\right)^{g(i)/i} \approx (1/e)^{g(i)/i}\rightarrow 1 $$
and so: 
$$ \liminf\_{i\rightarrow\infty} \theta\_i \geq 1/2 $$
It follows that $\theta\_i \geq 1/4$ for all sufficiently large positive integers $i$. Hence, all sufficiently large positive integers $i$ have the property that, with probability at least 1/4, there is an index $j>i$ such that $j\in B$ and $X\_j=1$. So $X\_i$ cannot converge to 0 with probability 1 over $i \in B$. $\Box$




---


Example for Claim 3: Consider the same $\{X\_1, X\_2, X\_3, \ldots\}$ sequence from Claim 2, but now form a new (dependent) sequence $\{Y\_1, Y\_2, Y\_3, \ldots\}$ by: 
$$ \{X\_1, X\_1, \: \: X\_2, X\_2, X\_2, X\_2, \: \: X\_3, X\_3, X\_3, X\_3, X\_3, X\_3, X\_3, X\_3, \ldots\} $$
Specifically, the $Y\_i$ elements are filled in over *frames*, where each frame $k \in \{1, 2, 3, \ldots\}$ has size $2^k$ and consists of the same value $X\_k$. It is clear that $Y\_i$ converges to $0$ in probability (since $X\_i$ converges to $0$ in probability). 


Now take any (potentially random) set $B$ that contains almost all positive integers (the set $B$ is allowed to depend on the $\{X\_i\}$ realizations). For all sufficiently large positive integers $k$, this set $B$ must contain at least half of the indices in frame $k$. But, with probability 1, $X\_k=1$ for an infinite number of integers $k$. It follows that, with probability 1, $Y\_i=1$ for an infinite number of elements $i \in B$. So, with probability 1, $Y\_i$ does not converge to 0 over $i \in B$. 


