Let $(a\_k)\_{k \geq 1}$ be random variables taking values on a finite subset $B$. Assume that
$$
(1) \quad \Pr\Big (\lim\_{n\rightarrow +\infty}d(\frac{1}{n}\sum\_{k=1}^n 1\_{[a\_k = b]}, [v\_\ell(b,\theta\_0),v\_u(b,\theta\_0)])= 0\Big)=1 \quad \forall b\in B
$$
where $d$ stays for distance (i.e., absolute value difference); $v\_\ell$ and $v\_u$ are real-valued functions of $b\in B$ and $\theta \in \mathbb{R}$; $\theta\_0\in \mathbb{R}$ is a *specific* value of $\theta$.


Define
$$
\Theta\_n\equiv \Big\{\theta \in \mathbb{R}: \frac{1}{n}\sum\_{k=1}^n 1\_{[a\_k = b]}\in [v\_\ell(b,\theta),v\_u(b,\theta)] \forall b \in B\Big\}
$$


**Question**: Is it possible to find an upper for $\Pr(\theta\_0\in \Theta\_n)$?


**My attempt so far:**
We have that $E\Big(\frac{1}{n}\sum\_{k=1}^n 1\_{[a\_k = b]}\Big)\leq v\_u(b,\theta\_0)$ (see [here](https://mathoverflow.net/questions/445629/bound-the-expectation-of-an-average#445629)). By Markov inequality
$$
\Pr \Big(\frac{1}{n}\sum\_{k=1}^n 1\_{[a\_k = b]} \geq r\Big)\leq \frac{v\_u(b,\theta\_0)}{r}
$$
for each $r>0$. From here?




---


As advised in the comment below, the problem as posed only provides trivial bounds. Therefore, in what follows, I add some additional structure. In particular, I show that
$$
(2) \quad \Pr(a\_k=b | a\_{1},\dots, a\_{k-1}) \in \big[\nu\_{\ell}(b, \theta\_0), \nu\_u(b,\theta\_0)\big] \quad \text{ for all } k, b
$$
is a sufficient condition for (1), in the hope that this can suggest additional relevant details.


Proof of (1):
From (2), we have that
$$
\frac{1}{n}\sum\_{k=1}^n \Pr(a\_k=b | a\_{1},\dots, a\_{k-1})\in \big[\nu\_{\ell}(b, \theta\_0), \nu\_u(b,\theta\_0)\big] \text{ for all } n 
$$


Let
$$
S\_n(b)\equiv\sum\_{k=1}^n \frac{1}{k} \left({1}\_{[a\_k=b]}- \Pr(a\_k=b| a\_1, \dots, a\_{k-1})\right).
$$
Observe that the sequence $(S\_n(b))\_{n\geq 1}$ is an $L^2$-bounded
martingale. Therefore, $S\_n(b)$ converges almost surely as $n\to +\infty$. Further, $\frac{1}{n}\sum\_{k=1}^{n-1} S\_k(b)$ converges to the same limit as $S\_n(b)$.


Let
$$
Q\_n(b)\equiv \sum\_{k=1}^n \left(1\_{[a\_k=b]}- \Pr(a\_k=b| a\_1, \dots, a\_{k-1}\right).
$$


Observe that
$$
Q\_n(b) = n S\_n(b)-\sum\_{k=1}^{n-1} S\_{k}(b).
$$


By the convergence of $S\_n(b)$ and $\frac{1}{n}\sum\_{k=1}^{n-1} S\_k(b)$ to the same limit, $\frac{1}{n}{Q\_n(b)}$ converges almost surely to 0 as $n\to +\infty$.


Hence,
$$
\frac{1}{n}\sum\_{k=1}^n 1\_{[a\_k=b]}-\frac{1}{n}\sum\_{k=1}^n \Pr(a\_k=b| a\_1, \dots, a\_{k-1})\overset{a.s.}{\to} 0 \text{ as $n\to +\infty$.}
$$
Therefore, $\frac{1}{n}\sum\_{k=1}^n 1\_{[a\_k=b]}$ and $\frac{1}{n}\sum\_{k=1}^n \Pr(a\_k=b| a\_1, \dots, a\_{k-1})$ have the same limit points which belong to $ \big[\nu\_{\ell}(b, \theta\_0), \nu\_u(b,\theta\_0)\big] $ and (1) holds.



Bound the probability that a point belongs to a set
As I wrote before in a comment, the only upper bound on $\Pr(\theta\_0\in\Theta\_n)$ under these very general conditions is the trivial bound $1$.


Indeed, suppose that for some $b\_\*\in B$
\begin{equation}
 \Pr(a\_1=a\_2=\cdots=b\_\*)=1
\end{equation}
and for all $b\in B$
\begin{equation}
 v\_\ell(b,\theta\_0)=1(b=b\_\*)=v\_u(b,\theta\_0). 
\end{equation}
Then $\Pr(a\_k=b)=v\_\ell(b,\theta\_0)=v\_u(b,\theta\_0)$ for all natural $k$ and all $b\in B$. So, condition (1) holds and $\Pr(\theta\_0\in\Theta\_n)=1$ for all natural $n$.


