Bernstein’s Inequality can be stated as follows : Let $x\_1, x\_2, \dots, x\_n$ be independent bounded random variables such that $\mathbb{E}[x\_i] = 0$ and $|x\_i| \leq \zeta$ with probability $1$ and let $\sigma^2 = \tfrac{1}{n}\sum\_{1}^{n} Var\{x\_i\}$. Then for any $\epsilon > 0$, we have
$$
\mathbb{P} \left[ \frac{1}{n} \sum\_{i=1}^{n} x\_i \geq \epsilon \right] \leq \exp{\left\{ \frac{-n \epsilon^2}{ 2 \sigma^2 + 2\zeta \epsilon/3} \right\}}
$$


If instead of $|x\_i| \leq \zeta$ with probability $1$, it is the case that $ \sum\_{i=1}^{n} \mathbb{P}\left\{ |x\_i| > \zeta \right\} \leq \delta$, then is the following applicable?


$$
\mathbb{P} \left[ \frac{1}{n} \sum\_{i=1}^{n} x\_i \geq \epsilon \right] \leq \exp{\left\{ \frac{-n \epsilon^2}{ 2 \sigma^2 + 2\zeta \epsilon/3} \right\}} + \delta \qquad \qquad \qquad \qquad (1)
$$


I think, the above extension is similar to the extension of the Azuma-Hoeffding inequality proved in Theorem 32 of [Chung and Lu(2006)](https://projecteuclid.org/euclid.im/1175266369). The question concerning the extension of the Azuma-Hoeffding inequality was also asked [here](https://mathoverflow.net/questions/171479/extension-of-the-azuma-hoeffding-inequality-when-the-differences-are-bounded-wi).


Is it possible to extend Bernstein’s Inequality as eq. $(1)$ following the lines of the proof of Theorem 32 in [Chung and Lu(2006)](https://projecteuclid.org/euclid.im/1175266369)?


Addendum : The Freedman inequality for martingales is a result like the Bernstein inequality but where the random variables can be dependent. Is an extension like the above possible for Freedman inequality?


For reference Freedman inequality (Theorem 1.6 in [Freedman (1975)](https://projecteuclid.org/euclid.aop/1176996452)] ) can be stated as follows: let $x\_1, x\_2, \dots $ be a martingale difference sequence and $|x\_i| \leq \zeta$ for all $i$. Then
$$
\mathbb{P} \left[ \sum\_{i=1}^{n} x\_i \geq \epsilon, \sum\_{i=1}^{n} var(x\_i | \mathcal{F}\_{i-1}) \leq \eta \right] \leq \exp{\left\{ \frac{-\epsilon^2}{2\eta + 2 \zeta \epsilon/3 } \right\}}.
$$


If instead of $|x\_i| \leq \zeta$ with probability $1$, it is the case that $ \sum\_{i=1}^{n} \mathbb{P}\left\{ |x\_i| > \zeta \right\} \leq \delta$, then is the following applicable?
$$
\mathbb{P} \left[ \sum\_{i=1}^{n} x\_i \geq \epsilon, \sum\_{i=1}^{n} var(x\_i | \mathcal{F}\_{i-1}) \leq \eta \right] \leq \exp{\left\{ \frac{-\epsilon^2}{2\eta + 2 \zeta \epsilon/3 } \right\}} + \delta \qquad \qquad (2)
$$
Thank you.



Extension of Bernstein’s Inequality when the random variable is bounded with large probability
$\newcommand{\de}{\delta}$Your inequality (2) does hold. Actually, a better and more general bound holds. First here, let us standardize and simplify notations. Let us use $X\_i$ instead of $x\_i$, $x$ instead of $\epsilon$, $y>0$ instead of $\zeta$, $B^2>0$ instead of $\eta$, $Var\_{i-1}\,\cdot$ instead of $var(\cdot|\mathcal{F}\_{i-1})$, and $E\_{i-1}\,\cdot$ instead of $E(\cdot|\mathcal{F}\_{i-1})$.


Instead of the conditions that the $x\_i$'s are martingale differences and $\sum\_{i=1}^n P(|x\_i|>\zeta)\le\de$, let us use the more general conditions that the $X\_i$'s are supermartingale differences and
\begin{equation}
 P(\max\_{i=1}^n X\_i>y)\le\de.\tag{1}
\end{equation}


Let also $Y\_i:=X\_i\,1(X\_i\le y)$ and $Z\_i:=Y\_i\,1(V\_i\le B^2)=X\_i\,1(X\_i\le y,V\_i\le B^2)$, where
$$V\_i:=\sum\_{j=1}^i E\_{j-1}\,Y\_j^2.$$
Note that $V\_i$ is no greater than $\sum\_{j=1}^i E\_{j-1}\,X\_j^2$, which latter coincides with $\sum\_{j=1}^i Var\_{j-1}\,X\_j$ in the special case when the $X\_i$'s are martingale differences.


By (1),
$$P\Big(\sum\_{i=1}^n X\_i\ge x,V\_n\le B^2\Big)\le P\Big(\sum\_{i=1}^n Y\_i\ge x,V\_n\le B^2\Big)+\de.$$
Obviously, $V\_i\le V\_n$ for $i\le n$. So,
$$P\Big(\sum\_{i=1}^n Y\_i\ge x,V\_n\le B^2\Big)\le P\Big(\sum\_{i=1}^n Z\_i\ge x\Big).$$


Next, $E\_{i-1}Z\_i=1(V\_i\le B^2)E\_{i-1}Y\_i\le 1(V\_i\le B^2)E\_{i-1}X\_i\le0$, so that the $Z\_i$'s are supermartingale differences. Also, $E\_{i-1}Z\_i^2\le E\_{i-1}X\_i^2$. So, by [Theorem 8.2 on page 1702](https://projecteuclid.org/download/pdf_1/euclid.aop/1176988477), we have the Hoeffding-type inequality
\begin{equation\*}
 P\Big(\sum\_{i=1}^n Z\_i\ge x\Big)\le\exp\Big\{\frac{B^2}{y^2}\psi\Big(\frac{xy}{B^2}\Big)\Big\},
\end{equation\*}
where $\psi(u):=u-(1+u)\ln(1+u)$. Collecting the pieces, we get
\begin{equation\*}
 P\Big(\sum\_{i=1}^n X\_i\ge x,V\_n\le B^2\Big)\le\exp\Big\{\frac{B^2}{y^2}\psi\Big(\frac{xy}{B^2}\Big)\Big\}+\de. \tag{2}
\end{equation\*}
The latter bound is better than the Bernstein-type bound
\begin{equation\*}
 \exp\Big\{-\frac{x^2}{2B^2+2xy/3}\Big\}+\de, \tag{3}
\end{equation\*}
because, as shown in [Theorem 3](https://epubs.siam.org/doi/10.1137/1134032), the Hoeffding-type bound is the best exponential bound in its terms. Another, direct way to see that the bound in (2) is better than (3) is to use the inequality $\psi(u)\le-u^2/(2+2u/3)$ for real $u\ge0$.


Thus, the bound in (2) is better and more general than what you wanted.


