arXiv:1702.05933v2 [] 22 Jan 2018

Qualitative robustness for bootstrap approximations
Katharina Strohriegl,University of Bayreuth
katharina.strohriegl@uni-bayreuth.de
January 23, 2018

Abstract
An important property of statistical estimators is qualitative robustness, that is
small changes in the distribution of the data only result in small chances of the distribution of the estimator. Moreover, in practice, the distribution of the data is commonly
unknown, therefore bootstrap approximations can be used to approximate the distribution of the estimator. Hence qualitative robustness of the statistical estimator under the
bootstrap approximation is a desirable property. Currently most theoretical investigations on qualitative robustness assume independent and identically distributed pairs of
random variables. However, in practice this assumption is not fulfilled. Therefore, we
examine the qualitative robustness of bootstrap approximations for non-i.i.d. random
variables, for example α-mixing and weakly dependent processes. In the i.i.d. case qualitative robustness is ensured via the continuity of the statistical operator, representing
the estimator, see Hampel (1971) and Cuevas and Romo (1993). We show, that qualitative robustness of the bootstrap approximation is still ensured under the assumption
that the statistical operator is continuous and under an additional assumption on the
stochastic process. In particular, we require a convergence condition of the empirical
measure of the underlying process, the so called Varadarajan property.

Keywords: stochastic processes, qualitative robustness, bootstrap, α-mixing, weakly dependent AMS: 60G20, 62G08, 62G09, 62G35

1

Introduction

The overwhelming part of theoretical publications in statistical machine learning was done
under the assumption that the data is generated by independent and identically distributed
(i.i.d.) random variables. However, this assumption is not fulﬁlled in many practical applications so that non-i.i.d. cases increasingly attract attention in machine learning. An
important property of an estimator is robustness. It is well known that many classical estimators are not robust, which means that small changes in the distribution of the data generating process may highly aﬀect the results, see for example Huber (1981), Hampel (1968),
1

Jurečková and Picek (2006) or Maronna et al. (2006) for some books on robust statistics.
Qualitative robustness is a continuity property of the estimator and means roughly speaking:
small changes in the distribution of the data only lead to small changes in the distribution
(i.e. the performance) of the estimator. In this way the following kinds of "small errors"
are covered: small errors in all data points (rounding errors) and large errors in only a
small fraction of the data points (gross errors, outliers). Qualitative robustness of estimators has been deﬁned originally in Hampel (1968) and Hampel (1971) in the i.i.d. case and
has been generalized to estimators for stochastic processes in various ways, for example, in
Papantoni-Kazakos and Gray (1979), Bustos (1980), which will be the one used here, Cox
(1981), Boente et al. (1987), Zähle (2015), and Zähle (2016), for a more local consideration
of qualitative robustness, see for example Krätschmer et al. (2017).
Often the ﬁnite sample distribution of the estimator or of the stochastic process of interest is
unknown, hence an approximation of the distribution is needed. Commonly, the bootstrap
is used to receive an approximation of the unknown ﬁnite sample distribution by resampling
from the given sample.
The classical bootstrap, also called the empirical bootstrap, has been introduced by Efron
(1979) for i.i.d. random variables. This concept is based on drawing a bootstrap sample
∗ ) of size m ∈ N with replacement out of the original sample (Z , . . . , Z ), n ∈
(Z1∗ , . . . , Zm
1
n
N, and approximate the theoretical distribution Pn of (Z1 , . . . , Zn ) using the bootstrap
sample. For the empirical bootstrap the approximation of the distribution via the bootstrap
∗ ), hence P ∗ =
is given by
the empirical
distribution of the bootstrap sample (Z1∗ , . . . , Zm
n

P
m
1
∗
δZi denotes the dirac measure. The bootstrap sample itself has
⊗ni=1 m
i=1 δZi , where

1 Pn
distribution ⊗m
i=1 n
i=1 δZi .

For an introduction to the bootstrap see for example Efron and Tibshirani (1993) and
van der Vaart (1998, Chapter 3.6). Besides the empirical bootstrap many other bootstrap methods have been developed in order to ﬁnd good approximations also for noni.i.d. observations, see for example Singh (1981), Lahiri (2003), and the references therein.
In Section 2.2 the moving block bootstrap introduced by Künsch (1989) and Liu and Singh
(1992) is used to approximate the distribution of an α-mixing stochastic process.
It is, also in the non-i.i.d. case, still desirable that the estimator is qualitatively robust
even for the bootstrap approximation. That is, the distribution of the estimator under the
bootstrap approximation LPn∗ (Sn ), n ∈ N, of the assumed, ideal distribution Pn should still
be close to the distribution of the estimator under the bootstrap approximation LQ∗n (Sn ),
n ∈ N, of the real contaminated distribution Qn . Remember that this is a random object as
Pn∗ respectively Q∗n are random. For notational convenience all bootstrap values are noted
as usual with an asterisk.
To show qualitative robustness often generalizations of Hampel’s theorem are used, as it
is often hard to show qualitative robustness directly. For the i.i.d. case Hampel’s Theorem
ensures qualitative robustness of a sequence of estimators, if these estimators are continuous
and can be represented by a statistical operator which is continuous in the distribution of the
2

data generating stochastic process. Accordingly we try to ﬁnd results similar to Hampel’s
theorem for the case of bootstrap approximations for non-i.i.d. cases.
Generalizations of Hampel’s theorem to non-i.i.d. cases can be found in Zähle (2015) and
Zähle (2016). For a slightly diﬀerent generalization of qualitative robustness, Hampel’s
theorem has been formulated for strongly stationary and ergodic processes in Cox (1981) and
Boente et al. (1982). In Strohriegl and Hable (2016) a generalization of Hampel’s Theorem
to a broad class of non-i.i.d. stochastic processes is given. Cuevas and Romo (1993) describes
a concept of qualitative robustness of bootstrap approximations for the i.i.d. case and for
real valued estimators. Also a generalization of Hampel’s theorem to this case is given. In
Christmann et al. (2013, 2011) qualitative robustness of Efron’s bootstrap approximation
is shown for the i.i.d. case for a class of regularized kernel based learning methods, i. e. not
necessarily real valued estimators. Moreover Beutner and Zähle (2016) describes consistency
of the bootstrap for plug in estimators.
The next chapter contains a deﬁnition of qualitative robustness of the bootstrap approximation of an estimator and the main results. In Chapter 2.1 Theorem 2.2 shows qualitative
robustness of the bootstrap approximation of an estimator for independent but not necessarily identically distributed random variables, Chapter 2.2 contains Theorem 2.6 and 2.7
which generalize the result in Christmann et al. (2013) to α-mixing sequences with values
in Rd . All proofs are deferred to the appendix.

2

Qualitative robustness for bootstrap estimators

Throughout this paper, let (Z, dZ ) be a Polish space with some metric dZ and Borelσ-algebra B. Denote by M(Z N ) the set of all probability measures on (Z N , B ⊗N ). Let
(Z N , B ⊗N , M(Z N )) be the underlying statistical model. If nothing else is stated, we always
use Borel-σ-algebras for topological spaces. Let (Zi )i∈N be the coordinate process on Z N ,
that is Zi : Z N → Z, (zj )j∈N 7→ zi , i ∈ N. Then the process has law PN under PN ∈ M(Z N ).
Moreover let Pn := (Z1 , . . . , Zn ) ◦ PN be the n-th order marginal distribution of PN for every
n ∈ N and PN ∈ M(Z N ). We are concerned with a sequence of estimators (Sn )n∈N on the
stochastic process (Zi )i∈N . The estimator may take its values in any Polish space H with
some metric dH ; that is, Sn : Z n → H for every n ∈ N.
Our work applies to estimators which can be represented by a statistical operator S :
M(Z) → H, that is,

S Pwn = Sn (wn ) = Sn (z1 , . . . , zn )
∀ wn = (z1 , . . . , zn ) ∈ Z n , ∀ n ∈ N,
(1)

P
where Pwn denotes the empirical measure deﬁned by Pwn (B) := n1 ni=1 IB (zi ), B ∈
B, for the observations wn = (z1 , ..., zn ) ∈ Z n . Examples of such estimators are Mestimators, R-estimators, see Huber (1981, Theorem 2.6), or Support Vector Machines,
see Hable and Christmann (2011).
3

Based on the generalization of Hampel’s concept of Π-robustness from Bustos (1980), we
deﬁne qualitative robustness for bootstrap approximations for non-i.i.d sequences of random
variables. The stronger concept of Π-robustness is needed here, as we do not assume to have
i.i.d. random variables, which are used in Cuevas and Romo (1993).
Therefore the deﬁnition of qualitative robustness stated below is stronger than the deﬁnition
in Cuevas and Romo (1993), i. e. if we use this deﬁnition for the i.i.d. case the assumption
dBL (Pn , Qn ) = dBL (⊗ni=1 P, ⊗ni=1 Q) < δ implies dBL (P, Q) < δ, where dBL denotes the
bounded Lipschitz metric. This can be seen similar to the proof of Lemma 3.1 in Section
2.1.
Now, let PN∗ be the approximation of PN with respect to the bootstrap. Deﬁne the bootstrap sample (Z1∗ , . . . , Zn∗ ) as the ﬁrst n coordinate projections Zi∗ : Z N → Z, where
the law of the stochastic process (Zi∗ )i∈N has to be chosen according to the bootstrap
procedure. For the empirical bootstrap, for example, the bootstrap sample is chosen
via drawing with replacement from the given observations
z1 , . . . , zℓ , ℓ ∈ N. Hence the
1 Pℓ
distribution of the bootstrap sample
is ⊗n∈N ℓ  i=1 δzi , with ﬁnite sample distributions

1 Pℓ
1 Pℓ
∗
∗
n
⊗j=1 ℓ i=1 δzi = (Z1 , . . . , Zn ) ⊗n∈N ℓ i=1 δzi .
Contrarily to the classical case of qualitative robustness the distribution of the estimator
unP
der Pn∗ , LPn∗ (Sn ) is a random probability measure, as the distribution Pn∗ = ⊗ni=1 1ℓ ℓi=1 δZi∗ ,
Zi∗ : Z N → Z, is random. Hence the mapping zN 7→ LPn∗ (Sn ), zN ∈ Z N , is itself a
random variable with values in M(H), i. e. on the space of probability measures on H,
equipped with the weak topology on M(H). The measurability of this mapping is ensured
by Beutner and Zähle (2016, Lemma D1).
Contrarily to the original deﬁnitions of qualitative robustness in Bustos (1980) the bounded
Lipschitz metric dBL is used instead of the Prohorov metric π for the deﬁnition of qualitative
robustness of the bootstrap approximation below. This is equivalent to Cuevas and Romo
(1993). Let X be a separable metric space, then the bounded Lipschitz metric on the space
of probability measures M(X ) on X is deﬁned by:
Z

Z
dBL (P, Q) := sup
f dP − f dQ ; f ∈ BL(X ), kf kBL ≤ 1
(y)|
where k·kBL := |·|1 +k·k∞ denotes the bounded Lipschitz norm with |f |1 = supx6=y |f (x)−f
d(x,y)
and k · k∞ the supremum norm kf k∞ := supx |f (x)| and the space of bounded Lipschitz
functions is deﬁned as BL := {f : X → R | f Lipshitz and kf kBL < ∞}. This is due to technical reasons only. Both metrics metricize the weak topology on the space of all probability
measures M(X ), for Polish spaces X , see, for example, Huber (1981, Chapter 2, Corollary
4.3) or Dudley (1989, Theorem 11.3.3), and therefore can be replaced while adapting δ on
the left hand-side of implication (2). If X is a Polish space, so is M(X ) with respect to the
weak topology, see Huber (1981, Chapter 2, Theorem 3.9). Hence the bounded Lipschitz
metric on the right-hand side of implication (2) operates on a space of probability measures
on the Polish space M(X ). Therefore the Prohorov metric and the bounded Lipschitz metric can again be replaced while adapting ε in (2). Similar to Cuevas and Romo (1993) the

4

proof of the theorems below rely on the fact that the set of bounded Lipschitz functions
BL is a uniform Glivenko-Cantelli class, which implies uniform convergence of the bounded
Lipschitz metric of the empirical measure to a limiting distribution, see Dudley et al. (1991).
Therefore the deﬁnition is given with respect to the bounded Lipschitz metric.
Definition 2.1 (Qualitative robustness for bootstrap approximations)
Let PN ∈ M(Z N ) and let PN∗ ∈ M(Z N ) be the bootstrap approximation of PN . Let P ⊂
M(Z N ) with PN ∈ P. Let Sn : Z n → H, n ∈ N, be a sequence of estimators. Then the
sequence of bootstrap approximations (LPn∗ (Sn ))n∈N is called qualitatively robust at PN with
respect to P if, for every ε > 0, there is δ > 0 such that there is n0 ∈ N such that for every
n ≥ n0 and for every QN ∈ P,
dBL (Pn , Qn ) < δ ⇒ dBL (L(LPn∗ (Sn )), L(LQ∗n (Sn ))) < ε.

(2)

Here L(LPn∗ (Sn )) (respectively L(LQ∗n (Sn ))) denotes the distribution of the bootstrap approximation of the estimator Sn under Pn∗ (respectively Q∗n ).
This deﬁnition of qualitative robustness with respect to the subset P indicates that we do
not show (2) for arbitrary probability measures QN ∈ M(Z N ). All of our results require
the contaminated process to at least have the same structure as the ideal process. This is
due to the use of the bootstrap procedure. The empirical bootstrap, which is used below,
only works well for a few processes, see for example Lahiri (2003), hence the assumptions
on the contaminated process are necessary. To our best knowledge there are no results
concerning qualitative robustness of the bootstrap approximation for general stochastic
processes without any assumptions on the second process and it is probably very hard to
show this for every QN ∈ M(Z N ), respectively P = M(Z N ). Another diﬀerence to the
classical deﬁnition of qualitative robustness in Bustos (1980) is the restriction to n ≥ n0 .
As the results for the bootstrap are asymptotic results, we can not achieve the equicontinuity
for every n ∈ N, but only asymptotically.
As the estimators can be represented by a statistical operator which depends on the empirical
measure it is crucial to concern stochastic processes which at last provide convergence of their
empirical measure. Therefore, Strohriegl and Hable (2016) proposed to choose Varadarajan
process. Let (Ω, A, µ) be a probability space. Let (Zi )i∈N , Zi : Ω → Z, i ∈ N, be a stochastic
process and Wn := (Z1 , . . . , Zn ). Then the stochastic process (Zi )i∈N is called a (strong)
Varadarajan process if there exists a probability measure P ∈ M(Z) such that
π(PWn , P ) −−−−→ 0 almost surely.
n→∞

The stochastic process (Zi )i∈N is called weak Varadarajan process if
π(PWn , P ) −−−−→ 0 in probability.
n→∞

Examples for Varadarajan processes are certain Markov Chains, some mixing processes,
ergodic process and processes which satisfy a law of large numbers for events in the sense
of Steinwart et al. (2009, Deﬁnition 2.1), see Strohriegl and Hable (2016) for details.
5

2.1

Qualitative robustness for independent not identically distributed
processes

In this section we relax the i.i.d. assumption in view of the identical distribution. We assume the random variables Zi , i ∈ N, to be independent, but not necessarily identically
distributed.
The result below generalizes Christmann et al. (2013, Theorem 3) and Christmann et al.
(2011), as the assumptions on the stochastic process are weaker as well as those on the
statistical operator. Compared to Theorem 3 in Cuevas and Romo (1993), which shows
qualitative robustness of the sequence of bootstrap estimators with values in R, we have
to strengthen the assumptions on the sample space, but do not need the estimator to
be uniformly continuous. But keep in mind, that the assumption dBL (Pn , Qn ) < δ implies dBL (P, Q) < δ, which is used for the i.i.d. case, in Christmann et al. (2013) and
Cuevas and Romo (1993).
Theorem 2.2 Let the sequence of estimators (Sn )n∈N be represented by a statistical operator S : M(Z) → H via (1) for a Polish space H and let (Z, dZ ) be a totally bounded metric
space.
Let PN = ⊗i∈N P i , P i ∈ M(Z) be an infinite product measure such that the coordinate process (Zi )i∈N , Zi : Z N → zi ,i ∈ N, is a strong Varadarajan process with limiting distribution
P . Moreover define P := QN ∈ M(Z N ); QN = ⊗i∈N Qi , Qi ∈ M(Z) . Let S : M(Z) →
H be continuous at P with respect to dBL and let the estimators Sn : Z n → H, n ∈ N, be
continuous.
Then the sequence of bootstrap approximations (LPn∗ (Sn ))n∈N , is qualitatively robust at PN
with respect to P.
Remark 2.3 The required properties on the statistical operator S and on the sequence of
estimators (Sn )n∈N in Theorem 2.2 ensure the qualitative robustness of (Sn )n∈N , as long as
the assumptions on the underlying stochastic processes are fulfilled.
The proof shows that the bootstrap approximation of every sequence of estimators (Sn )n∈N
which is qualitatively robust in the sense of the definitions in Bustos (1980) and Strohriegl and Hable
(2016, Definition 1) is qualitatively robust in the sense of Theorem 2.2.
Hence Hampel’s theorem for the i.i.d. case can be generalized to bootstrap approximations
and to the case of not necessarily identically distributed random variables if qualitative
robustness is based on the deﬁnition of Π-robustness.
Unfortunately, the assumption on the space (Z, dZ ) to be totally bounded seems to be
necessary. In the proof of Theorem 2.2 we use a result of Dudley et al. (1991) to show
uniformity on the space of probability measures M(Z). This result needs the bounded
Lipschitz functions to be a uniform Glivenko-Cantelli class, which is equivalent to (Z, dZ )
being totally bounded, see Dudley et al. (1991, Proposition 12). In order to weaken the
6

assumption on (Z, dZ ), probably another way to show uniformity on the space of probability
measures M(Z) has to be found.
A short look on the metrics used on Z n is advisable. We consider Z n as the n-fold product
space of the Polish space (Z, dZ ). The product space Z n is again a Polish space (in the
product topology) and it is tempting to use a p-product metric dn,p on Z n , that is,


dZ (z1 , z1′ ), . . . , dZ (zn , zn′ ) p
(3)
dn,p (z1 , . . . , zn ), (z1′ , . . . , zn′ ) =

where k · kp is a pn -norm on Rn for 1 ≤ p ≤ ∞. For example, dn,2 is the Euclidean metric
on Rn and dn,∞ (z1 , . . . , zn ), (z1′ , . . . , zn′ ) = maxi d(zi , zi′ ); all these metrics are strongly
equivalent. However, these common metrics do not cover the intuitive meaning of qualitative
robustness as the distance between two points in Z n (i.e., two data sets) is small only if
all coordinates are close together (small rounding errors). So points where only a small
fraction of the coordinates are far-oﬀ (gross errors) are excluded. Using these metrics, the
qualitative robustness of the sample mean at every PN ∈ M(Z N ) can be shown, see e.g.
Strohriegl and Hable (2016, Proposition 1). But the sample mean is a highly non-robust
estimator, as gross errors have great impact on the estimate. Following Boente et al. (1987),
we use the metric dn on Z n :


dn (z1 , . . . , zn ), (z1′ , . . . , zn′ ) = inf ε > 0 : ♯{i : d(zi , zi′ ) ≥ ε}/n ≤ ε .
(4)

This metric on Z n covers both kinds of "small errors". Though dn is not strongly equivalent to dn,p in general, it is topologically equivalent to the p-product metrics dn,p , see
Strohriegl and Hable (2016, Lemma 1). Hence, Z n is metrizable also with metric dn . Moreover the continuity of Sn on Z n is with respect to the product topology on Z n which can,
due to the topological equivalence of these two metrics, be seen with respect to the common
metrics dn,p .

The next part gives two examples of stochastic processes of independent, but not necessarily
identically distributed random variables, which are Varadarajan processes. In particular
these stochastic processes even satisfy a strong law of large numbers for events (SLLNE)
in the sense of Steinwart et al. (2009) and therefore are, due to Strohriegl and Hable (2016,
Theorem 2), strong Varadarajan processes. The ﬁrst example is rather simple and describes
a sequence of univariate normal distributions.
Example 1 Let (ai )i∈N ⊂ R be a sequence with limi→∞ ai = a ∈ R and let |ai | ≤ c, for
some constant c > 0 for all i ∈ N. Let (Zi )i∈N , Zi : Ω → R, be a stochastic process where
Zi , i ∈ N, are independent and Zi ∼ N (ai , 1), i ∈ N. Then the process (Zi )i∈N is a strong
Varadarajan process.
The second example are stochastic processes where the distributions of the random variables
Zi , i ∈ N, are lying in a so-called shrinking ε-neighbourhood of a probability measure P .
Example 2 Let (Z, B) be a measurable space and let (Zi )i∈N be a stochastic process with
independent random variables Zi : Ω → Z, Zi ∼ P i , where
P i = (1 − εi )P + εP̃ i
7

for a sequence εi → 0, i → ∞, εi > 0 and P̃ i , P ∈ M(Z), i ∈ N. Then the process (Zi )i∈N
is a strong Varadarajan process.
The next corollary shows, that Support Vector Machines are qualitatively robust. For a
detailed introduction to Support Vector Machines see e.g., Schölkopf and Smola (2002) and
Steinwart and Christmann (2008). Let Dn := (z1 , z2 , . . . , zn ) = ((x1 , y1 ), (x2 , y2 ), . . . , (xn , yn ))
be a given dataset.
Corollary 2.4 Let Z = X × Y, Y ⊂ R closed, be a totally bounded, metric space and let
(Zi )i∈N be a stochastic process where the random variables Zi , i ∈ N, are independent and
Zi ∼ P i := (1 − εi )P + εi P̃ i , P, P̃ i ∈ M(Z). Moreover let (λn )n∈N be a sequence of positive
real valued numbers with λn → λ0 , n → ∞, for some λ0 > 0. Let H be a reproducing
kernel Hilbert space with continuous and bounded kernel k and let Sλn : (X × Y)n → H be
the SVM estimator, which maps Dn to fL∗ ,Dn ,λn for a continuous and convex loss function
L : X × Y × Y → [0, ∞[. It is assumed that L(x, y, y) = 0 for every (x, y) ∈ X × Y and that
L is additionally Lipschitz continuous in the last argument.
Then we have for every ε > 0 there is δ > 0 such that there is n0 ∈ N such that for all
n ≥ n0 and for every process (Z̃i )i∈N , where Z̃i are independent and have distribution Qi ,
i ∈ N:
dBL (Pn , Qn ) < δ ⇒ dBL (L(LPn∗ (Sn )), L(LQ∗n (Sn ))) < ε.
That is, the sequence of bootstrap approximations is qualitatively robust if the second
(contaminated) process (Z̃i )i∈N is still of the same kind, i.e. still independent, as the original
uncontaminated process (Zi )i∈N .

2.2

Qualitative robustness for the moving block bootstrap of α-mixing
processes

Dropping the independence assumption we now focus on real valued mixing processes, in
particular on strongly stationary α-mixing or strong mixing stochastic processes. The mixing
notion is an often used and well-accepted dependence notion which quantiﬁes the degree
of dependence of a stochastic process. There exist several types of mixing coeﬃcients, but
all of them are based on diﬀerences between probabilities µ(A1 ∩ A2 ) − µ(A1 )µ(A2 ). There
is a large literature on this dependence structure. For a detailed overview on mixing, see
Bradley (2005), Bradley (2007a,b,c), and Doukhan (1994) and the references therein. The
α-mixing structure has been introduced in Rosenblatt (1956). Also examples of relations
between dependence structures and mixing coeﬃcients can be found in the references above.
Let Ω be a set equipped with two σ-algebras A1 and A2 and a probability measure µ. Then
the α-mixing coeﬃcient is deﬁned by
α(A1 , A2 , µ) := sup{|µ(A1 ∩ A2 ) − µ(A2 )µ(A2 )| | A1 ∈ A1 , A2 ∈ A2 }.
By deﬁnition the coeﬃcients equal zero, if the σ-algebras are independent.
8

Moreover mixing can be deﬁned for stochastic processes. We follow Steinwart et al. (2009,
Deﬁnition 3.1):
Definition 2.5 Let (Zi )i∈N be a stochastic process, Zi : Ω → Z, i ∈ N, and let σ(Zi ) be the
σ-algebra generated by Zi , i ∈ N. Then the α-bi- and the α-mixing coefficients are defined
by
α((Z)i∈N , µ, i, j) = α(σ(Zi ), σ(Zj ), µ)
α((Z)i∈N , µ, n) = sup α(σ(Zi ), σ(Zi+n ), µ).
i≥1

A stochastic process (Zi )i∈N is called α- mixing with respect to µ if
lim α((Z)i∈N , µ, n) = 0.

n→∞

It is called weakly α-bi-mixing with respect to µ if
n
n
1 XX
α((Z)i∈N , µ, i, j) = 0.
n→∞ n2

lim

i=1 j=1

Instead of Efron’s empirical bootstrap another bootstrap approach is used in order to represent the dependence structure of an α-mixing process. Künsch (1989) and Liu and Singh
(1992) introduced the moving block bootstrap (MBB). Often resampling of single observations can not preserve the dependence structure of the process, therefore they decided to
take blocks of length b of observations instead. The dependence structure of the process is
preserved, within these blocks. The block length b increases with the number of observations n for asymptotic considerations. A slight modiﬁcation of the original moving block
bootstrap, see for example Politis and Romano (1990) and Shao and Yu (1993), is used in
the next two theorems in order to avoid edge eﬀects.
The proofs are based on central limit theorems for empirical processes. There are several
results concerning the moving block bootstrap of the empirical process in case of mixing processes, see for example Bühlmann (1994), Naik-Nimbalkar and Rajarshi (1994), and
Peligrad (1998, Theorem 2.2) for α-mixing sequences and Radulović (1996) and Bühlmann
(1995) for β-mixing sequences. To our best knowledge there are so far no results concerning
qualitative robustness for bootstrap approximations of estimators for α-mixing stochastic
processes. Therefore, Theorem 2.6 shows qualitative robustness for a stochastic process with
values in R. The proof is based on Peligrad (1998, Theorem 2.2), which provides a central
limit theorem under assumptions on the process, which are weaker than those in Bühlmann
(1994) and Naik-Nimbalkar and Rajarshi (1994). In the case of Rd -valued, d > 1, stochastic
processes, stronger assumptions on the stochastic process are needed, as the central limit
theorem in Bühlmann (1994) requires stronger assumptions, see Theorem 2.7.
Let Z1 , . . . , Zn , n ∈ N, be the ﬁrst n projections of a real valued stochastic process (Zi )i∈N
and let b ∈ N, b < n, be the block length. Then, for ﬁxed n ∈ N, the sample can be
9

divided into blocks Bi,b := (Zi , . . . , Zi+b−1 ). If i > n − b + 1, we deﬁne Zn+j = Zj , for the
missing elements of the blocks. To get the MBB bootstrap sample Wn∗ = (Z1∗ , . . . , Zn∗ ), ℓ
numbers I1 , . . . , Iℓ from the set {1, . . . , n} are randomly chosen with replacement. Without
loss of generality it is assumed that n = ℓb, if n is not a multiple of b we simply cut
the last block, which is usually done in literature. Then the sample consists of the blocks
∗ =
∗
= ZI2 , . . . , Zℓb
BI1 ,b , BI2 ,b , . . . , BIℓ ,b , that is Z1∗ = ZI1 , Z2∗ = ZI1 +1 , . . . , Zb∗ =I1 +b−1 , Zb+1
ZIℓ +b−1 .
As we are interested in estimators Sn , n ∈ N, which can be represented by a statistical
operator S : M(Z) → H via S(Pwn ) = Sn (z1 , . . . , zn ), P
for a Polish space H, see (1), the
1
empirical measure of the bootstrap sample PWn∗ = n ni=1 δZi∗ should approximate the
P
empirical measure of the original sample PWn = n1 ni=1 δZi . Contrarily to qualitative
robustness in the case of independent and not necessarily identically distributed random
variables (Theorem 2.2), the assumptions on the statistical operator S are strengthened for
the case of α-mixing sequences. In particular the statistical operator S is assumed to be
uniformly continuous for all P ∈ (M(Z), dBL ). For the ﬁrst theorem we assume the random
variables Zi , i ∈ N, to be real valued and bounded. Without loss of generality we assume
0 ≤ Zi ≤ 1, otherwise a transformation leads to this assumption. For the bootstrap for
the true as well as for the contaminated process, we assume the block length b(n) and the
number of blocks ℓ(n) to be sequences of integers satisfying
nh ∈ O(b(n)), b(n) ∈ O(n1/3−a ), for some 0 < h <

1
1
− a, 0 < a < ,
3
3

b(n) = b(2q ) for 2q ≤ n < 2q+1 , q ∈ N, b(n) → ∞, n → ∞ and b(n) · ℓ(n) = n, n ∈ N.
Theorem 2.6 Let PN ∈ M(RN ) be a probability measure on (RN , B ⊗N ) such that the coordinate process (Zi )i∈N , Zi : RN → R is bounded, strongly stationary, and α-mixing with
X

m>n

α(σ(Z1 , . . . , Zi ), σ(Zi+m , . . .), PN ) = O(n−γ ), i ∈ N, for some γ > 0.

(5)

Let P ⊂ M(RN ) be the set of probability measures such that the coordinate process fulfils
the properties above for the same γ > 0. Let H be a Polish space, with some metric dH ,
let (Sn )n∈N be a sequence of estimators which can be represented by a statistical operator
S : M(R) → H via (1). Moreover let Sn be continuous and let S be additionally uniformly
continuous with respect to dBL . Then the sequence of estimators (Sn )n∈N is qualitatively
robust at PN with respect to P.
The assumptions on the stochastic process are on the one hand, together with the assumptions on the block length, used to ensure the validity of the bootstrap approximation and
on the other hand, together with the assumptions on the statistical operator, respectively
the sequence of estimators, to ensure the qualitative robustness.

10

The next theorem generalizes this result to stochastic processes with values in [0, 1]d , d > 1,
instead of [0, 1] ⊂ R. Therefore, for example, the bootstrap version of the SVM estimator
is qualitatively robust under weak conditions. The proof of the next theorem follows the
same lines as the proof of the theorem above, but another central limit theorem, which is
shown in Bühlmann (1994), is used. Therefore the assumptions on the mixing property of
the stochastic process are stronger and the random variables Zi , i ∈ N, are assumed to
have continuous marginal distributions. Again the bootstrap sample results of a moving
block bootstrap where ℓ(n) blocks of length b(n) are chosen, again assuming ℓ(n) · b(n) = n.
Moreover, let b(n) be a sequences of integers satisfying
1

b(n) = O(n 2 −a ) for some a > 0.
Theorem 2.7 Assume Z = [0, 1]d , d > 1. Let PN be a probability measure such that the
coordinate process (Zi )i∈N , Zi : Z N → Z is strongly stationary and α-mixing with
∞
X

1

(m + 1)8d+7 (α(σ(Z1 , . . . , Zi ), σ(Zi+m , . . .), PN )) 2 < ∞, i ∈ N.

(6)

m=0

Assume that Zi has continuous marginal distributions for all i ∈ N. Define the set of
probability measures P ⊂ M(Z) such that the coordinate process is strongly stationary and
α-mixing as in (6).
Let H be a Polish space, wit some metric dH , (Sn )n∈N be a sequence of estimators such that
Sn : Z n → H is continuous and assume that Sn can be represented by a statistical operator
S : M(Z) → H via (1) which is additionally uniformly continuous with respect to dBL .
Then the sequence of estimators (Sn )n∈N is qualitatively robust at PN with respect to P.
Although the assumptions on the statistical operator S, compared to Theorem 2.2, were
strengthened in order to generalize the qualitative robustness to α-mixing sequences in
Theorem 2.6 and 2.7, M-estimators are still an example for qualitative robust estimators
if the sample space (Z, dZ ), Z ⊂ R is compact. The compactness of (Z, dZ ) implies the
compactness of the space (M(Z), dBL ), see Parthasarathy (1967, Theorem 6.4). As the
statistical operator S is continuous, the compactness of M(Z) implies the uniform continuity
of S. Another example of M-estimators which are uniformly continuous even if the input
space is not compact is given in Cuevas and Romo (1993, Theorem 4).
Acknowledgements: This research was partially supported by the DFG Grant 291/2-1
"Support Vector Machines bei stochastischer Unabhängigkeit". Moreover I would like to
thank Andreas Christmann for helpful discussions on this topic.

3

Proofs

This section contains the proofs of the main theorems and corollaries.
11

3.1

Proofs of Section 2.1

Before proving Theorem 2.1, we state a rather technical lemma, connecting the product
measure
⊗ni=1 P i ∈ M(Z n ) of independent random variables to their mixture measure
1 Pn
i
i=1 P ∈ M(Z). Let (Z, dZ ) be a Polish space.
n
Lemma 3.1 Let Pn , Qn ∈ M(Z n ) such that Pn = ⊗ni=1 P i and Qn = ⊗ni=1 Qi , P i , Qi ∈
M(Z), i ∈ N. Then for all δ > 0:
!
n
n
1X i 1X i
dBL (Pn , Qn ) ≤ δ ⇒ dBL
P ,
Q ≤ δ.
n
n
i=1

i=1

Proof: Let BL1 be the set of bounded Lipschitz functions with kf kBL ≤ 1.By assumption
we have dBL (Pn , Qn ) ≤ δ. Moreover for a function f : Z → R:
Z
Z
Z

i
f (zi ) dP i (zi ) d ⊗j6=i P j (zj ) .
(7)
f (zi ) dP (zi ) =
Z n−1

Z

Z

Then,
"

" n
# Z
#
n
1X i
1X i
f (zi ) d
f (zi ) d
sup
P (zi ) −
Q (zi )
n
n
Z
f ∈BL1 (Z) Z
i=1
i=1

Z
n Z
1X
i
i
=
sup
f (zi ) dQ (zi )
f (zi ) dP (zi ) −
f ∈BL1 (Z) n i=1
Z
Z
Z
n Z

1X
(7)
= sup
f (zi ) dP i (zi ) d ⊗j6=i P j (zj )
f ∈BL1 (Z) n i=1
Z n−1 Z

Z
Z

i
j
f (zi ) dQ (zi ) d ⊗j6=i Q (zj )
−
Z

=

sup
f ∈BL1 (Z)
n

Z n−1 Z
n Z
X

1
n

i=1

1X
≤
sup
n
f ∈BL1 (Z)
i=1

Z

Zn

Zn

f (zi ) d

f (zi ) d



⊗nj=1 P j (zj )


⊗nj=1 P j (zj )

−

−

Z

Zn

Z

Zn

f (zi ) d



⊗nj=1 Qj (zj )




f (zi ) d ⊗nj=1 Qj (zj ) .

Now every function f ∈ BL1 (Z) can be identiﬁed as a function f˜ : Z n → Z, (z1 , . . . , zn ) 7→
f˜(z1 , . . . , zn ) := f (zi ). This function is also Lipschitz continuous on Z n :
|f˜(z1 , . . . , zn )−f˜(z1′ , . . . , zn′ )| = |f (zi ) − f (zi′ )|

≤ |f |1 d(zi , zi′ ) ≤ |f |1 (dZ (z1 , z1′ ) + . . . + dZ (zi , zi′ ) + . . . + dZ (zn , zn′ )),
12

where dZ (z1 , z1′ ) + . . . + dZ (zi , zi′ ) + . . . + dZ (zn , zn′ ) induces the product topology on Z n .
That is f˜ ∈ BL1 (Z n ). Note that this is also true for every p-product metric dn,p in Z n ,
1 ≤ p ≤ ∞, as they are strongly equivalent. Hence,
!
Z
Z
n
n
n
1X
1X i 1X i
g dQn
g dPn −
P ,
Q ≤
sup
dBL
n
n
n
Zn
g∈BL1 (Z n ) Z n
i=1

i=1

i=1

n
1X
≤
dBL (Pn , Qn ) ≤ δ,
n
i=1

which yields the assertion.



Proof of Theorem 2.2: To prove Theorem 2.2 we ﬁrst use the triangle inequality to
split the bounded Lipschitz distance between the distribution of the estimator Sn , n ∈ N,
into two parts regarding the distribution of the estimator under the joint distribution Pn of
(Z1 , . . . , Zn ):
dBL (LPn∗ (Sn ), LQ∗n (Sn )) ≤ dBL (LPn∗ (Sn ), LPn (Sn )) + dBL (LPn (Sn ), LQ∗n (Sn )) .
{z
} |
{z
}
|
I

II

Then the representation of the estimator Sn by the statistical operator S and the continuity of this operator in P together with the Varadarajan property and the independence
assumption on the stochastic process yield the assertion.
First we regard part I: Deﬁne the distribution PN ∈ M(Z N ) and let PN∗ be the bootstrap
approximation of PN . Deﬁne, for n ∈ N, the random variables
Wn : Z N → Z n , Wn = (Z1 , . . . , Zn ), zN 7→ Wn (zN ) = wn = (z1 , . . . , zn ), and
Wn′ : Z N → Z n , Wn′ = (Z1′ , . . . , Zn′ ), zN 7→ wn′ ,
such that Wn (PN ) = Pn and Wn′ (PN∗ ) = Pn∗ .
Denote the bootstrap sample by Wn∗ := (Z1∗ , . . . , Zn∗ ), Wn∗ : Z N → Z n , zN 7→ wn∗ .
As Efron’s empirical bootstrap is used, the bootstrap sample, which is chosen
Pvia resampling
with replacement out of Z1 , . . . , Zℓ , ℓ ∈ N, has distribution Zi∗ ∼ PWℓ = 1ℓ ℓj=1 δZj , i ∈ N,
approximation of Pℓ , ℓ ∈ N,
respectively Wn∗ := (Z1∗ , . . . , Zn∗ ) ∼ ⊗ni=1 PWℓ . The bootstrap P
is the empirical measure of the bootstrap sample Pℓ∗ = ⊗ℓi=1 n1 nj=1 δZj∗ .
∗ , and W′ by K ∈ M(Z N × Z N × Z N ).
Further denote the joint distribution of WN , WN
N
N
Then, KN has marginal distributions KN (B1 × Z N × Z N ) = PN (B1 ) for all B1 ∈ B ⊗N ,
KN (Z N × B2 × Z N ) = ⊗i∈N PWn (B2 ) for all B2 ∈ B ⊗N , and KN (Z N × Z N × B3 ) = PN∗ (B3 )
for all B3 ∈ B ⊗N .

Then,
LPn (Sn ) = Sn (Pn ) = Sn ◦ Wn (PN ) and LPn∗ (Sn ) = Sn (Pn∗ ) = Sn ◦ Wn′ (PN∗ )
13

and therefore
dBL (LPn∗ (Sn ), LPn (Sn )) = dBL (L(Sn ◦ Wn′ ), L(Sn ◦ Wn )).
By assumption the coordinate process (Zi )i∈N consists of independent random variables,
hence we have Pn = ⊗ni=1 P i , for P i = Zi (PN ), i ∈ N.
Moreover (Z, dZ ) is assumed to be a totally bounded metric space. Then, due to Dudley et al.
(1991, Proposition 12), the set BL1 (Z, dZ ) is a uniform Glivenko-Cantelli class. That is, if
Zi ∼ P i.i.d. i ∈ N, we have for all η > 0:


N
lim sup PN
zN ∈ Z | sup dBL (PWm (zN ) , P ) > η
= 0.
n→∞ P ∈M(Z)

m≥n

∗ ), m ∈ N, which is found by resampling
Applying this to the bootstrap sample (Z1∗ , . . . , Zm
with replacement out of the original sample (Z1 , . . . , Zn ), we have, for all wn ∈ Z n ,


lim
sup ⊗i∈N Pwn
zN ∈ Z N | sup dBL (PWm
∗ (z ) , Pwn ) > η
= 0.
N
n→∞ P

m≥n

wn ∈M(Z)

Let ε > 0 be arbitrary but ﬁxed. Then, for every δ0 > 0 there is n1 ∈
n ≥ n1 and all Pwn ∈ M(Z):


δ0
n
∗
n
∗
⊗i=1 Pwn
wn ∈ Z | dBL (Pwn , Pwn ) ≤
≥1−
4

N such that for all
ε
.
8

(8)

And, using the same argumentation
for the sequence of random variables Zi′ , i ∈ N, which
1 Pn
are i.i.d. and have distribution n i=1 δZi∗ = PWn∗ :


∗
N
lim
sup PN
zN ∈ Z | sup dBL (PWm
′ (z ) , Pw∗ ) > η
= 0.
N
n
n→∞ P

∗ ∈M(Z)
wn

m≥n

Respectively, for every δ0 > 0 there is n2 ∈ N such that for all n ≥ n2 and all Pwn∗ ∈ M(Z):


δ0
ε
∗
′
n
Pn
wn ∈ Z | dBL (Pwn′ , Pwn∗ ) ≤
(9)
≥1− .
2
8
As the process (Zi )i∈N is a strong Varadarajan process by assumption, there exists a probability measure P ∈ M(Z) such that
dBL (PWn , P ) −→ 0 almost surely with respect to PN , n → ∞.
That is, for every δ0 > 0 there is n3 ∈ N such that for all n ≥ n3 :


ε
δ0
n
≥1− .
Pn
wn ∈ Z | dBL (Pwn , P ) ≤
2
4
14

(10)

The continuity of the statistical operator S : M(Z) → H in P ∈ M(Z) yields: for every
ε > 0 there exists δ0 > 0 such that for all Q ∈ M(Z):
dBL (P, Q) ≤ δ0

⇒

ε
dH (S(P ), S(Q)) ≤ .
4

(11)

As the Prohorov metric πdH is bounded by the Ky Fan metric, see Dudley (1989, Theorem
11.3.5) we conclude:
πdH (LPn∗ (Sn ), LPn (Sn )) = πdH (Sn ◦ Wn′ , Sn ◦ Wn )



≤ inf ε̃ > 0 | KN dH (Sn ◦ Wn′ , Sn ◦ Wn ) > ε̃ ≤ ε̃


= inf ε̃ > 0 | (Wn , Wn∗ , Wn′ )(KN ) (wn , wn∗ , wn′ ) ∈ Z n × Z n × Z n |

dH (Sn (wn′ ), Sn (wn )) > ε̃, wn∗ ∈ Z n ≤ ε̃ .
Due to the deﬁnition of the statistical operator S, this is equivalent to


inf ε̃ > 0 | (Wn , Wn∗ , Wn′ )(KN ) (wn , wn∗ , wn′ ) ∈ Z n × Z n × Z n |
dH (S(Pwn′ ), S(Pwn )) > ε̃, wn∗ ∈ Z n



(12)

≤ ε̃ .

The triangle inequality
dH (S(Pwn′ ), S(Pwn )) ≤ dH (S(Pwn′ ), S(P )) + dH (S(P ), S(Pwn )),
and the continuity of the statistical operator S, see (11), then yield, for all ε > 0,
o
n
ε
(Wn , Wn∗ , Wn′ )(KN ) (wn , wn∗ , wn′ ) ∈ Z n × Z n × Z n | dH (S(Pwn′ ), S(Pwn )) > , wn∗ ∈ Z n
2
n
ε
∗
′
∗
′
n
n
n
≤ (Wn , Wn , Wn )(KN ) (wn , wn , wn ) ∈ Z × Z × Z | dH (S(Pwn′ ), S(P )) >
4
o
ε
or dH (S(P ), S(Pwn )) > , wn∗ ∈ Z n
4
(11)

≤ (Wn , Wn∗ , Wn′ )(KN ) (wn , wn∗ , wn′ ) ∈ Z n × Z n × Z n | dBL (Pwn′ , P ) > δ0
or dBL (P, Pwn ) > δ0 , wn∗ ∈ Z n }) .

Using the triangle inequality,

and

dBL (Pwn′ , P ) ≤ dBL (Pwn′ , Pwn∗ ) + dBL (Pwn∗ , P )

dBL (Pwn∗ , P ) ≤ dBL (Pwn∗ , Pwn ) + dBL (Pwn , P ),

15

(13)
(14)

gives for all n ≥ max{n1 , n2 , n3 }:

(Wn , Wn∗ , Wn′ )(KN ) (wn , wn∗ , wn′ ) ∈ Z n × Z n × Z n | dBL (Pwn′ , P ) > δ0
or dBL (P, Pwn ) > δ0 , wn∗ ∈ Z n })


δ0
(wn , wn∗ , wn′ ) ∈ Z n × Z n × Z n | dBL (Pwn′ , Pwn∗ ) >
2

δ0
or dBL (Pwn∗ , P ) >
or dBL (P, Pwn ) > δ0
2

(14)
δ0
∗
′
≤ (Wn , Wn , Wn )(KN )
(wn , wn∗ , wn′ ) ∈ Z n × Z n × Z n | dBL (Pwn′ , Pwn∗ ) >
2

δ0
δ0
or dBL (P, Pwn ) >
or dBL (Pwn∗ , Pwn ) >
4
4




δ0
δ0
∗
′
n
n
≤ Pn
wn ∈ Z | dBL (Pwn′ , Pwn∗ ) >
wn ∈ Z | dBL (Pwn , P ) >
+ Pn
2
4


δ0
+ ⊗ni=1 Pwn
wn∗ ∈ Z n | dBL (Pwn∗ , Pwn ) >
4
(8),(9),(10) ε
ε ε
ε
<
+ +
=
.
8 4 8
2

(13)

≤ (Wn , Wn∗ , Wn′ )(KN )

Hence, for all ε > 0 there are n1 , n2 , n3 ∈ N such that vor all n ≥ max{n1 , n2 , n3 }, the
inﬁmum in (12) is bounded by 2ε . Therefore
πdH (LPn∗ (Sn ), LPn (Sn )) <

ε
.
2

The equivalence between the Prohorov metric and the bounded Lipschitz metric for Polish
spaces, see Huber (1981, Chapter 2, Corollary 4.3), yields the existence of n0,1 ∈ N such
that for all n ≥ n0,1 :
ε
(15)
dBL (LPn∗ (Sn ), LPn (Sn )) < .
2
To prove the convergence of the term in part II, consider the distribution QN ∈ M(Z N ) and
let Q∗N be the bootstrap approximation of QN . Deﬁne, for n ∈ N, the random variables
W̃n : Z N → Z n , W̃n = (Z̃1 , . . . , Z̃n ), zN 7→ w̃n with distribution W̃n (QN ) = Qn ,
W̃n′ : Z N → Z n , W̃n′ = (Z̃1′ , . . . , Z̃n′ ), zN 7→ w̃n′ , with distribution W̃n′ (Q∗N ) = Q∗n , and
the bootstrap sample
W̃n∗ : Z N → Z n , W̃n∗ = (Z̃1∗ , . . . , Z̃n∗ ), zN 7→ w̃n∗ , with distribution
P
ℓ
⊗ni=1 QW̃ℓ = ⊗ni=1 1ℓ i=1 δZ̃i .
Moreover let K̃N ∈ M(Z N × Z N × Z N × Z N ) denote the joint distribution of WN , W̃N ,
∗ , and W̃′ . Then, K̃ ∈ M(Z N × Z N × Z N × Z N ) has marginal distributions P , Q ,
W̃N
N
N
N
N
⊗i∈N QW̃n , and Q∗N .

16

First, similar to the argumentation for part I, Efron’s bootstrap and Dudley et al. (1991,
Proposition 12) give for w̃n ∈ Z n :


N
⊗n∈N Qw̃n
lim
sup
zN ∈ Z | sup dBL (QW̃∗ (zN ) , Qw̃n ) > η
= 0.
n→∞ Q

m

m≥n

w̃n ∈M(Z)

Hence, for arbitrary, but ﬁxed ε > 0, for every δ0 > 0 there is n4 ∈ N such that for all
n ≥ n4 and all Qw̃n ∈ M(Z):


δ0
ε
∗
n
n
w̃n ∈ Z | dBL (Qw̃n∗ , Qw̃n ) ≤
⊗i=1 Qw̃n
(16)
≥1− .
6
10
Further,
lim

n→∞ Q

sup
∗ ∈M(Z)
w̃n

Q∗N


zN ∈ Z N | sup dBL (QW̃′
m≥n

m

∗
(zN ) , Qw̃n ) > η



= 0.

Respectively,
for every δ0 > 0 there is n5 ∈ N such that for all n ≥ n5 and all Qw̃n∗ =
1 Pn
∗
i=1 δz̃i ∈ M(Z):
n
Q∗n



δ0
ε
′
n
w̃n ∈ Z | dBL (Qw̃n′ , Qw̃n∗ ) ≤
≥1− .
6
10

(17)

Moreover, as the random variables Zi , Zi ∼ P i , i ∈ N,
Pare independent, the bounded
Lipschitz distance between the empirical measure and n1 ni=1 P i can be bounded, due to
Dudley et al. (1991, Theorem 7). As totally bounded spaces are particularly separable,
see Denkowski et al. (2003, below Corollary 1.4.28), Dudley et al. (1991, Proposition 12)
provides that BL1 (Z, dZ ) is a uniform Glivenko-Cantelli class. The proof of this proposition
does not depend on the distributions of the random variables Zi , i ∈ N, and is therefore also
valid for independent and not necessarily identically distributed random variables. Hence
Dudley et al. (1991, Theorem 7) yields for all η > 0:
!
)!
(
n
X
1
= 0,
zN ∈ Z N | sup dBL PWm (zN ) ,
PN
lim
sup
Pi > η
n→∞ (P i )
n
N
m≥n
i∈N ∈(M(Z))
i=1

as long as the assumptions of Proposition 12 in Dudley et al. (1991) apply. As BL1 (Z, dZ )
is bounded, we have F0 = BL1 (Z, dZ ), see Dudley et al. (1991, page 499, before Proposition
10), hence it is suﬃcient to show that BL1 (Z, dZ ) is image admissible Suslin. By assumption (Z, dZ ) is totally bounded, hence BL1 (Z, dZ ) is separable with respect to k · k∞ , see
Strohriegl and Hable (2016, Lemma 3). As f ∈ BL1 (Z, dZ ) implies kf k∞ ≤ 1, the space
BL1 (Z, dZ ) is a bounded subset of (Cb (Z, dZ ), k · k∞ ), which is due to Dudley (1989, Theorem 2.4.9) a complete space. Now, BL1 (Z, dZ ) is a closed subset of (Cb (Z, dZ ), k · k∞ ) with
17

respect to k · k∞ . Hence BL1 (Z, dZ ) is complete, due to Denkowski et al. (2003, Proposition
1.4.17). Therefore BL1 (Z, dZ ) is separable and complete with respect to k · k∞ and particularly a Suslin space, see Dudley (2014, p.229). As Lipschitz continuous functions are also
equicontinuous, Dudley (2014, Theorem 5.28 (c)) gives that BL1 (Z, dZ ) is image admissible
Suslin.
Hence, Dudley et al. (1991, Theorem 7) yields
!
n
1X i
−→ 0 almost surely with respect to PN , n → ∞,
P
dBL PWn ,
sup
n
(P i )i∈N ∈(M(Z))N
i=1

and
n

dBL

sup
(Qi )i∈N ∈(M(Z))N

1X i
QW̃n ,
Q
n
i=1

!

−→ 0 almost surely with respect to QN , n → ∞.

That is, there is n6 ∈ N such that for all n ≥ n6
(
wn ∈ Z n | dBL

Pn

and Qn

(

w̃n ∈ Z n | dBL

n

1X i
P wn ,
P
n

1
Qw̃n ,
n

i=1
n
X
i=1

Qi

!

δ0
≤
6

)!

≥1−

ε
,
10

(18)

!

δ0
≤
6

)!

≥1−

ε
.
10

(19)

Moreover, due to Lemma 3.1, we have
δ0
dBL (Pn , Qn ) ≤
6

n

⇒

dBL

n

1X i 1X i
P ,
Q
n
n
i=1

i=1

!

≤

δ0
.
6

(20)

Then the strong Varadarajan property of (Zi )i∈N yields that there is n7 ∈ N such that for
all n ≥ n7 :


ε
δ0
n
(21)
Pn
≥1− .
wn ∈ Z | dBL (Pwn , P ) ≤
6
10
Similar to the argumentation for part I we conclude, using again the boundedness of the
Prohorov metric πdH by the Ky Fan metric, see Dudley (1989, Theorem 11.3.5):
πdH (LPn (Sn ), LQ∗n (Sn )) = πdH (Sn ◦ Wn , Sn ◦ W̃n′ )

= inf{ε̃ > 0 | (Wn , W̃n , W̃n∗ , W̃n′ )(K̃N ) (wn , w̃n , w̃n∗ , w̃n′ ) ∈ Z n × Z n × Z n × Z n |

dH (Sn (wn ), Sn (w̃n′ )) > ε̃, w̃n , w̃n∗ ∈ Z n ≤ ε̃}.

Due to the deﬁnition of the statistical operator S, this is equivalent to

inf{ε̃ > 0 | (Wn , W̃n , W̃n∗ , W̃n′ )(K̃N ) (wn , w̃n , w̃n∗ , w̃n′ ) ∈ Z n × Z n × Z n × Z n |

dH (S(Pwn ), S(Qw̃n′ )) > ε̃, w̃n , w̃n∗ ∈ Z n ≤ ε̃}.
18

Moreover the triangle inequality yields
dH (S(Pwn ), S(Qw̃n′ )) ≤ dH (S(Pwn ), S(P )) + dH (S(P ), S(Qw̃n′ )).
Hence, for all n ≥ max{n4 , n5 , n6 , n7 }, we obtain
n
(Wn , W̃n , W̃n∗ , W̃n′ )(K̃N ) (wn , w̃n , w̃n∗ , w̃n′ ) ∈ Z n × Z n × Z n × Z n |
o
ε
dH (S(Pwn ), S(Qw̃n′ )) > , w̃n , w̃n∗ ∈ Z n
2
n
≤ (Wn , W̃n , W̃n∗ , W̃n′ )(K̃N )

(wn , w̃n , w̃n∗ , w̃n′ ) ∈ Z n × Z n × Z n × Z n |
o
ε
ε
dH (S(Pwn ), S(P )) > or dH (S(P ), S(Qw̃n′ )) > , w̃n , w̃n∗ ∈ Z n .
4
4

The continuity of the statistical operator S in P , see (11), gives
dBL (P, QW̃′ ) ≤ δ0

⇒

and dBL (P, PWn ) ≤ δ0

⇒

n

ε
,
4
ε
dH (S(P ), S(PWn )) ≤ .
4
dH (S(P ), S(QW̃′ )) ≤
n

Further, the triangle inequality yields
n

dBL (P, Qw̃n′ ) ≤ dBL (P, Pwn ) + dBL
+ dBL

1X i
P
P wn ,
n
i=1
!

n
1X i
Q , Qw̃n
n
i=1

!

+ dBL

n

n

i=1

i=1

1X i 1X i
P ,
Q
n
n

!

+ dBL (Qw̃n , Qw̃n∗ ) + dBL (Qw̃n∗ , Qw̃n′ ).

(22)

Therefore we conclude, for all n ≥ max{n4 , n5 , n6 , n7 },

(Wn , W̃n , W̃n∗ , W̃n′ )(K̃N ) (wn , w̃n , w̃n∗ , w̃n′ ) ∈ Z n × Z n × Z n × Z n |
o
ε
ε
dH (S(Pwn ), S(P )) > or dH (S(P ), S(Qw̃n′ )) > , w̃n , w̃n∗ ∈ Z n
4
4
(11)

≤ (Wn , W̃n , W̃n∗ , W̃n′ )(K̃N ) (wn , w̃n , w̃n∗ , w̃n′ ) ∈ Z n × Z n × Z n × Z n |

dBL (Pwn , P ) > δ0 or dBL (P, Qw̃n′ ) > δ0 , w̃n , w̃n∗ ∈ Z n
(22)


(wn , w̃n , w̃n∗ , w̃n′ ) ∈ Z n × Z n × Z n × Z n |
!
n
δ0
1X i
δ0
>
P
or dBL Pwn ,
dBL (Pwn , P ) >
6
n
6
i=1
!
!
n
n
n
δ0
1X i
1X i 1X i
δ0
or dBL
P ,
Q >
Q , Qw̃n >
or dBL
n
n
6
n
6
i=1
i=1
i=1

δ0
δ0
or dBL (Qw̃n∗ , Qw̃n′ ) >
or dBL (Qw̃n , Qw̃n∗ ) >
.
6
6

≤ (Wn , W̃n , W̃n∗ , W̃n′ )(K̃N )

19


P
P
Now, assume dBL (Pn , Qn ) ≤ δ60 , then (20) yields dBL n1 ni=1 P i , n1 ni=1 Qi ≤ δ60 , therefore this term can be omitted. Note that this is only proven for the p-product metrics on
Z n and not for the metric dn from (4). For this metric we need a diﬀerent argumentation,
which is stated below the next calculation.
Hence, for all n ≥ max{n4 , n5 , n6 , n7 },

(Wn , W̃n , W̃n∗ , W̃n′ )(K̃N ) (wn , w̃n , w̃n∗ , w̃n′ ) ∈ Z n × Z n × Z n × Z n |

dH (S(Pwn ), S(Qw̃n′ )) > ε, w̃n , w̃n∗ ∈ Z n

(20)




(wn , w̃n , w̃n∗ , w̃n′ ) ∈ Z n × Z n × Z n × Z n |
!
!
n
n
δ0
δ0
1X i
1X i
δ0
dBL (Pwn , P ) >
>
or dBL Pwn ,
P
or dBL
Q , Qw̃n >
6
n
6
n
6
i=1
i=1

δ0
δ0
or dBL (Qw̃n∗ , Qw̃n′ ) >
or dBL (Qw̃n , Qw̃n∗ ) >
6
6


δ0
≤ Pn
wn ∈ Z n | dBL (Pwn , P ) >
6
!
)!
(
n
X
δ
1
0
Pi >
wn ∈ Z n | dBL Pwn ,
+ Pn
n
6
i=1
!
(
)!
n
δ0
1X i
n
+ Qn
w̃n ∈ Z | dBL
Q , Qw̃n >
n
6
i=1


 δ0
∗
n
n
w̃n ∈ Z | dBL Qw̃n , Qw̃n∗ >
+ ⊗i=1 Qw̃n
6



δ
0
w̃n′ ∈ Z n | dBL Qw̃n∗ , Qw̃n′ >
+ Q∗n
6
(16),(17)(18),(19),(21) ε
ε
ε
ε
ε
ε
+
+
+
+
= .
<
10 10 10 10 10
2
≤ (Wn , W̃n , W̃n∗ , W̃n′ )(K̃N )

In order to show the above bound for the metric dn , see (4), on Z n , we use another variant
of the triangle inequality in (22):
dBL (P, Qw̃n′ ) ≤ dBL (P, Pwn ) + dBL (Pwn , Qw̃n ) + dBL (Qw̃n , Qw̃n∗ ) + dBL (Qw̃n∗ , Qw̃n′ ). (23)
δ2

Assume dBL (Pn , Qn ) ≤ 640 . Then, the strong equivalence between the Prohorov metric
and the bounded Lipschitz metric
on Polish spaces, see Huber (1981, Chapter 2, Corollary
p
4.3), yields πdn (Pn , Qn ) ≤ dBL (Pn , Qn ) ≤ δ80 . Due to Dudley (1989, Theorem 11.6.2),
probability measure µ ∈ M(Z n × Z n ) with
πdn (Pn , Qn ) ≤ δ80 implies the existence of a n
o

≤
(wn , w̃n ) ∈ Z n × Z n | dn (wn , w̃n ) > δ80
 δ0
δ0
δ0
1 Pn
1 Pn
i=1 δzi , n
i=1 δz̃i ≤ 8 and
8 . By a simple calculation dn (wn , w̃n ) ≤ 8 implies πdn n
we have:


δ0
δ0
n
n
≤ .
µ
(wn , w̃n ) ∈ Z × Z | πdn (Pwn , Qw̃n ) >
8
8

marginal distributions Pn and Qn , such that µ

20

Again the equivalence between the metrics π and dBL yields:
µ



δ0
δ0
(wn , w̃n ) ∈ Z n × Z n | dBL (Pwn , Qw̃n ) >
≤ .
4
8

∗ , and W̃′ such that the distriNow we choose the joint distribution K̃N of WN , W̃N , W̃N
N
N
N
n
n
n
bution of (Wn , W̃n ) : Z × Z → Z × Z is µ ∈ M(Z × Z n ). Then we conclude:

(Wn , W̃n , W̃n∗ , W̃n′ )(K̃N ) (wn , w̃n , w̃n∗ , w̃n′ ) ∈ Z n × Z n × Z n × Z n |
o
ε
ε
dH (S(Pwn ), S(P )) > or dH (S(P ), S(Qw̃n′ )) > , w̃n , w̃n∗ ∈ Z n
4
4
(11),(23)

≤ (Wn , W̃n , W̃n∗ , W̃n′ )(K̃N ) (wn , w̃n , w̃n∗ , w̃n′ ) ∈ Z n × Z n × Z n × Z n |
δ0
δ0
dBL (Pwn , P ) >
or dBL (Pwn , Qw̃n ) >
4
4

δ0
δ0
or dBL (Qw̃n∗ , Qw̃n′ ) >
.
or dBL (Qw̃n , Qw̃n∗ ) >
4
4


δ0
n
≤
Pn
wn ∈ Z | dBL (Pwn , P ) >
4


δ0
+µ
(wn , w̃n ) ∈ Z n × Z n | dBL (Pwn , Qw̃n ) >
4


 δ0
+ ⊗ni=1 Qw̃n
w̃n∗ ∈ Z n | dBL Qw̃n , Qw̃n∗ >
4



δ0
w̃n′ ∈ Z n | dBL Qw̃n∗ , Qw̃n′ >
+ Q∗n
.
4

Now, adapting the inequalities in (16), (17), and (21) in ε respectively n yields the boundδ2
edness of the above term by 2ε for dBL (Pn , Qn ) ≤ 640 and for all n ≥ {n4 , n5 , n7 }.
Now we can go on with the proof similar for both kinds of metrics on Z n .

The equivalence between the Prohorov metric and the bounded Lipschitz metric on Polish
spaces, see Huber (1981, Chapter 2, Corollary 4.3), yields the existence of n0,2 ∈ N such

that for all n ≥ n0,2 , dBL (Pn , Qn ) ≤

δ0
6

(respectively dBL (Pn , Qn ) ≤

ε
dBL (LPn (Sn ), LQ∗n (Sn )) < .
2

δ02
64 )

implies

(24)

Now, (15) and (24) yield for all n ≥ max{n0,1 , n0,2 }:
dBL (LPn∗ (Sn ), LQ∗n (Sn )) < ε.

(25)

Recall that LPn∗ (Sn ) =: ζn and LQ∗n (Sn ) =: ξn are random quantities with values in M(H).
Hence (25) is equivalent to


E dBL (LPn∗ (Sn ), LQ∗n (Sn )) < ε, for all n ≥ max{n0,1 , n0,2 },
21

respectively
E [dBL (ζn , ξn )] < ε, for all n ≥ max{n0,1 , n0,2 }.
Therefore, for all f ∈ BL1 (M(Z)) and for all n ≥ max{n0,1 , n0,2 }:
Z
Z
f d(L(ζn )) − f d(L(ξn )) = |Ef (ζn ) − Ef (ξn )| ≤ E |f (ζn ) − f (ξn )|
≤ E (|f |1 dBL (ζn , ξn ))

< ε,

by a variant of Strassen’s Theorem, see Huber (1981, Chapter 2, Theorem 4.2, (2)⇒(1)).
That is,
dBL (L(LPn∗ (Sn )), L(LQ∗n (Sn ))) < ε for all n ≥ max{n0,1 , n0,2 }.
Hence for every ε > 0 we ﬁnd δ =

δ0
6

and n0 = max{n0,1 , n0,2 } such that for all n ≥ n0 :

dBL (Pn , Qn ) < δ

⇒

dBL (L(LPn∗ (Sn )), L(LQ∗n (Sn ))) < ε,

which yields the assertion.



Proof of Example 1:
Without any restriction we assume a = 0. Otherwise regard the process Zi − a, i ∈ N. By
assumption, the random variables Zi , i ∈ N, are independent. Hence IB ◦ Zi , i ∈ N, are
independent, see for example Hoﬀmann-Jørgensen (1994, Theorem 2.10.6) for all measurable
B ∈ B, as IB is a measurable function. According to Steinwart et al. (2009, Proposition
2.8), (Zi )i∈N
P satisﬁes the SLLNE if there is a probability measure P in M(Z) such that
limn→∞ n1 ni=1 Eµ IB ◦ Zi = P (B) for all measurable B ∈ B. Hence:
n

n

1X
1X
E µ I B ◦ Zi =
n
n
i=1

i=1

1

Z

n

1X
IB dZi (µ) =
n
i=1

Z

IB fi dλ1 ,

2

where fi (x) = √12π e− 2 (x−ai ) denotes the density of the normal distribution N (0, 1) with
respect to the Lebesgue measure λ1 . Moreover deﬁne g : R → R by

− 1 (x+c)2

, x < −c
 e 2
1
√ ,
−c ≤ x ≤ c
x ∈ R.
g(x) =
2π

 − 21 (x−c)2
, c<x
e

Therefore |fi | ≤ |g|, for all i ∈ N, g is integrable and due to Lebesgue’s Theorem, see for
example Hoﬀmann-Jørgensen (1994, Theorem 3.6):
n

1X
n→∞ n
lim

i=1

Z

IB fi dλ1 = lim

n→∞

Z

n

1X
IB fi dλ1 =
n
i=1

22

Z

n

1X
IB fi dλ1 .
n→∞ n
lim

i=1

(26)

1 2

We have fi → f0 , where f0 = √12π e− 2 x for all x ∈ R, as ai → 0 and therefore the Lemma of
Kronecker,P
see for example Hoﬀmann-Jørgensen (1994, Theorem 4.9, Equation 4.9.1) yields:
limn→∞ n1 ni=1 fi (x) = f0 (x) for all x ∈ X .

Now (26) yields the SLLNE:
n

1X
n→∞ n
lim

i=1

Z

IB fi dλ1 =

Z

IB f0 dλ1 = P (B), for al B ∈ B.

With Strohriegl and Hable (2016, Zheorem 2) the Varadarajan property is given.



Proof of Example 2:
Similar to the proof of Example 1, we ﬁrst show the SLLNE, that is there exists a probability
measure P ∈ M(Z) such that
n

1X
n→∞ n
lim

i=1

Z

IB ◦ Zi dµ = P (B), for all measurable B ⊂ Ω.

Now let B ⊂ Ω be an arbitrary measurable set. Then:
n

n Z
n Z
1X
1X
i
IB dP = lim
IB d[(1 − εi )P + εi P̃ i ]
IB ◦ Zi dµ = lim
n→∞ n
n→∞ n
Z
Z
i=1
i=1
i=1
Z
Z
n Z
n
n
X
X
1X
1
1
IB dP + lim
IB dP̃ i .
(27)
IB dP − lim
εi
εi
= lim
n→∞ n
n→∞ n
n→∞ n
Z
Z
Z

1X
lim
n→∞ n

Z

i=1

As, 0 ≤

1
n

Pn

i=1 εi

R

i=1

IB dP ≤

1
n

n

Pn

i=1 εi

1X
lim
εi
n→∞ n
i=1

Z

i=1

and εi → 0, we have
n

1X
IB dP ≤ lim
εi −→ 0, n → ∞
n→∞ n
i=1

and similarly
n

1X
εi
lim
n→∞ n
i=1

Z

n

1X
εi −→ 0,
n→∞ n

IB dP̃ i ≤ lim

i=1

n → ∞.

Hence (27) yields
n

n

i=1

i=1

1X
1X
lim
IB ◦ Zi = lim
n→∞ n
n→∞ n

Z

IB dP = P (B)

and therefore, due to Strohriegl and Hable (2016, Theorem 2), the assertion.
23



Proof of Corollary 2.4:
Due to Example 2, the stochastic process is a Varadarajan process. Hable and Christmann
(2011, Theorem 3.2) ensures the continuity of the statistical operator S : M(Z) → H, P 7→
fL∗ ,P,λ for a ﬁxed value λ ∈ (0, ∞). Moreover Hable and Christmann (2011, Corollary
3.4) yields the continuity of the estimator Sn : Z n → H, Dn 7→ fL∗ ,Dn ,λ for every ﬁxed
λ ∈ (0, ∞). Hence for ﬁxed λ > 0 the bootstrap approximation of the SVM estimator is
qualitatively robust, for the given assumptions. Moreover the proof of Theorem 2.2, equation
(25), and the equivalence between between bounded Lipschitz metric and Prokhorov distance
yield: for every ε > 0 there is δ > 0 such that there is n0 ∈ N such that for all n ≥ n0 and
if dBL (Pn , Qn ) ≤ δ:
π(LPn∗ (Sn ), LQ∗n (Sn )) < ε almost surely.
Similarly to the proof of the qualitative robustness in Strohriegl and Hable (2016, Theorem
4) we get: for every ε > 0 there is nε , such that for all n ≥ nε :
kfL∗ ,Dn ,λn − fL∗ ,Dn ,λ0 kH ≤

ε
.
3

And the same argumentation as in the proof of the qualitative robustness of the SVM
estimator for the non-i.i.d. case in Strohriegl and Hable (2016, Theorem 4) for the cases
n0 ≤ n ≤ nε and n > nε yields the assertion.


3.2

Proofs of Section 2.2

Proof of Theorem 2.6:
Proof of Theorem 2.6: Let PN∗ , Q∗N ∈ M(Z N ) be the bootstrap approximations of the
true distribution PN and the contaminated distribution QN . First, the triangle inequality
yields:
dBL (LPn∗ (Sn ), LQ∗n (Sn ))

≤ dBL (LPn∗ (Sn ), LPn (Sn )) + dBL (LPn (Sn ), LQn (Sn )) + dBL (LQn (Sn ), LQ∗n (Sn )) .
{z
} |
|
{z
} |
{z
}
I

II

III

First, we regard the term in part II. Let σ(Zi ),Pi ∈ N, be the σ-algebra generated by Zi .
Due to the assumptions on the mixing process m>n α(σ(Z1 , . . . , Zi ), σ(Zi+m , . . .), PN ) =
O(n−γ ), i ∈ N, γ > 0, the sequence (α(σ(Z1 , . . . , Zi ), σ(Zi+m , . . .), µ))m∈N is a null sequence. Moreover it is bounded by the deﬁnition of the α-mixing coeﬃcient which, due to

24

the strong stationarity, does not depend on i. Therefore
n
n
n
n
1 XX
1 XX
α((Zi )i∈N , PN , i, j) = 2
α(σ(Zi ), σ(Zj ), PN )
n2
n
i=1 j=1

≤

2
n2

≤

2
n2

i=1 j=1
n
n X
X
i=1 j≥i
n
n X
X

α(σ(Zi ), σ(Zj ), PN )
α(σ(Z1 , . . . , Zi ), σ(Zj , . . .), PN )

i=1 j≥i

n n−i
2 XX
α(σ(Z1 , . . . , Zi ), σ(Zi+ℓ , . . .), PN )
= 2
n
i=1 ℓ=0

stationarity

≤

n

2X
α(σ(Z1 , . . . , Zi ), σ(Zi+ℓ , . . .), PN ), i ∈ N
n
ℓ=0

−→ 0, n → ∞.

Hence, the process is weakly α-bi-mixing with respect to PN , see Deﬁnition 2.5. Due to the
stationarity assumption,
the process (Zi )i∈N is additionally asymptotically mean stationary,
P
that is limn→∞ n1 ni=1 EIB ◦Zi = P (B) for all B ∈ A for a probability measure P . Therefore
the process satisﬁes the WLLNE, see Steinwart et al. (2009, Proposition 3.2), and therefore
is a weak Varadarajan process, see Strohriegl and Hable (2016, Theorem 2).
As the process is assumed to be a Varadarajan process and due to the assumptions on the sequence of estimators (Sn )n∈N , qualitative robustness of (Sn )n∈N is ensured by Strohriegl and Hable
(2016, Theorem 1). Together with the equivalence between the Prohorov metric and the
bounded Lipschitz metric for Polish spaces, see Huber (1981, Chapter 2, Corollary 4.3), it
follows:
For every ε > 0 there is δ > 0 such that for all n ∈ N and for all Qn ∈ M(Z n ) we have:
dBL (Pn , Qn ) < δ

⇒

ε
dBL (LPn (Sn ), LQn (Sn )) < .
3

This implies

ε
E [dBL (LPn (Sn ), LQn (Sn ))] < .
3
Hence the convergence of the term in part II is shown.

(28)

To prove the convergence of the term in part I, consider the distribution PN ∈ M(Z N )
and let PN∗ be the bootstrap approximation of PN , via the blockwise bootstrap. Deﬁne, for
n ∈ N, the random variables
Wn : Z N → Z n , Wn = (Z1 , . . . , Zn ), zN 7→ wn = (z1 , . . . , zn ), and
Wn′ : Z N → Z n , Wn′ = (Z1′ , . . . , Zn′ ), zN 7→ wn′ ,
such that Wn (PN ) = Pn and Wn′ (PN∗ ) = Pn∗ .
Moreover denote the bootstrap sample by Wn∗ : Z N → Z n , Wn∗ := (Z1∗ , . . . , Zn∗ ), zN 7→ wn∗ ,
25

and the distribution
of Wn∗ by P n . The blockwise bootstrap approximation of Pm , m ∈ N,
1 Pn
∗
m
is Pm = ⊗j=1 n i=1 δZi∗ , m ∈ N. Note that the sample Z1∗ , . . . , Zn∗ depends and on the
blocklength b(n) and on the number of blocks ℓ(n).
∗ , and W′ by K ∈ M(Z N × Z N × Z N ).
Further denote the joint distribution of WN , WN
N
N
Then, KN has marginal distributions KN (B1 × Z N × Z N ) = PN (B1 ) for all B1 ∈ B ⊗N ,
KN (Z N × B2 × Z N ) = P N (B2 ) for all B2 ∈ B ⊗N , and KN (Z N × Z N × B3 ) = PN∗ (B3 ) for all
B3 ∈ B ⊗N .
Then,

LPn (Sn ) = Sn (Pn ) = Sn ◦ Wn (PN ) and LPn∗ (Sn ) = Sn (Pn∗ ) = Sn ◦ Wn′ (PN∗ )
and therefore
dBL (LPn∗ (Sn ), LPn (Sn )) = dBL (L(Sn ◦ Wn′ ), L(Sn ◦ Wn )).
By assumption we have 0 ≤ zi ≤ 1, i ∈ N. Hence Zi (zN ) = zi ∈ [0, 1], i. e. Z = [0, 1], which
is a totally bounded metric space. Therefore the set BL1 ([0, 1]) is a uniform GlivenkoCantelli class, due to Dudley et al. (1991, Proposition 12). Similar to part I of the proof of
Theorem 2.2, the blockwise bootstrap structure and the Glivenko-Cantelli property yield:

lim

n→∞ P

sup
∗ ∈M(Z)
wn

PN∗



N

zN ∈ Z | sup dBL (PWm
′ (z ) , Pw∗ ) > η
N
n
m≥n



= 0.

Respectively, for ﬁxed ε > 0, for every δ0 > 0 there is n1 ∈ N such that for all n ≥ n1 and
all Pwn∗ ∈ M(Z):


ε
δ0
∗
′
n
(29)
≥1− .
Pn
wn ∈ Z | dBL (Pwn′ , Pwn∗ ) ≤
2
6
P
P
Regard the process Gn (t) = √1n ni=1 I{Zi∗ ≤t} − √1n ni=1 I{Zi ≤t} , t ∈ R. Due to the assumptions on the process and on the moving block bootstrap, Theorem 2.3 in Peligrad (1998)
yields the almost sure convergence in distribution to a Brownian bridge G:
n

n

i=1

i=1

1 X
1 X
√
I{Zi∗ ≤t} − √
I{Zi ≤t} −→D G(t),
n
n

t∈R

(30)

almost surely with respect to PN , n → ∞, in the Skorohod topology on D[0, 1]. Here −→D
indicates convergence in distribution and D[0, 1] denotes the space of cadlag functions on
[0, 1], for details see for example Billingsley (1999, p. 121).
This is equivalent to
n

n

i=1

i=1

1 X
1 X
√
I{Zi∗ ≤t} − √
I{Zi ≤t} −→D G(t), almost surely with respect to PN , n → ∞,
n
n
26

for all continuity points t of G, see Billingsley (1999, (12.14), p. 124).
Multiplying by

√1
n

yields for any ﬁxed continuity point t ∈ R :

n

n

i=1

i=1

1X
1X
1
I{Zi∗ ≤t} −
I{Zi ≤t} − √ G(t) −→D 0 almost surely with respect to PN , n → ∞.
n
n
n
As convergence in distribution to a ﬁnite constant implies convergence in probability, see
for example van der Vaart (1998, Theorem 2.7(iii)), and as √1n G(t) → 0 in probability, for
all t ∈ R:
n

n

i=1

i=1

1X
1X
I{Zi∗ ≤t} −
I{Zi ≤t} −→P 0 almost surely with respect to PN , n → ∞,
n
n
for all continuity points t of G, where −→P denotes the convergence in probability.
Hence, Dudley (1989, Theorem 11.12) yields the convergence of the corresponding probability measures:
!
n
n
1X
1X
dBL
δZi∗ ,
δZi −→P 0 almost surely with respect to PN , n → ∞.
n
n
i=1

i=1

Respectively
dBL (PWn∗ , PWn ) −→P 0 almost surely with respect to PN , n → ∞.

Deﬁne the set Bn = wn ∈ Z n | dBL (PWn∗ , Pwn ) −→P 0, n → ∞ . Hence,
o
n
=1
Pn (Bn ) = PN zN ∈ Z N | Wn (zN ) ∈ Bn

and, for all wn ∈ Bn , there is n2,wn ∈ N such that for all n ≥ n2,wn ∈ N:


 δ0
ε
∗
n
< .
Pn
wn ∈ Z | dBL Pwn∗ , Pwn >
4
6

(31)

(32)

By assumption we have 0 ≤ zi ≤ 1, i ∈ N. Hence the space of probability measures
{Pwn | wn ∈ [0, 1]n } is a subset of M([0, 1]) and therefore tight, as [0,1] is a compact
space, see e. g. (Klenke, 2013, Example 13.28). Then Prohorov’s Theorem, see for example Billingsley (1999, Theorem 5.1) yields relative compactness of M([0, 1], dBL ) and in
particular the relative compactness of the set {Pwn | wn ∈ [0, 1]n }. As M([0, 1], dBL ) is
a complete space, see Dudley (1989, Theorem 11.5.5), relative compactness equals total

27

boundedness. That is, there exists a ﬁnite dense subset P̃ of {Pwn | wn ∈ [0, 1]n } such that
for all ρ > 0 and Pwn ∈ {Pwn | wn ∈ [0, 1]n } there is P̃ρ ∈ P̃ such that
(33)

dBL (P̃ρ , Pwn ) ≤ ρ.
The triangle inequality yields:





dBL Pwn∗ , Pwn ≤ dBL Pwn∗ , P̃ρ + dBL P̃ ρ , Pwn .

Deﬁne ρ = δ40 . Then (32) yields for every P̃ρ ∈ P̃ the existence of an integer n ≥ n2,P̃ ∈ N
such that, for all n ≥ n2,P̃ and all wn ∈ Bn :
Pn



 δ0
wn∗ ∈ Z n | dBL Pwn∗ , Pwn >
2

 δ 

 δ

0
0
∗
n
or dBL P̃ρ , Pwn >
wn ∈ Z | dBL Pwn∗ , P̃ρ >
≤ Pn
4
4

 δ  (32) ε

(33)
0
∗
n
≤ Pn
wn ∈ Z | dBL Pwn∗ , P̃ρ >
<
.
4
6

Hence, for all n ≥ n2 := maxP̃ ∈P̃ {n2,P̃ } and for all wn ∈ Bn , we have:
sup
Pwn ∈M(Z)

Pn



wn∗

n

∈ Z | dBL P

∗
wn

, P wn



δ0
>
2



ε
< .
6

(34)

Due to the uniform continuity of the operator S, for every ε > 0 there is δ0 > 0 such that
for all P, Q ∈ M(Z) :
dBL (P, Q) ≤ δ0

⇒

ε
dH (S(P ), S(Q)) ≤ .
3

(35)

Moreover, the triangle inequality yields:
dBL (Pwn′ , Pwn ) ≤ dBL (Pwn′ , Pwn∗ ) + dBL (Pwn∗ , Pwn ).

(36)

Again we use the relation between the Prohorov metric πdH and the Ky Fan metric, Dudley
(1989, Theorem 11.3.5):
πdH LPn∗ (Sn ), LPn (Sn )) = πdH (Sn ◦ Wn′ , Sn ◦ Wn )
o
o
n
n
∗
≤ ε̃
dH (Sn ◦ Wn′ , Sn ◦ Wn ) > ε̃, wN
∈ ZN
≤ inf ε̃ > 0 | KN


= inf ε̃ > 0 | (Wn , Wn∗ , Wn′ )(KN ) (wn , wn∗ , wn′ ) ∈ Z n × Z n × Z n |

dH (Sn (wn′ ), Sn (wn )) > ε̃, wn∗ ∈ Z n ≤ ε̃ .
28

Due to the deﬁnition of the statistical operator S, this is equivalent to

inf{ε̃ > 0 | (Wn , Wn∗ , Wn′ )(KN ) (wn , wn∗ , wn′ ) ∈ Z n × Z n × Z n |

dH (S(Pwn′ ), S(Pwn ) > ε̃, wn∗ ∈ Z n ≤ ε̃}.
Due to the uniform continuity of S, see (35), we obtain, for all n ≥ max{n1 , n2 } :
n
o
ε
(Wn , Wn∗ , Wn′ )(KN ) (wn , wn∗ , wn′ ) ∈ Z n × Z n × Z n | dH (S(Pwn′ ), S(Pwn )) > , wn∗ ∈ Z n
3
(35)


≤ (Wn , Wn∗ , Wn′ )(KN ) (wn , wn∗ , wn′ ) ∈ Z n × Z n × Z n | dBL (Pwn′ , Pwn ) > δ0 , wn∗ ∈ Z n

= (Wn , Wn∗ , Wn′ )(KN ) (wn , wn∗ , wn′ ) ∈ Z n × Z n × Z n |

{wn ∈
/ Bn , dBL (Pwn′ , Pwn ) > δ0 } or {wn ∈ Bn , dBL (Pwn′ , Pwn ) > δ0 }, wn∗ ∈ Z n

≤ (Wn , Wn∗ , Wn′ )(KN ) (wn , wn∗ , wn′ ) ∈ Z n × Z n × Z n |

wn ∈
/ Bn , dBL (Pwn′ , Pwn ) > δ0 , wn∗ ∈ Z n

+(Wn , Wn∗ , Wn′ )(KN ) (wn , wn∗ , wn′ ) ∈ Z n × Z n × Z n |

wn ∈ Bn , dBL (Pwn′ , Pwn ) > δ0 , wn∗ ∈ Z n

(31)
= (Wn , Wn∗ , Wn′ )(KN ) (wn , wn∗ , wn′ ) ∈ Z n × Z n × Z n |

wn ∈ Bn , dBL (Pwn′ , Pwn ) > δ0 , wn∗ ∈ Z n .
The triangle inequality, (36), then yields for all n ≥ max{n1 , n2 }:


(Wn , Wn∗ , Wn′ )(KN ) (wn , wn∗ , wn′ ) ∈ Z n × Z n × Z n | wn ∈ Bn , dBL (Pwn′ , Pwn ) > δ0 , wn∗ ∈ Z n
n
(36)
≤ (Wn , Wn∗ , Wn′ )(KN ) (wn , wn∗ , wn′ ) ∈ Z n × Z n × Z n | {wn ∈ Bn
δ0 o
δ0
∗
∗
′
and dBL (Pwn , Pwn ) > } or {wn ∈ Bn and dBL (Pwn , Pwn ) > }
2
2


δ
0
≤ Pn∗
wn′ ∈ Z n | wn ∈ Bn , dBL (Pwn′ , Pwn∗ ) >
2


δ0
∗
n
wn ∈ Z | wn ∈ Bn , dBL (Pwn∗ , Pwn ) >
+ Pn
2
(29),(32) ε
ε
ε
+
=
.
<
6 6
3
The equivalence between the Prohorov metric and the bounded Lipschitz metric on Polish
spaces, see Huber (1981, Chapter 2, Corollary 4.3), yields the existence of ñ1 such that for
every n ≥ ñ1 :
ε
dBL (LPn∗ (Sn ), LPn (Sn )) < .
3
And therefore

 ε

E dBL LPn∗ (Sn ), LPn (Sn ) < .
3
29

(37)

For the convergence of the term in part III the same argumentation as for part I can be
applied, as the assumptions on QN and Q∗N are the same as for PN and P∗N . In particular for
every ε > 0 there is ñ2 ∈ N such that for all n ≥ ñ2 :
 ε
dBL LQ∗n (Sn ), LQn (Sn ) < ,
3

respectively

 ε

E dBL LQ∗n (Sn ), LQn (Sn ) < .
3

(38)

Hence, (28), (37), and (38) yield, for all n ≥ max{ñ1 , ñ2 }:
 ε ε ε

E dBL LPn∗ (Sn ), LQ∗n (Sn ) < + + = ε.
3 3 3
As LPn∗ (Sn ) and LQ∗n (Sn ) are random variables itself we have, due to Huber (1981, Chapter
2 Theorem 4.2, (2)⇒(1)), for all n ≥ max{ñ1 , ñ2 }:

dBL L(LPn∗ (Sn )), L(LQ∗n (Sn )) < ε.
Hence, for all ε > 0 there is δ > 0 such that there is n0 = max{ñ1 , ñ2 } ∈ N such that, for
all n ≥ n0 :
dBL (Pn , Qn ) < δ ⇒ dBL (L(LPn∗ (Sn )), L(LQ∗n (Sn ))) < ε
and therefore the assertion.



Proof of Theorem 2.7:
Proof of Theorem 2.7: The proof follows the same lines as the proof of Theorem 2.6 and
therefore we only state the diﬀerent steps. Again we start with the triangle inequality:
dBL (LPn∗ (Sn ), LQ∗n (Sn ))

≤ dBL (LPn∗ (Sn ), LPn (Sn )) + dBL (LPn (Sn ), LQn (Sn )) + dBL (LQn (Sn ), LQ∗n (Sn )) .
{z
} |
{z
} |
{z
}
|
I

II

III

To proof the convergence of the term in part II, we need the weak Varadarajan property
of the stochastic process. Due to the deﬁnition α(σ(Z1 , . . . , Zi ), σ(Zi+ℓ , . . .), µ) ≤ 2 for all
ℓ ∈ N, i ∈ N, and obviously:
α(σ(Z1 , . . . , Zi ), σ(Zi+ℓ , . . .), PN ) ≤ ℓ + 1, ℓ > 0.

30

(39)

Hence, due to the strong stationarity of the stochastic process, we have:
n
n
n
n
1 XX
1 XX
α((Z
)
,
P
,
i,
j)
=
α(σ(Zi ), σ(Zj ), PN )
i i∈N
N
n2
n2
i=1 j=1

i=1 j=1

≤

n
n
2 XX
α(σ(Zi ), σ(Zj ), PN )
n2

≤

2
n2

=

n n−i
2 XX
α(σ(Z1 , . . . , Zi ), σ(Zi+ℓ , . . .), PN )
n2

i=1 j≥i
n
n X
X

α(σ(Z1 , . . . , Zi ), σ(Zj , . . .), PN )

i=1 j≥i

i=1 ℓ=0
n
stationarity 2 X

≤

=
(39)

≤
(6)

n

ℓ=0

α(σ(Z1 , . . . , Zi ), σ(Zi+ℓ , . . .), PN ), i ∈ N

n
1
1
2X
(α(σ(Z1 , . . . , Zi ), σ(Zi+ℓ , . . .), PN )) 2 (α(σ(Z1 , . . . , Zi ), σ(Zi+ℓ , . . .), PN )) 2 , i ∈ N
n
ℓ=0
n
X

2
n

ℓ=0

1

(ℓ + 1) (α(σ(Z1 , . . . , Zi ), σ(Zi+ℓ , . . .), PN )) 2 , i ∈ N

−→ 0, n → ∞.
Now, the same argumentation as in the proof of Theorem 2.6 yields the weak Varadarajan
property and therefore, for all ε > 0,
ε
E [dBL (LPn (Sn ), LQn (Sn ))] < .
3

(40)

Regarding the term in part I, we use a central limit theorem for the blockwise bootstrapped
empirical process by Bühlmann (1994, Corollary 1 and remark) to show its convergence.
Again, regard the distribution PN ∈ M(Z N ) and let PN∗ be the bootstrap approximation of
PN , via the blockwise bootstrap. Deﬁne, for all n ∈ N, the random variables
Wn : Z N → Z n , Wn = (Z1 , . . . , Zn ), zN 7→ wn , and
Wn′ : Z N → Z n , Wn′ = (Z1′ , . . . , Zn′ ), zN 7→ wn′ ,
such that Wn (PN ) = Pn and Wn′ (PN∗ ) = Pn∗ .
Moreover denote the bootstrap sample by Wn∗ : Z N → Z n , Wn∗ := (Z1∗ , . . . , Zn∗ ), zN 7→
∗ =
wn∗ , and
the distribution of Wn∗ by P n . The bootstrap approximation of Pm is Pm
P
n
1
m
∗
∗ , m ∈ N, by deﬁnition of the bootstrap procedure. Note that
⊗m
j=1 n
i=1 δZi = ⊗j=1 PWn
∗
∗
the sample Z1 , . . . , Zn depends and on the blocklength b(n) and on the number of blocks
ℓ(n).
∗ , and W′ by K ∈ M(Z N × Z N × Z N ).
Further denote the joint distribution of WN , WN
N
N
Then, KN has marginal distributions KN (B1 × Z N × Z N ) = PN (B1 ) for all B1 ∈ B ⊗N ,

31

KN (Z N × B2 × Z N ) = P N (B2 ) for all B2 ∈ B ⊗N , and KN (Z N × Z N × B3 ) = PN∗ (B3 ) for all
B3 ∈ B ⊗N .
Then,
LPn (Sn ) = Sn (Pn ) = Sn ◦ Wn (PN ) and LPn∗ (Sn ) = Sn (Pn∗ ) = Sn ◦ Wn′ (PN∗ )
and therefore
dBL (LPn∗ (Sn ), LPn (Sn )) = dBL (L(Sn ◦ Wn′ ), L(Sn ◦ Wn )).
As Z = [0, 1]d is compact, it is in particular totally bounded. Hence the set BL1 (Z, dZ )
is a uniform Glivenko-Cantelli class, due to Dudley et al. (1991, Proposition 12). Similar
to part I of the proof of Theorem 2.6, the bootstrap structure and the Glivenko-Cantelli
property given above yield for arbitrary, but ﬁxed ε > 0:
for every δ0 > 0 there is n0 ∈ N such that, for all n ≥ n0 and all Pwn∗ ∈ M(Z),


δ0
ε
∗
n
′
Pn
wn ∈ Z | dBL (Pwn′ , Pwn∗ ) ≤
≥1− .
2
6
Now, regard the empirical process of (Z1 , . . . , Zn ). Set t = (t1 , . . . , td ) ∈ Rd . Moreover
t < b means ti < bi for all i ∈ {1, . . . , d}. Hence we can deﬁne the empirical process and
the blockwise bootstrapped empirical process by
n

1X
I{Zi ≤t}
n

n

and

i=1

1X
I{Zi∗ ≤t} .
n
i=1

P
P
Regard the process Gn (t) = √1n ni=1 I{Zi∗ ≤t} − √1n ni=1 I{Zi ≤t} , t ∈ [0, 1]d . Now, due to the
assumptions on the stochastic process and on the moving block bootstrap, Bühlmann (1994,
Corollary 1 and remark) yields the almost sure convergence in distribution to a Gaussian
process G:
n
n
1 X
1 X
∗
√
I{Zi ≤t} − √
I{Zi ≤t} −→D G(t), t ∈ [0, 1]d ,
n
n
i=1

i=1

almost surely with respect to PN , n → ∞, in the (extended) Skorohod topology on D d ([0, 1]).
The space D d ([0, 1]) is a generalization of the space of cadlag functions on [0, 1], see
Billingsley (1999, Chapter 12), and consists of functions f : [0, 1]d → R. A detailed description of this space and the extended Skorohod topology can be found in Straf (1972,
1969) and Bickel and Wichura (1971). The deﬁnition of the space D d ([0, 1]) can, for example, be found in Bickel and Wichura (1971, Chapter 3).
Straf (1972, Lemma 5.4) yields, that the above convergence in the Skorohod topology is
equivalent to the convergence for all continuity points t of G. Hence,
n

n

i=1

i=1

1 X
1 X
√
I{Zi∗ ≤t} − √
I{Zi ≤t} −→D G(t) almost surely with respect to PN , n → ∞,
n
n
32

for all continuity points t of G.
Multiplying by

√1
n

yields, for every continuity point t of G,

n

n

i=1

i=1

1X
1X
1
I{Zi∗ ≤t} −
I{Zi ≤t} − √ G(t) −→D 0 almost surely with respect to PN , n → ∞.
n
n
n
As convergence in distribution to a constant implies convergence in probability, see e. g.
van der Vaart (1998, Theorem 2.7(iii)) and as √1n G(t) converges in probability to 0, for all
ﬁxed continuity points t ∈ [0, 1]d of G:
n

n

i=1

i=1

1X
1X
I{Zi∗ ≤t} −
I{Zi ≤t} −→P 0 almost surely with respect to PN , n → ∞.
n
n
This yields the convergence of the corresponding probability measures, see for example
Billingsley (1995, Chapter 29) for a theory on Rd :
dBL (

n

n

i=1

i=1

1X
1X
δZi∗ ,
δZi ) −→P 0 almost surely with respect to PN , n → ∞,
n
n

respectively
dBL (PWn∗ , PWn ) −→P 0 almost surely with respect to PN , n → ∞.
As the space [0, 1]d is compact, we can use an argumentation similar to the proof of Theorem
2.6. Then, for every ε > 0, there is n1 ∈ N such that for all n ≥ n1
 ε
dBL LPn∗ (Sn ), LPn (Sn ) < ,
3
respectively,

 ε

E dBL LPn∗ (Sn ), LPn (Sn ) < .
(41)
3
The convergence of the term in part III follows simultaneously to part I for the distributions
QN and Q∗N . Hence, for every ε > 0, there is n2 ∈ N such that for all n ≥ n2
 ε

(42)
E dBL LQ∗n (Sn ), LQn (Sn ) < .
3
The combination of (40), (41), and (42) yields for all n ≥ max{n1 , n2 }:
 ε ε ε

E dBL LPn∗ (Sn ), LQ∗ (Sn ) < + + = ε.
3 3 3

As LPn∗ (Sn ) and LQ∗n (Sn ) are random variables itself we have, due to Huber (1981, Chapter
2, Theorem 4.2, (2)⇒(1)), for all n ≥ max{n1 , n2 } :

dBL L(LPn∗ (Sn )), L(LQ∗n (Sn )) < ε.
33

Hence, for all ε > 0 there is δ > 0 such that there is n0 = max{n1 , n2 } ∈ N such that for all
n ≥ n0 :
dBL (Pn , Qn ) < δ ⇒ dBL (L(LPn∗ (Sn )), L(LQ∗n (Sn ))) < ε.
This yields the assertion.



References
E. Beutner and H. Zähle. Functional delta-method for the bootstrap of quasi-Hadamard
diﬀerentiable functionals. Electron. J. Stat., 10, 2016.
P. J. Bickel and M. J. Wichura. Convergence criteria for multiparameter stochastic processes
and some applications. Ann. Math. Statist., 42:1656–1670, 1971.
P. Billingsley. Probability and measure. Wiley Series in Probability and Mathematical
Statistics. John Wiley & Sons, Inc., New York, third edition, 1995.
P. Billingsley. Convergence of probability measures. Wiley Series in Probability and Statistics: Probability and Statistics. John Wiley & Sons, Inc., New York, second edition,
1999.
G. Boente, R. Fraiman, and V. J. Yohai. Qualitative robustness for general stochastic
processes. Technical report, Department of Statistics, University of Washington, 1982.
G. Boente, R. Fraiman, and V. J. Yohai. Qualitative robustness for stochastic processes.
The Annals of Statistics, 15(3):1293–1312, 1987.
R. C. Bradley. Basic properties of strong mixing conditions. A survey and some open
questions. Probab. Surv., 2:107–144, 2005.
R. C. Bradley. Introduction to strong mixing conditions. Vol. 1. Kendrick Press, Heber City,
UT, 2007a.
R. C. Bradley. Introduction to strong mixing conditions. Vol. 2. Kendrick Press, Heber City,
UT, 2007b.
R. C. Bradley. Introduction to strong mixing conditions. Vol. 3. Kendrick Press, Heber City,
UT, 2007c.
P. Bühlmann. Blockwise bootstrapped empirical process for stationary sequences. Ann.
Statist., 22(2):995–1012, 1994.
P. Bühlmann. The blockwise bootstrap for general empirical processes of stationary sequences. Stochastic Process. Appl., 58(2):247–265, 1995.
O. Bustos. On qualitative robustness for general processes. unpublished manuscript, 1980.

34

A. Christmann, M. Salibian-Barrera, and S. Van Aelst. On the stability of bootstrap estimators. arXiv preprint arXiv:1111.1876, 2011.
A. Christmann, M. Salibián-Barrera, and S. Van Aelst. Qualitative robustness of bootstrap
approximations for kernel based methods. In Robustness and complex data structures,
pages 263–278. Springer, Heidelberg, 2013.
D. D. Cox. metrics on stochastic processes and qualitative robustness. Technical report,
Department of Statistics, University of Washington, 1981.
A. Cuevas and J. Romo. On robustness properties of bootstrap approximations. J. Statist.
Plann. Inference, 37(2):181–191, 1993.
Z. Denkowski, S. Migórski, and N. S. Papageorgiou. An introduction to nonlinear analysis:
applications. Kluwer Academic Publishers, Boston, MA, 2003.
P. Doukhan. Mixing. Springer, New York, 1994.
R. M. Dudley. Real Analysis and Probability. Chapman&Hall, New York, 1989.
R. M. Dudley. Uniform central limit theorems, volume 63 of Cambridge Studies in Advanced
Mathematics. Cambridge University Press, Cambridge, 2014.
R. M. Dudley, E. Giné, and J. Zinn. Uniform and universal Glivenko-Cantelli classes. J.
Theoret. Probab., 4(3):485–510, 1991.
B. Efron. Bootstrap methods: another look at the jackknife. Ann. Statist., 7(1):1–26, 1979.
B. Efron and R. J. Tibshirani. An introduction to the bootstrap, volume 57 of Monographs
on Statistics and Applied Probability. Chapman and Hall, New York, 1993.
R. Hable and A. Christmann. On qualitative robustness of support vector machines. Journal
of Multivariate Analysis, 102:993–1007, 2011.
F. R. Hampel. Contributions to the theory of robust estimation. PhD thesis, Univ. California,
Berkeley, 1968.
F. R. Hampel. A general qualitative deﬁnition of robustness. Annals of Mathematical
Statistics, 42:1887–1896, 1971.
J. Hoﬀmann-Jørgensen. Probability with a view toward statistics. Vol. I. Chapman & Hall
Probability Series. Chapman & Hall, New York, 1994.
P. J. Huber. Robust statistics. John Wiley & Sons Inc., New York, 1981.
J. Jurečková and J. Picek. Robust statistical methods with R. Chapman & Hall/CRC, Boca
Raton, FL, 2006.
A. Klenke. Probability theory: a comprehensive course. Springer Science & Business Media,
2013.
35

V. Krätschmer, A. Schied, and H. Zähle. Domains of weak continuity of statistical functionals with a view toward robust statistics. J. Multivariate Anal., 158:1–19, 2017.
H. R. Künsch. The jackknife and the bootstrap for general stationary observations. Ann.
Statist., 17(3):1217–1241, 1989.
S. N. Lahiri. Resampling methods for dependent data. Springer Series in Statistics. Springer,
New York, 2003.
R. Y. Liu and K. Singh. Moving blocks jackknife and bootstrap capture weak dependence.
In Exploring the limits of bootstrap (East Lansing, MI, 1990), Wiley Ser. Probab. Math.
Statist. Probab. Math. Statist., pages 225–248. Wiley, New York, 1992.
R. A. Maronna, R. D. Martin, and V. J. Yohai. Robust statistics. Wiley Series in Probability
and Statistics. John Wiley & Sons Ltd., Chichester, 2006.
U. V. Naik-Nimbalkar and M. B. Rajarshi. Validity of blockwise bootstrap for empirical
processes with stationary observations. Ann. Statist., 22(2):980–994, 1994.
P. Papantoni-Kazakos and R. M. Gray. Robustness of estimators on stationary observations.
The Annals of Probability, 7(6):989–1002, 1979.
K. R. Parthasarathy. Probability measures on metric spaces, volume 352. American Mathematical Soc., 1967.
M. Peligrad. On the blockwise bootstrap for empirical processes for stationary sequences.
Ann. Probab., 26(2):877–901, 1998.
D. N. Politis and J. P. Romano. A circular block-resampling procedure for stationary data.
In Exploring the limits of bootstrap (East Lansing, MI, 1990), Wiley Ser. Probab. Math.
Statist. Probab. Math. Statist., pages 263–270. 1990.
D. Radulović. The bootstrap for empirical processes based on stationary observations.
Stochastic Process. Appl., 65, 1996.
M. Rosenblatt. A central limit theorem and a strong mixing condition. Proc. Nat. Acad.
Sci. U. S. A., 42:43–47, 1956.
B. Schölkopf and A. J. Smola. Learning with Kernels. Massachusetts Institute of Technology,
Cambridge, 2002.
Q. M. Shao and H. Yu. Bootstrapping the sample means for stationary mixing sequences.
Stochastic Process. Appl., 48(1):175–190, 1993.
K. Singh. On the asymptotic accuracy of Efron’s bootstrap. Ann. Statist., 9(6):1187–1195,
1981.
I. Steinwart and A. Christmann. Support vector machines. Information Science and Statistics. Springer, New York, 2008.
36

I. Steinwart, D. Hush, and C. Scovel. Learning from dependent observations. Journal of
Multivariate Analysis, 100:175–194, 2009.
M. L. Straf. A general skorohod space, 1969.
M. L. Straf. Weak convergence of stochastic processes with several parameters. pages
187–221, 1972.
K. Strohriegl and R. Hable. On qualitative robustness for stochastic processes. Metrika,
pages 895–917, 2016.
A. W. van der Vaart. Asymptotic statistics, volume 3 of Cambridge Series in Statistical and
Probabilistic Mathematics. Cambridge University Press, Cambridge, 1998.
H. Zähle. Qualitative robustness of statistical functionals under strong mixing. Bernoulli,
21(3):1412–1434, 2015.
H. Zähle. A deﬁnition of qualitative robustness for general point estimators, and examples.
Journal of Multivariate Analysis, 143:12–31, 2016.

37

