ESAIM: Probability and Statistics

Will be set by the publisher

arXiv:1412.7103v4 [] 7 Jul 2016

URL: http://www.emath.fr/ps/

ADAPTIVE CONFIDENCE BANDS FOR MARKOV CHAINS AND
DIFFUSIONS: ESTIMATING THE INVARIANT MEASURE AND THE DRIFT ∗

Jakob Söhl 1 and Mathias Trabs 2
Abstract. As a starting point we prove a functional central limit theorem for estimators of the
invariant measure of a geometrically ergodic Harris-recurrent Markov chain in a multi-scale space. This
allows to construct confidence bands for the invariant density with optimal (up to undersmoothing) L∞ diameter by using wavelet projection estimators. In addition our setting applies to the drift estimation
of diffusions observed discretely with fixed observation distance. We prove a functional central limit
theorem for estimators of the drift function and finally construct adaptive confidence bands for the
drift by using a completely data-driven estimator.

1991 Mathematics Subject Classification. Primary 62G15; secondary 60F05, 60J05, 60J60, 62M05.
The dates will be set by the publisher.

Introduction
Diffusion processes are prototypical examples of the theory of stochastic differential equations as well as of
continuous time Markov processes. At the same time diffusions are widely used in applications, for instance, to
model molecular movements, climate data or in econometrics. Focusing on Langevin diffusions, we will consider
the solution of the stochastic differential equation
dXt = b(Xt )dt + σdWt ,

t > 0,

with unknown drift function b : R → R, a volatility parameter σ > 0 and with a Brownian motion W = {Wt :
t > 0}. The problem of statistical estimation based on discrete observations from this model is embedded
into the framework of geometrically ergodic Harris-recurrent Markov chains. We study the estimation of the
invariant density of such Markov chains. The drift function b depends nonlinearly on the invariant density µ so
that the two estimation problems of b and µ are closely related. We prove functional central limit theorems for
Keywords and phrases: Adaptive confidence bands, diffusion, drift estimation, ergodic Markov chain, stationary density, Lepski’s
method, functional central limit theorem
The authors acknowledge intensive and very helpful discussions with Richard Nickl. J.S. thanks the European Research
Council (ERC) for support under Grant No. 647812. M.T. is grateful to the Statistical Laboratory of the University of
Cambridge for its hospitality during a visit from February to March 2014, where this research was initiated, and to the Deutsche
Forschungsgemeinschaft (DFG) for the research fellowship TR 1349/1-1. Part of the paper was carried out while M.T. was
employed at the Humboldt-Universität zu Berlin.
1 Statistical Laboratory, Department of Pure Mathematics and Mathematical Statistics, University of Cambridge, CB3 0WB
Cambridge, UK. Email: j.soehl@statslab.cam.ac.uk
2 Department of Mathematics, University of Hamburg, Bundesstraße 55, 20146 Hamburg, Germany. Email: mathias.trabs@unihamburg.de
∗

c EDP Sciences, SMAI 1999

2

TITLE WILL BE SET BY THE PUBLISHER

estimators of both b and µ in multi-scale spaces. This allows the construction of confidence bands for µ. Owing
to the nonlinear dependence the construction of confidence bands for b is more involved. In this more difficult
situation and by using a self-similarity assumption we make the additional step of constructing confidence bands
for b that shrink at a rate adapting to the unknown smoothness.
Estimating the invariant density of a Markov process has been of interest for a long time. An early treatment
is given by Roussas [39], who considered kernel estimators and showed consistency and asymptotic normality of
the estimators under the strong Doeblin condition. Rosenblatt [38] analysed kernel estimators under the weaker
condition G2 on the Markov chain. More general δ-sequences were used for the estimation by Castellana and
Leadbetter [6], who prove pointwise consistency and under strong mixing assumptions asymptotic normality.
Yakowitz [47] shows asymptotic normality of kernel density estimators for the invariant density of Markov chains
without using assumptions on the rates of mixing parameter sequences. Adaptive estimation was considered by
Lacour [29], who estimates the invariant density and the transition density of Markov chains by model selection
and proves that the estimators attain the minimax convergence rate under L2 -loss. For stationary processes
Schmisser [40] estimates the derivatives of the invariant density by model selection, derives the convergence rates
of the estimators and pays special attention to the case of discretely observed diffusion processes. We see that
asymptotic normality has been widely considered in the nonparametric estimation of invariant densities and
thus implicitly also confidence intervals. However, we are not aware of any extensions of the pointwise results to
uniform confidence bands for invariant densities, which are, for instance, necessary to construct goodness-of-fit
tests of the Kolmogorov–Smirnov type.
The statistical properties of the diffusion model depend crucially on the observation scheme. If the whole path
(Xt )06t6T is observed for some time horizon T > 0, we speak of continuous observations. The case of discrete
observations (Xk∆ )k=0,...,n−1 with observation distance ∆ > 0 is distinguished into high-frequency observations,
i.e. ∆ ↓ 0, and low-frequency observations, where ∆ > 0 is fixed. While in the first two settings path properties
of the process can be used, statistical inference for low-frequency observations has to rely on the Markovian
structure of the observations. A review on parametric estimation in diffusion models is given by Kutoyants [28]
and Aı̈t-Sahalia [2]. Nonparametric results are summarized in Gobet et al. [20], where also estimators based
on low-frequency observations are introduced and analysed. These low-frequency estimators rely on a spectral
identification of diffusion coefficients which have been introduced by Hansen and Scheinkman [22] and Hansen
et al. [23]. On the same observation scheme Kristensen [27] studies a pseudo-maximum likelihood approach
in a semiparametric model. Nonparametric estimation based on random sampling times of the diffusion has
been studied in Chorowski and Trabs [12]. While we pursue a frequentist approach, the Bayesian approach is
also very attractive. Based on low-frequency observations van der Meulen and van Zanten [44] have proved
consistency of the Bayesian method and Nickl and Söhl [36] showed posterior contraction rates.
As usual, nonparametric estimators depend on some tuning parameters, such as the bandwidth for classical
kernel estimators. Choosing these parameters in a data-driven way, Spokoiny [41] initiated adaptive drift
estimation in the diffusion model based on continuous observations. This was further developed by Dalalyan
[14] and Löcherbach et al. [31]. Based on high-frequency observations, adaptive estimation was studied by
Hoffmann [25] as well as Comte et al. [13]. In the low-frequency case the question of adaptive estimation has
been studied by Chorowski and Trabs [12]. In this work we go one step further not only constructing a (rate
optimal) adaptive estimator for the drift, but constructing adaptive confidence bands.
Statistical applications require tests and confidence statements. Negri and Nishiyama [35] as well as Masuda
et al. [33] have constructed goodness-of-fit tests for diffusions based on high-frequency observations. Low [32]
has shown that even in a simple density estimation problem no confidence bands exist which are honest and
adaptive at the same time. Circumventing this negative result by a “self-similarity” condition, Giné and Nickl
[18] have constructed honest and adaptive confidence bands for density estimation. Hoffmann and Nickl [26]
have further studied necessary and sufficient conditions for the existence of adaptive confidence bands and the
“self-similarity” condition has led to several recent papers on adaptive confidence bands, notably Chernozhukov
et al. [11] and Szabó et al. [42]. The present paper extends the theory of adaptive confidence bands beyond the

TITLE WILL BE SET BY THE PUBLISHER

3

classical nonparametric models of density estimation, white noise regression and the Gaussian sequence model
which have been treated in the above papers.
In order to derive confidence bands, we first have to establish a uniform central limit theorem. The empirical
measure of the observations X0 , . . . , X(n−1)∆ is the canonical estimator for the invariant measure of a Markov
chain or diffusion. Considering a wavelet projection estimator, we obtain a smoothed version of the empirical
measure, which is subsequently used to estimate the drift function in the case of diffusions. Thus a natural
starting point is a functional central limit theorem for the invariant measure. Since our observations are not
independent, the standard empirical process theory does not apply. Instead we have to use the Markov structure
of the chain (Xk∆ )k . In the continuous time analogue the Donsker theorem for diffusion processes has been
studied by van der Vaart and van Zanten [45]. In the case of low-frequency observations, the estimation problem
is ill-posed and we have nonparametric convergence rates under the uniform loss. For the asymptotic behaviour
of the estimation error in the uniform norm we would expect a Gumbel distribution as shown by Giné and Nickl
[18] in the density estimation case using extreme value theory. Recent papers by Castillo and Nickl [7, 8] show
that we can hope for parametric rates and an asymptotic normal distribution if we consider instead a weaker
norm for the loss. More precisely, the estimation error can be measured in a multi-scale space where the wavelet
coefficients are down-weighted appropriately. The resulting norm corresponds to a negative Hölder norm.
Following this approach and relying on a concentration inequality by Adamczak and Bednorz [1], our first
result is a functional central limit theorem for rather general geometrically ergodic, Harris-recurrent Markov
chains. This could also be of interest for the theory on Markov chain Monte Carlo (MCMC) methods considering
that the central limit theorem measures the distance between a target integral and its approximation,
ˆ

f (z)µ(dz) and

R

n−1
1X
f (Zk ),
n
k=0

respectively, where (Zk )k is a Markov chain with invariant measure µ, cf. Geyer [16]. Nevertheless, our
focus is on the statistical point of view. The functional central limit theorem immediately yields non-adaptive
confidence bands and as in Castillo and Nickl [8] these have an L∞ -diameter shrinking with (almost) the optimal
nonparametric rate. This small deviation from the optimal rate corresponds to the usual undersmoothing in
the construction of nonparametric confidence sets.
Applying the results for general Markov chains to diffusion processes observed at low frequency, we obtain
a functional central limit theorem for estimators of the drift function. Inspired by Giné and Nickl [18], in a
last demanding step the smoothness of b and the corresponding size of the confidence band is estimated to find
adaptive confidence bands. The adaptive procedure relies on Lepski’s method. In order to make the construction
of adaptive confidence bands feasible, we impose a self-similarity assumption on the drift function.
This work is organized as follows: In Section 1 we study general Markov chains and prove the functional
central limit theorem and confidence bands under appropriate conditions on the chain. These results are applied
to diffusion processes in Section 2. The adaptive confidence bands for the drift estimator are constructed in
Section 3. Some proofs are postponed to the last two sections.

1. Confidence bands for the invariant probability density of Markov
processes
1.1. Preliminaries on Markov chains
We start with recalling some facts from the theory of Markov chains. For all basic definition and results we
refer to Meyn and Tweedie [34]. Let Z = (Zk ), k = 0, 1, . . . , be a time-homogeneous Markov chain with state
space (R, B(R)). To fix the notation, let Px and Pν denote the probability measure of the chain with initial
conditions Z0 = x ∈ R and Z0 ∼ ν, respectively. The corresponding expectations will be denoted by Ex and
Eν , the Markov chain transition kernel by P (x, A), x ∈ R, A ∈ B(R). The transition operator is defined by
(P f )(x) = Ex [f (Z1 )].

4

TITLE WILL BE SET BY THE PUBLISHER

From the general theory of Markov chains we know that for a Harris-recurrent Markov chain Z the existence
of a unique invariant probability measure µ is equivalent to the drift condition
P V (x) − V (x) 6 −1 + c1C (x)
for some petite set C, some c < ∞ and some non-negative function V , which is finite at some x0 ∈ R. If Z is
additionally aperiodic, then this drift condition is already equivalent to Z being ergodic
kP n (x, ·) − πkT V → 0,

as n → ∞,

for all x ∈ R,

denoting the total variation norm of a measure by k · kT V . If we impose a stronger drift condition, namely the
geometric drift towards C, we obtain even geometric ergodicity: For a ψ-irreducible and aperiodic Markov chain
Z satisfying
(P V )(x) − V (x) 6 −λV (x) + c1C (x),

for all x ∈ R,

(1)

for a petite set C, some λ > 0, c < ∞ and a function V : R → [1, ∞), it holds for some r > 1, R < ∞,
X
rn kP n (x, ·) − µkT V 6 RV (x), for all x ∈ R.
n>0

Note that ψ-irreducibility together with the geometric drift condition (1) implies already that Z is positive
Harris with invariant probability measure µ.
The geometric ergodicity yields the following central limit theorem, see Chen [10, Thm. II.4.1]. The weakest
form of ergodicity so that the central limit theorem holds is ergodicity of degree 2 which is slightly weaker than
the geometric ergodicity that we have assumed here.
Proposition 1.1. Let (Zk )k>0 be a geometrically ergodic Markov chain with arbitrary initial condition and
invariant probability measure µ, then there exists for every bounded function f = (f1 , . . . , fd ) : R → Rd a
symmetric, positive semidefinite matrix Σf = (Σfi ,fj )i,j=1,...,d such that
n−1/2

 n−1
X
k=0


d
f (Zk ) − nEµ [f (Z0 )] −→ N (0, Σf ),

as n → ∞.

For i, j ∈ {1, . . . , d} the asymptotic covariances are given by
Σfi ,fj := lim n−1 Covµ
n→∞

 n−1
X
k=0

fi (Zk ),

n−1
X
k=0


fj (Zk )

(2)

∞



 X
Eµ (fi (Z0 ) − Eµ [fi ])(fj (Zk ) − Eµ [fj ])
= Eµ (fi (Z0 ) − Eµ [fi ])(fj (Z0 ) − Eµ [fj ]) +
k=1

+

∞
X

k=1



Eµ (fi (Zk ) − Eµ [fi ])(fj (Z0 ) − Eµ [fj ]) .

In order to lift this “pointwise” result to a functional central limit theorem, we will in addition need a
Pn−1
concentration
inequality for a preciser control on how the sum n−1 k=0 f (Zk ) deviates from the integral
´
f (z)µ(dz) for finite sample sizes. To this end, we strengthen the aperiodicity assumption to strong aperiodicity
[see 34, Prop. 5.4.5], that is there exists a set C ∈ B(R), a probability measure ν with ν(C) > 0 and a constant
δ > 0 such that
P (x, B) > δν(B),

for all x ∈ C, B ∈ B(R).

(3)

5

TITLE WILL BE SET BY THE PUBLISHER

Any set C satisfying this condition is called small set. Recall that any small set is a petite set.
Proposition 1.2 (Theorem 9 by Adamczak and Bednorz [1]). Let Z = (Zk )k>0 be a Harris recurrent, strongly
aperiodic Markov chain on (R, B(R)) with unique invariant measure µ. For some set C ∈ B(R) with µ(C) > 0
let Z satisfy the drift condition (1) and the small set condition (3).
Let f ∈ L2 (µ) be bounded. For any 0 < τ < 1 there are constants K, c2 depending only on δ, V , λ, c and τ
and a constant c1 depending additionally on the initial value x ∈ R such that for any t > 0
Px

 n−1
X
k=0

f (Zk ) − nEµ [f (Z0 )] > t


6 K exp −c1



t τ
kf k∞





+ K exp



−

c 2 t2
1/2

nΣf + t max(kf k∞ (log n)1/τ , Σf )


,

where Σf is given by (2) with d = 1.
As a last ingredient we need to bound the´ asymptotic variance Σf in Propositions 1.1 and 1.2 in terms of
kf k2L2 (µ) for the centred function f := f − f dµ. The geometric ergodicity only yields a bound O(kf k2∞ ).
Therefore, we require that the transition operator is a contraction in the sense that there exists some ρ ∈ (0, 1)
satisfying
ˆ
kP gkL2 (µ) 6 ρkgkL2(µ) for all g ∈ L2 (µ) with
g dµ = 0.
(4)
This property is also known as ρ-mixing. It corresponds to a Poincaré inequality [cf. 3, Thm. 1.3] and its
relation to drift conditions is analysed by Bakry et al. [3]. If (4) is fulfilled, the Cauchy–Schwarz inequality
yields
∞
∞
X
X

1+ρ
2
k
kf kL2 (µ) kP f kL2 (µ) 6 1 + 2
ρk kf k2L2 (µ) = 1−ρ
(5)
Σf = Σf 6 kf kL2 (µ) + 2
kf k2L2 (µ) .
k=1

k=1

1.2. A functional central limit theorem
The basic idea is to prove a functional central limit theorem for the invariant probability measure µ by
choosing an orthonormal basis, applying the pointwise central limit theorem to the basis functions (Proposition 1.1) and extending this result to finite linear combinations with the help of the concentration inequality
(Proposition 1.2). Provided µ has some regularity, the approximation error due to considering only a finite basis
expansion of µ will be negligible. Noting that it is straightforward to extend the results to any compact subset
of R, we focus on a central limit theorem on a bounded interval [a, b] with −∞ < a < b < ∞.
Let (ϕj0 ,l , ψj,k : j > j0 , l, k ∈ Z), for some j0 > 0, a scaling function ϕ and a wavelet function ψ, be a
regular compactly supported L2 -orthonormal wavelet basis of L2 (R). For the sake of clarity we throughout use
Daubechies’ wavelets of order N ∈ N, but any other compactly supported regular wavelet basis can be applied
as well. As a standing assumption we suppose that N is chosen large enough such that the Hölder regularity of ϕ
and ψ is larger than the regularity required for the invariant measure. The approximation spaces for resolution
levels J > j0 are defined as
VJ := span{ϕj0 ,l , ψj,k : j = j0 , . . . , J, l, k ∈ Z},
The projection onto VJ is denoted by πJ . Since j0 is fixed and to simplify the notation, we write ψ−1,l := ϕj0 ,l .
Using the first n ∈ N steps Z0 , Z1 , . . . , Zn−1 of a realisation of the chain, we define the empirical measure
µn :=

n−1
1X
δZk ,
n
k=0

6

TITLE WILL BE SET BY THE PUBLISHER

where δx denotes the Dirac measure at the point x ∈ R. The canonical projection wavelet estimator of µ is
given by
µ
bJ := πJ (µn ) =

X
l∈Z

µ
b−1,l ψ−1,l +

For any ψj,k Proposition 1.1 yields that
random variable

J X
X

j=j0 k∈Z

µ
bj,k ψj,k ,

µ
bj,k := hψj,k , µn i :=

ˆ

ψj,k dµn .

(6)

√
n(µn − µ)(ψj,k ) converges in distribution for n → ∞ to a Gaussian

Gµ (j, k) ∼ N (0, Σψj,k ) with covariances

Eµ [Gµ (j, k)Gµ (l, m)] = Σψj,k ,ψl,m .

(7)

Using the techniques from Castillo and Nickl [8], this pointwise convergence of µn can be extended to a uniform
central limit theorem on [a, b] for the projection estimator µ
bJ in the multi-scale sequence spaces which are
defined as follows: Noting that the Daubechies wavelets fulfil supp ϕ ⊆ [0, 2N − 1] and supp ψ ⊆ [−N + 1, N ],
cf. Härdle et al. [24, Chap. 7], the sets L := K−1 := {k ∈ Z : 2j0 a − 2N + 1 6 k 6 2j0 b} and Kj := {k ∈ Z :
2j a − N 6 k 6 2j b + N − 1} contain all indices of ϕj0 ,· and ψj,· , respectively, whose support intersects with the
interval [a, b]. For a monotonously increasing weighting sequence w = (wj )j=−1,j0 ,j0 +1,j0 +2,... with wj > 1 and
w−1 := 1 we define the multi-scale sequence spaces as
n
M := M(w) := x = (xjk ) : kxkM(w) :=

sup

max

j∈{−1,j0 ,j0 +1,... } k∈Kj

o
|xjk |
<∞ ,
wj

Since the Banach space M(w) is non-separable, we define the separable, closed subspace
o
n
|xjk |
=0 .
M0 := M0 (w) := x = (xjk ) : lim max
j→∞ k∈Kj wj
Let us assume that µ is absolutely continuous with respect to the Lebesgue measure and denote the density
likewise by µ. If the density is bounded on D = [a − 2−j0 (2N − 1), b + 2−j0 (2N − 1)], the orthonormality
and the support of (ψj,k ) and (5) yield Σψj,k = O(kµk∞ ). Standard estimates of the supremum of normal
random variables yield√that the maximum over the 2j variables Gµ (j, ·) of a resolution level j are of the order
√
maxk |Gµ (j, k)| = OP ( j), see (14) below. Since the cardinality of Kj is of the order 2j , a weighting wj = j
seems to be appropriate and indeed we conclude as Castillo and Nickl [8, Prop. 3]:
Lemma 1.3. Let µ admit a Lebesgue density which is bounded on D. Then Gµ from (7) satisfies E[kGµ kM(w) ] <
√
∞
√ for the weights wj = j. Moreover, L(Gµ ) is a tight Gaussian Borel probability measure in M0 (w) if
j/wj → 0.
Let us now summarise the assumptions on the Markov chain, which are needed to prove the functional central
limit theorem and for the construction of confidence bands. For any regularity s > 0, denoting the integer part
of s by [s], the Hölder space on a domain D is defined by
s

n

C (D) := f : D → R kf kC s :=

[s]
X

k=0

kf (k) k∞ + sup
x6=y

|f [s] (x) − f [s] (y)| o
.
|x − y|s−[s]

Assumption A. Let (Zk )k>0 be a Harris recurrent, strongly aperiodic Markov chain on (R, B(R)) with initial
condition Z0 = x. Let the invariant probability measure have a density µ in C s (D) for some s > 0 and some
sufficiently large set D ⊆ R containing [a, b]. Let the drift condition (1) and small set condition (3) be satisfied
for some C ∈ B(R) with µ(C) > 0. Further suppose that the transition operator is an L2 (µ)-contraction fulfilling
(4) with ρ ∈ (0, 1).

7

TITLE WILL BE SET BY THE PUBLISHER

Remark 1.4. As we have discussed above it suffices to verify that the chain (Zk )k>0 is ψ-irreducible and
satisfies (1) and (3) in order to conclude that the (Zk )k>0 is Harris recurrent, strongly aperiodic and has a
unique invariant probability measure.
Now we can show the functional central limit theorem for µ
bJ in the space M0 (w). Note that the natural
nonparametric choice Jn given by 2Jn ∼ n1/(2s+1) satisfies the conditions of the following theorem. Recall
that weak convergence of laws L(X) of random variables X on a metric space (S, d) can be metrised by the
bounded-Lipschitz metric
ˆ
βS (µ, ν) :=
sup
F (x)(µ(dx) − ν(dx)) with
F :kF kBL 61

S

kF kBL := sup |F (x)| +
x∈S

|F (x) − F (y)|
.
d(x, y)
x,y∈S:x6=y
sup

Theorem 1.5. Grant Assumption A and let w = (wj ) be increasing and satisfy
Jn ∈ N fulfil, for some τ ∈ (0, 1),
√ −Jn (2s+1)/2 −1
n2
wJn = o(1),

Then µ
bJn from (6) satisfies, for n → ∞,
√

√
j/wj → 0 as j → ∞. Let

(log n)2/τ n−1 2Jn Jn = O(1).

d

n(b
µJn − µ) −→ Gµ

in M0 (w).

Proof. We follow the strategy of [8, Thm. 1]. First we deal with the bias term. By the s-Hölder regularity of
µ we have [19, Definition (5.90) and Proposition 5.3.13]
sup 2j(2s+1)/2 |hψj,k , µi| < ∞
j,k

and thus by the assumption on Jn
kµ − πJn (µ)kM = sup max wj−1 |hψjk , µi| . sup wj−1 2−j(2s+1)/2 = o(n−1/2 ).
j>Jn k∈Kj

j>Jn

√
Defining νn := n(b
µJn − πJn (µ)), we decompose the stochastic error, for J < Jn to be specified later,
βM0 (L(νn ), L(Gµ )) 6 βM0 (L(νn ), L(νn ) ◦ πJ−1 ) + βM0 (L(νn ) ◦ πJ−1 , L(Gµ ) ◦ πJ−1 )
+ βM0 (L(Gµ ) ◦ πJ−1 , L(Gµ )).

(8)

In the sequel we will separately show that all three terms converge to zero. Let ε > 0. By definition of the
βM0 -norm we estimate the first term by
βM0 (L(νn ), L(νn ) ◦ πJ−1 ) =

sup
F :kF kBL 61



E F (νn ) − F (πJ (νn ))


 √
6 E k n(πJn − πJ )(µn − µ)kM
h
i
√
6 max (wj−1 j 1/2 )E max max j −1/2 |h n(µn − µ), ψj,k i| .
J<j6Jn

J<j6Jn k∈Kj

(9)

By the assumptions on w and due to the factor in front of the expectation, the above display can be bounded
by ε/3 if J is chosen large enough and provided that the expectation can be bounded by a constant independent
of J and n. To apply the concentration inequality in Proposition 1.2, note that Σψj,k = O(kµk∞ ) by (5) and

8

TITLE WILL BE SET BY THE PUBLISHER

√
√
√
jkψj,k k∞ = j2j/2 = O( n(log n)−1/τ ) for j 6 Jn . Hence, for any M > 0 large enough we obtain for
constants ci > 0, i = 1, 2, . . . ,
i
h
√
E max max j −1/2 |h n(µn − µ), ψj,k i|
J<j6Jn k∈Kj
ˆ ∞ 

√
6M+
P
max max j −1/2 |h n(µn − µ), ψj,k i| > u du
J<j6Jn k∈Kj
M
ˆ ∞ 
X
p 
√
6M+
P |h n(µn − µ), ψj,k i| > ju du
M

J<j6Jn ,k∈Kj

X

.M+

2j

.M+

∞

M

J<j6Jn

X

ˆ

2

J<j6Jn

j





exp − c1 (log n)j τ uτ + exp − c2 ju2 /(1 + u) du

 e−c3 (jM)τ log n
j τ log n

+

(10)

e−c4 jM 
j

τ

. M + e−c5 JM . M + 1,
where we have used in the next to last estimate that Jn . log n and thus j τ log n & j for all j 6 Jn .
To bound the second term in (8), we use Proposition 1.1 and the Cramér–Wold device to see that it is smaller
than ε/3 for fixed J and n sufficiently large. It remains to consider the third term in (8) which can be estimated
similarly to (9), using that E[supj maxk j −1/2 |Gµ (j, k)|] < ∞ by Lemma 1.3.


1.3. The construction of confidence bands
Using the multi-scale central limit theorem, we now construct confidence bands for the density of the invariant
probability measure. For some confidence level α ∈ (0, 1) the natural idea is to take
n
ζα o
ζα o n
= f : sup wj−1 |hf − µ
bJn , ψj,k i| < √ ,
Cn (ζα ) := f : kf − µ
bJn kM < √
n
n
j,k

where ζα is chosen such that P (kGµ kM < ζα ) > 1 − α. For this set the asymptotic coverage follows immediately
from Theorem 1.5. However, Cn (ζα ) is too large in terms of the L∞ ([a, b])-diameter

|Cn (ζα )|∞ := sup sup |f (x) − g(x)| : f, g ∈ Cn (ζα ) .
x∈[a,b]

To obtain the (nearly) optimal L∞ -diameter, we need to control the large resolution levels. As suggested by
Castillo and Nickl [8], we use a-priori knowledge of the regularity s to define

(11)
C n := C n (ζα , s, un ) := Cn (ζα ) ∩ f : kf kC s 6 un
for a sequence un → ∞.

Proposition 1.6. Grant Assumption A with s > 0 and let w = (wj ) be increasing and satisfy
α ∈ (0, 1) let ζα > 0 be such that P (kGµ kM > ζα ) 6 α and choose Jn := Jn (s) such that
2 Jn =

 n 1/(2s+1)
.
log n

√
Then the confidence set C n = C n (ζα , s, un ) from (11) with un := wJn / Jn satisfies
lim inf P (µ ∈ C n ) > 1 − α
n→∞

and

|C n |∞ = OP

 n −s/(2s+1) 
un .
log n

√
j/wj → 0. For

9

TITLE WILL BE SET BY THE PUBLISHER

Proof. Let us first verify lim inf n→∞ P (µ ∈ C n ) > 1 − α. Since µ ∈ C s (un ) for large enough n, Theorem 1.5
yields
√
µJn − µkM < ζα ) > P (kGµ kM < ζα ) > 1 − α.
lim inf P (µ ∈ C n ) = lim inf P ( nkb
n→∞

n→∞

To bound the diameter let f, g ∈ C n . Using kf − µ
bJn kM = OP (n−1/2 ) and f − g ∈ C s (2un ), we obtain
kf − gkL∞ ([a,b]) .
6

X

2j/2 max |hf − g, ψj,k i| +
k∈Kj

j6Jn

X

X

j>Jn

2j/2 max |hf − g, ψj,k i|
k∈Kj

bJn , ψj,k i| + max |hg − µ
bJn , ψj,k i|
2j/2 max |hf − µ
k∈Kj

j6Jn

+

X

j>Jn

k∈Kj

2−js 2j(s+1/2) max |hf − g, ψj,k i|
k∈Kj

bJn kM
6 kf − µ
bJn kM + kg − µ

 X

j6Jn

2j/2 wj + kf − gkC s



2
wJn + OP 2−Jn s un
= OP n




= OP n−1/2 2Jn /2 Jn1/2 un + OP 2−Jns un .




−1/2 Jn /2

Plugging in the choice of Jn , we finally have n−1/2



X

2−js

j>Jn

p
2Jn Jn . (n/ log n)−s/(2s+1) = 2−Jn s .

(12)


A multi-scale confidence band as in (11) allows for the construction of a classical L∞ -band on [a, b] around
−s/(2s+1)
un . As we can deduce from
µ
bJn as follows: Let us denote the almost optimal diameter by ρn := logn n
(12), there is a constant D > 0 such that we have kf − µ
bJn kL∞ ([a,b]) 6 Dρn for any f ∈ Cn . Hence, the band

Cen := f : [a, b] → R kf − µ
bJn kL∞ ([a,b]) 6 Dρn

contains C n which only improves the coverage. Consequently, Cen is an L∞ -confidence band with level α which
shrinks with almost optimal rate ρn . In addition, a multi-scale confidence band as in (11) allows for simultaneous
confidence intervals in all wavelet coefficients. This is especially useful for goodness-of-fit tests where the optimal
L∞ -diameter of C n is a measure of the power of the test.
In order to apply the confidence band (11) we need the regularity s of the invariant density and a critical
value ζα such that P (kGµ kM < ζα ) > 1 − α for α ∈ (0, 1). Adaptive confidence bands will be presented later
in the context of diffusions. So let us suppose for a moment that the regularity s is known. Then the problem
reduces to the construction of the critical value to which the remainder of this section is devoted.
A first observation is that if several independent copies of the diffusion are observed then one could calculate
for each copy an estimator µ
bJn and obtain estimators for the values ζα from the distribution of the estimators
µ
bJn around their joint mean. Since the assumption of many independent copies is not realistic we will not pursue
this further. Instead of the consistent estimation of the lowest possible ζα we restrict ourselves to estimating
an upper bound, which yields possibly more conservative confidence sets. By the concentration of Gaussian
measures we know for any κ > 0 that

2
P kGµ kM > E[kGµ kM ] + κ 6 e−κ /(2Σ) ,

where Σ := supj,k E[|Gµ (j, k)|2 ] = supj,k Σψj,k , see for example Ledoux [30, Thm. 7.1]. Hence, an upper bound
for ζα is given by
p
2Σ log α−1 + E[kGµ kM ].

10

TITLE WILL BE SET BY THE PUBLISHER

The expected value E[kGµ kM ] can be bounded as in Proposition 2 by Castillo and Nickl [8], depending on Σ
again. We obtain the following upper bound for ζα :
√
√
Lemma 1.7. Let j0 > 1 and w = (wj ) satisfy w−1 = j0 and inf j wj / j > 1, j > j0 , and define Σ :=
supj,k Σψj,k . Then P (kGµ kM > ζ α ) 6 α holds for
ζ α (Σ) :=

p
2 log α−1 + 2C +

32 −2j0
3C 2

√
Σ

(13)

with C := (supj>j0 (4 log |Kj | + 2 log 2)/j)1/2 .
Proof. The cardinality of Kj is denoted by |Kj |. Recall that a standard normal random variable Z satisfies
E[eZ

2

/4

]=

√
2

2
1
and P (Z > κ) 6 √ e−κ /2 , κ > 0.
κ 2π

1/2

For each j > j0 and κ = 2 supk Σψj,k Jensen’s inequality thus yields



2
2  1/2
E[max |Gµ (j, k)|] 6 κ log E emaxk |Gµ (j,k)| /κ
k
p
1/2
1/2
6 C Σj,
6 2 sup Σψj,k log |Kj | + 12 log 2

(14)

k

for the constant C := (supj (4 log |Kj |+2 log 2)/j)1/2 . Theorem 7.1 in [30] yields for all t, T such that t > Σ1/2 CT
and T > 1
 X

P kGµ kM > t 6
P | max Gµ (j, k) − E[max Gµ (j, k)]| > twj − E[max |Gµ (j, k)|]
k

j

k

k

X 
p 
P | max Gµ (j, k) − E[max Gµ (j, k)]| > (t − C) Σj
6
k

j

62

X

2

e−jt

k

(T −1)2 /(2ΣT 2 )

.

j

Recall that j = j0 , j0 , j0 + 1, j0 + 2, . . . in the above sum. Using Fubini’s theorem and the Gaussian tail bound,
we conclude
ˆ ∞
Xˆ ∞
2
2
E[kGµ kM ] 6 Σ1/2 CT +
P (kGµ kM > t)dt 6 Σ1/2 CT + 2Σ1/2
e−jt (1−1/T ) /2 dt
Σ1/2 CT

6Σ

1/2

j

CT

2
4Σ1/2 T 2−2(T −1) j0
2Σ1/2 T X −(2 log 2)j(T −1)2
1/2
e
6 Σ CT +
.
CT +
C(T − 1)2 j
C(T − 1)2 (1 − 2−2(T −1)2 )

Choosing the T = 2, we obtain E[kGµ kM ] 6 (2C +

32 −2j0
)Σ1/2 .
3C 2



From the above lemma we see that Σ is the key quantity for the construction of the critical values ζα . A
b n := (maxj6Jn ,k Σ
b ψj,k ), where Σ
b ψj,k are estimators of Σψj,k based on n observations.
natural estimator for Σ is Σ
Since Jn tends to infinity, the maximum over all j 6 Jn converges to the supremum over all j so that we are
b ψ we propose the initial monotone sequence
asymptotically estimating the right quantity. For the estimators Σ
j,k
estimators based on autocovariations by Geyer [16], which are consistent over-estimates, and this yields almost
surely
b n > Σ,
lim inf Σ
n→∞

11

TITLE WILL BE SET BY THE PUBLISHER

which suffices for our purposes.
The estimation of Σψj,k amounts to the estimation of the asymptotic variance Σf in (2) for a known function
f and this problem is studied in the MCMC-literature. In addition to the sequence estimators, Geyer [16]
discusses two other constructions together with their advantages and disadvantages. Robert [37] constructs
another estimator applying renewal theory, which is however difficult to calculate. A more recent estimator
using i.i.d. copies of the process X is given by Chauveau and Diebolt [9].
As an alternative to the above estimation of Σ in (13) an upper bound could be estimated as follows: Using (5)
we can bound Σ from above,
1+ρ
1+ρ
Σ 6 sup 1−ρ
kψ j,k kL2 (µ) 6 sup 1−ρ
kψj,k kL2 kµk∞ =
j,k

j,k

1+ρ
1−ρ kµk∞ ,

where we can plug in estimators for kµk∞ and ρ. Considering a wavelet ψj,k localised around the maximum of
µ we see that the second inequality should provide a good bound. To estimate kµk∞ a calculation along the
lines of the bound (12) shows that for µ ∈ C s (D) with Jn as in Proposition 1.6
kb
µJn − µkL∞ ([a,b]) = OP

 log n s/(2s+1)
n


un ,

(15)

√
where un = wJn / Jn . Provided the supremum of µ is attained in [a, b] or µ admits some positive global Hölder
regularity, we conclude that kµk∞ can be estimated by kb
µJn k∞ with the above rate and is in particular a
consistent estimator, which is all that is needed. For the estimation of ρ we observe that it is the second largest
eigenvalue of the transition operator P∆ . Gobet et al. [20] estimate this eigenvalue in a reflected diffusion model
by constructing first an empirical transition matrix for the transition operator restricted to a finite dimensional
space and then taking the second largest eigenvalue of the empirical transition matrix as an estimator for ρ,
there denoted by κ1 . They give a rate for their estimator, in particular the estimator is consistent.
Let us finally note that the estimation of ζα can be circumvented by a Bayesian approach as studied by
Castillo and Nickl [8] as well as Szabó et al. [42] in simpler statistical problems. The papers analyse Bayesian
credible sets in the density estimation model and in the white noise regression model as well as in the Gaussian
sequence model and show that they are frequentist confidence sets. Estimating the drift of a diffusion from
low-frequency observations is a more complicated statistical model. Consistency of the Bayesian approach in
this setting has been established by van der Meulen and van Zanten [44] and has been extended to the multidimensional case by Gugushvili and Spreij [21]. Recently Nickl and Söhl [36] have shown Bayesian posterior
contraction rates for scalar diffusions with unknown drift and unknown diffusion coefficient observed at low
frequency.

2. Application to diffusion processes
2.1. Estimation of the invariant density and its consequences
We now apply the results from the previous section to diffusion processes. At the same time we extend the
results from inference on the invariant probability measure to confidence bands for the drift function. Let us
consider the diffusion
dXt = b(Xt )dt + σdWt ,
t > 0, X0 = x,
(16)
with a Brownian motion Wt , an unknown drift function b : R → R, a volatility parameter σ > 0 and starting
point x ∈ R. We observe X at equidistant time points 0, ∆, 2∆, . . . , (n− 1)∆ for some fixed observation distance
∆ > 0 and sample size n → ∞. Our aim is inference on the drift b.
Underlying the sequence of observations (X∆k )k>0 is a Markov structure described by the transition operator
P∆ f (x) := E[f (X∆ )|X0 = x].

12

TITLE WILL BE SET BY THE PUBLISHER

The semi-group (Pt : t > 0) has the infinitesimal generator L on the space of twice continuously differentiable
functions given by
2
Lf (x) = Lb f (x) := b(x)f ′ (x) + σ2 f ′′ (x).
(17)
If there is an´ invariant density µ = µb , the operator L is symmetric with respect to the scalar product of
L2 (µ) = {f : |f |2 dµ < ∞}. We impose the following assumptions on the diffusion:

Assumption B. In model (16) let b be continuously differentiable and satisfy b ∈ C s (D) for s > 1 and a
sufficiently large set D ⊆ R containing the interval [a, b] for a < b. Let σ be in a fixed bounded interval away
from the origin. Suppose that b′ is bounded and that there are M, r > 0 such that
sign(x)b(x) 6 −r,

for all |x| > M.

More precisely, we will need D = [a−21−j0 (2N −1), b+21−j0 (2N −1)]. Due to the global Lipschitz continuity
and the assumptions on the drift, equation (16) has a unique strong solution. Moreover, Xt is a Markov process
with invariant probability density given by
ˆ x


−2
−2
µ(x) = C0 σ exp 2σ
b(y)dy , x ∈ R,
(18)
0

with normalization constant C0 > 0, cf. Bass [4, Chaps. 1,4]. The corresponding Markov chain Z with Zk = Xk∆
satisfies Assumption A from the previous section.
Proposition 2.1. If the diffusion process (16) satisfies Assumption B, then the Markov chain (Xk∆ )k>0 satisfies
Assumption A where µ ∈ C s+1 (D).

Proof. By a time-change argument we can set σ = 1 without loss of generality. Gihman and Skorohod [17,
Thm. 13.2] have given an´ explicit formula for the transition density p∆ (x, y) with respect to the Lebesgue
measure, i.e., P∆ (x, B) = B p∆ (x, y)dy for all B ∈ B(R). In particular, p∆ (x, y) is strictly positive and thus Z
is ψ-irreducible, where ψ is given by the Lebesgue measure on R.
Moreover, (x, y) 7→ p∆ (x, y) is continuous such that for any compact interval C ⊆ R we have δ := δ(C) :=
inf x,y∈C p∆ (x, y) > 0 and the small set condition (3) is satisfied:
ˆ
ˆ
dy = δ|C|ν(B),
p∆ (x, y)dy > δ
P∆ (x, B) =
B∩C

B

where |C| denotes the Lebesgue measure of C and ν is the uniform distribution on C. It also follows that the
Markov chain is strongly aperiodic.
To show the drift condition (1), we first construct a Lyapunov function for the infinitesimal generator (which
is the continuous time analogue of the drift operator P − Id), that is we find a function V > 1 such that
LV (x) 6 −λV (x) + c1C (x),

x ∈ R.

(19)

Let V be a smooth function with V (x) = ea|x| for |x| > R for some R > 0. Due to the assumptions on b, we
then obtain for these x and R large enough
 2

1
a
LV (x) = V ′′ (x) + b(x)V ′ (x) =
+ a sign(x)b(x) V (x) 6 −λV (x)
2
2
for sufficiently small a, λ and thus the previous inequality is satisfied with C = [−R, R]. To carry this result
over to the drift condition (1), we adopt the approach by Galtchouk and Pergamenshchikov [15, Prop. 6.4]:
Itô’s formula yields for all 0 6 t 6 ∆
ˆ t
ˆ t
V (Xt ) = V (x) +
L(V )(Xs )ds +
V ′ (Xs )dWs .
0

0

13

TITLE WILL BE SET BY THE PUBLISHER

´∆
´∆
We note that Fubini’s theorem yields Eµ [ 0 V ′ (Xs )2 ds] = 0 Eµ [V ′ (X0 )2 ]ds < ∞ for constants a small enough
´∆
by (18) and by the assumptions on b. Consequently we have Ex [ 0 V ′ (Xs )2 ds] < ∞ for almost all x ∈ R. By
´∆
the explicit formula of p∆ (x, y) we conclude that Ex [ 0 V ′ (Xs )2 ds] < ∞ for all x ∈ R. Hence, the stochastic
integral is a martingale (under Px ) and Z(t) := Pt V (x) satisfies
Z ′ (t) = Ex [L(V )(Xt )] = −λZ(t) + ψ(t),

ψ(t) := Ex [L(V )(Xt ) + λV (Xt )],

where we have ψ(t) 6 cPx (Xt ∈ C) 6 c by (19). Solving this differential equation, we obtain for all t ∈ [0, ∆]
Z(t) = Z(0)e

−λt

+

ˆ

t

e−λ(t−s) ψ(s)ds 6 V (x)e−λt + c

0

1 − e−λ∆
.
λ

Therefore, the drift condition follows:
P∆ V (x) − V (x) 6 (e−λ∆ − 1)V (x) +

c
e (x) + c 1{|x|6R} (x),
6 −λV
λ
λ

e > 0 are chosen such that (1 − e−λ∆ − λ)V
e (x) > c/λ for |x| > R. In combination with the
where R > 0 and λ
ψ-irreducibility the drift condition shows that the Markov chain is positive Harris recurrent.
Since our diffusion is symmetric, in the sense that the transition operator is symmetric with respect to
L2 (µ), we argue as Bakry et al. [3, Sect. 4.3], using that the Poincaré inequality is implied by a Lyapunov–
Poincaré inequality and we thus have the contraction property (4) [3, Thm. 1.3]. Finally, the smoothness of b
in combination with the formula for the invariant probability density (18) imply that µ is in C s+1 (D).

Theorem 1.5 and Proposition 1.6 yield immediately
Corollary 2.2. Grant Assumption B and let w = (wj ) be increasing and satisfy
projection estimator µ
bJn from (6) with 2Jn = (n/ log n)1/(2s+3) satisfies
√

d

n(b
µJn − µ) −→ Gµ

√
j/wj → 0. Then the wavelet

in M0 (w).

Moreover, the confidence
√ band C n = C n (ζα , s + 1, un ) from (11) with critical value ζα such that P (kGµ kM >
ζα ) 6 α and un = wJn / Jn satisfies
lim inf P (µ ∈ C n ) > 1 − α
n→∞

and

|C n |∞ = OP

 n −(s+1)/(2s+3) 
un .
log n

2.2. Drift estimation via plug-in
Supposing from now on that σ = 1 and rewriting the formula of the invariant measure (18), we see that
b(x) =

′
1
log µ(x) .
2

(20)

Obviously, b depends on µ in a nonlinear way and the estimation problem is ill-posed because b is a function
of the derivative µ′ . In general, the same calculation leads to a formula for the function b(x)/σ 2 . Note that
all shape properties of the drift function, like monotonicity, extrema, etc. are already determined by b/σ 2 . As
demonstrated by Gobet et al. [20], the information on σ is encoded in the transition operator of the underlying
Markov chain. However, the estimation procedure in this latter article is quite involved and the construction
of adaptive confidence bands in the general setting is beyond the scope of the present article. In the following
we always set σ = 1. Note that if we have an estimator for σ at hand, for instance from a short high-frequency
time series of the diffusion, the results easily carry over to an unknown volatility σ > 0.

14

TITLE WILL BE SET BY THE PUBLISHER

Denoting the set of continuous functions on the real line by C(R), we introduce the map

ξ : f ∈ C 1 (R) : f > 0, kf kL1 = 1 → C(R),

f 7→

f′
,
2f

´·
which is one-to-one with inverse function ξ −1 (g) = exp(2 0 g(y)dy − cg ) with normalization constant cg ∈ R
and for any function g in the range of ξ. We can thus estimate the drift function of the diffusion by the plug-in
estimator ξ(b
µJn ).
Using the confidence set C n (ζα , s + 1, un ) for the invariant density µ from (11), a confidence band for the
drift can be constructed via

Dn := Dn (ζα , s, un ) := ξ(f ) : f ∈ C n (ζα , s + 1, un ) .

(21)

Since ξ is one-to-one, an immediate consequence of Corollary 2.2 is that we have for the coverage probability
lim inf n→∞ P (b ∈ Dn ) = lim inf n→∞ P (µ ∈ C n ) > 1 − α. To bound the diameter of Dn , we first note that ξ is
locally Lipschitz continuous: For f, g ∈ C 1 (R) both bounded away from zero on [a, b] we have in L∞ ([a, b])
g′
1 f ′ − g′
1 g′
1 f′
−
(g − f )
6
+
2 f
g ∞
2
f
2 fg
∞
∞
1 −1
′
′
−1
6 kf k∞ kf − g k∞ + kξ(g)k∞ kf k∞ kf − gk∞
2

6 kf −1 k∞ 12 + kξ(g)k∞ kf − gkC 1 ([a,b]) .

kξ(f ) − ξ(g)k∞ =

(22)

For f, g ∈ C n we conclude in L∞ ([a, b])
kξ(f ) − ξ(g)k∞ 6 kξ(f ) − ξ(µ)k∞ + kξ(g) − ξ(µ)k∞


6 12 + kbk∞ kf −1 k∞ kf − µkC 1 ([a,b]) + kg −1 k∞ kg − µkC 1 ([a,b]) .
Analogously to (12) the choice 2Jn = (n/ log n)1/(2s+3) yields
kf − µkC 1 ([a,b]) = OP

 n −s/(2s+3) 
un
log n

for all f ∈ C n (ζα , s + 1, un ).

We conclude that f −1 is uniformly bounded in L∞ ([a, b]) for all f ∈ C n . Hence, we have proved
√
Proposition 2.3. Grant Assumption B with σ = 1, s > 0 and let w = (wj ) satisfy j/wj → 0. Then the
confidence
set Dn = Dn (ζα , s, un ) from (21) with critical value ζα satisfying P (kGµ kM > ζα ) 6 α, un =
√
wJn / Jn and Jn chosen such that 2Jn = (n/ log n)1/(2s+3) fulfils
lim inf P (b ∈ Dn ) > 1 − α
n→∞

and

Dn

∞

= OP

 n −s/(2s+3) 
un .
log n

Let us comment on the rate appearing in the previous proposition. Since the identification (20) incorporates
the derivative of the invariant measure, drift estimation is an inverse problem, which is ill-posed of degree one.
Therefore, the minimax rate for the pointwise or L2 -loss is n−s/(2s+3) . Considering the uniform loss, we obtain
the rate (n/ log n)−s/(2s+3)
. Finally, un → ∞ is the payment for undersmoothing (by using a weighting sequence
√
slightly larger than j). Note that we obtain a faster rate than Gobet et al. [20] who have proved that the
minimax rate for drift estimation for the mean integrated squared error is n−s/(2s+5) if there is additionally an
unknown volatility function in front of the Brownian motion in (16).

15

TITLE WILL BE SET BY THE PUBLISHER

In fact the map ξ is not only Lipschitz continuous, but even Hadamard differentiable (on appropriate function
spaces) with derivative at µ
 ′
1 h
′
ξµ (h) =
, h ∈ M0 (w).
(23)
2 µ
Using the delta method [46, Thm. 20.8], we obtain a functional central limit theorem for the plug-in estimator
ξ(b
µJn ).
√
Theorem 2.4. Grant Assumption B with σ = 1 and let w = (wj ) be increasing and satisfy j/wj → 0,
wj 6 2jδ for some δ ∈ (0, 1/2). Let Jn ∈ N fulfil, for some τ ∈ (0, 1),
2(9/4+δ/2)Jn wJn n−1/2 = o(1),

√ −Jn (2s+3)/2 −1
n2
wJn = o(1),

For w
ej > 2j(1+δ) we have as n → ∞

√
d
n(ξ(b
µJn ) − b) −→ ξµ′ (Gµ )

(log n)2/τ n−1 2Jn Jn = O(1).

in M0 (w).
e

The proof of this theorem is postponed to Section 5.1. Similarly as in (11) confidence bands for the drift
function can alternatively be constructed by


ζα
Dn (ζ α , s, un ) := f : kf − ξ(b
µJn )kM(w)
e < √ , kf kC s 6 un ,
n
for α ∈ (0, 1/2), a quantile ζ α such that P (kξµ′ (Gµ )kM(w)
e < ζ α ) > 1 − α and a sequence un → ∞. With
√
Jn
−Jn
1/(2s+3)
2 = (n/ log n)
and un = w
eJn 2
/ Jn this leads to asymptotic coverage of at least 1 − α and a
diameter decaying at rate (n/ log n)−s/(2s+3) un . Note that in contrast to Dn the diameter of Dn is slightly
suboptimal due to the δ > 0 that appeared in Theorem 2.4 and which presumably could be removed by a more
technical proof. Based on a direct estimator of the drift we will construct a similar confidence band with the
optimal diameter (up to undersmoothing) in the next section.
Comparing both constructions of confidence sets, we see that Dn can be understood as the variance stabilised
version of Dn : The critical value of Dn depends on the unknown µ only through the covariance structure of the
limit processes Gµ which seems to be unavoidable due to the underlying Markov chain structure. In contrast ζ α
depends additionally on µ through the derivative ξµ′ . As a consequence the confidence band Dn has the same
diameter everywhere while the diameter of Dn changes.

2.3. A direct approach to estimate the drift
Instead of relying only on the estimator µ
bJn of the invariant density and the plug-in approach, we can
use a direct approach to estimate the drift and to obtain its confidence bands. Although there is a one-toone correspondence between the drift function b and the invariant measure µ, the drift is both the canonical
parameter of our model and the main parameter of interest in the context of diffusions. Since we aim for adapting
to the regularity of b, the direct estimation approach is natural and, additionally, the resulting confidence bands
will have a constant diameter.
Motivated by formula (20), we define our drift estimator for integers J, U > 0 as
′
′
1X X
bbJ,U = 1 πJ log µ
h log µ
bJ+U , ψj,k iψj,k ,
bJ+U =
2
2

(24)

j6J k∈Kj

using the wavelet projection estimator µ
bJ+U from (6). In contrast to the plug-in estimator in the previous section
the underlying bias-variance trade-off is now driven by the estimation problem of b and the outer projection πJ
onto level J. However, in order to linearise the estimation error, we need a stable prior estimator of µ such that

16

TITLE WILL BE SET BY THE PUBLISHER

we cannot simply use the empirical measure µn but instead use its projection onto some resolution level J + U
which is strictly larger than J. As a rule of thumb, U = Un can be chosen such that 2Un = log n implying that
an additional bias term from estimating µ is negligible. Linearising the estimation error, we obtain
hbbJ,U − b, ψj,k i = − µ
bJ+U − µ,

′
ψj,k

2µ

+ hRJ+U , ψj,k i,

j 6 J, k ∈ Kj ,

(25)

where the remainder is of order oP (n−1/2 ) for appropriate choices of J = Jn , cf. Lemma 4.1 below. In view
of the linear error term and our findings in Section 1, the limit process Gb in the multi-scale space M0 will be
given by
′
Gb (j, k) ∼ N (0, Σj,k ) where Σj,k := Σf,f is given by (2) with f = ψj,k
/(2µ)
(26)

′
′
/(2µ) and f2 = ψl,m
/(2µ). The illwith covariances E[Gb (j, k)Gb (l, m)] = Σf1 ,f2 from (2) with f1 = ψj,k
′
j
posedness of the problem is reflected by ψj,k being a factor 2 larger than ψj,k . We thus need larger weights for
high resolution levels to ensure that Gb takes values in M0 (w).

Definition 2.5. A weighting sequence w = (wj ) is called admissible, if it is monotonously increasing, satisfies
√
j2j /wj → 0 as j → ∞ and if there is some δ ∈ (1, 2] such that j 7→ 2jδ /wj is monotonously increasing for
large j.
The last condition in the definition is a mild technical assumption that
√ we will need in the multi-scale central
limit theorem below. For instance, any weighting sequence wj = uj j2j with uj = j p for some polynomial
rate p > 0 is admissible of degree one for any δ ∈ (1, 2]. Note that admissibility of w implies in particular
that wj . 2jδ which allows to compare the k · k∞ -norm with the k · kM -norm. We find an analogous result to
Lemma 1.3, cf. Castillo and Nickl [8, Prop. 3].
√
Lemma 2.6. Gb from (26) satisfies E[kGb kM(w) ] < ∞ for the weights w given by wj = j2j . Moreover, L(Gb )
is a tight Gaussian Borel probability measure in M0 (w) for any admissible sequence w.

For the following result suppose that the wavelet basis (ϕj0 ,l , ψj,k : j > j0 , l ∈ L, k ∈ Kj ) of L2 (R) is
sufficiently regular (i.e., satisfies (34) with γ > 3/2 + δ), for instance, Daubechies’ wavelets of order N > 20.
Theorem 2.7. Grant Assumption B with σ = 1 and let w = (wj ) be admissible. Let Jn → ∞, Un → ∞ fulfil
√ −Jn (s+1/2) −1
n2
wJn = o(1),

√ −(Jn+Un )(2s+1)
n2
= o(1),

Then bbJ,U from (24) satisfies, as n → ∞,

√
d
n(bbJn ,Un − b) −→ Gb

n−1/2 22(Jn +Un ) (Jn + Un ) = o(1).

in M0 (w)

for the tight Gaussian random variable Gb in M0 (w) given by (26).
The proof of this theorem is postponed to Section 4. The first condition on Jn is the bias condition for b in
M0 . The latter two conditions on Jn + Un are determined by a bias and a variance condition for µ which we
will need to bound the remainder RJn +Un from (25) in L∞ . If δ < 1/2 + s in Definition 2.5, then the second
condition is strictly weaker than the first one.
Similarly to the confidence band for µ in Proposition 1.6 we can now construct a confidence band for the
drift function b. For some α ∈ (0, 1) we consider
o
n
ζα
En := En (ζα , s, un ) := f : kf − bbJn ,Un kM < √ , kf kC s 6 un ,
n
where ζα is chosen such that P (kGb kM < ζα ) > 1 − α and (un )n is a diverging sequence.

(27)

TITLE WILL BE SET BY THE PUBLISHER

17

Proposition 2.8. Grant Assumption B with σ = 1, s > 1 and let w = (wj ) be admissible. For α ∈ (0, 1) let
ζα > 0 satisfy P (kGb kM > ζα ) 6 α and choose Jn := Jn (s) and Un → ∞ such that
2 Jn =

 n 1/(2s+3)
log n

and

2Un = O(log n).

√
Then the confidence set En = En (ζα , s, un ) from (27) with un := wJn 2−Jn / Jn satisfies
lim inf P (b ∈ En ) > 1 − α
n→∞

and

|En |∞ = OP

 n −s/(2s+3) 
un .
log n

Proof. The proof is essentially the same as for the confidence band of the invariant probability density. We
show that the asymptotic coverage probability is at least 1 − α and obtain for f, g ∈ En as in (12) the bound




kf − gk∞ = OP n−1/2 2Jn /2 wJn + OP 2−Jns un
√
Using un = wJn 2−Jn / Jn we thus have





kf − gk∞ = OP n−1/2 23Jn /2 Jn1/2 un + OP 2−Jns un .
The choice of Jn yields n−1/2

p
23Jn Jn . (n/ log n)−s/(2s+3) = 2−Jns .



3. Adaptive confidence bands for drift estimation
Inspired by Giné and Nickl [18], we will now construct an adaptive version of the confidence set En from (27).
To this end we estimate the regularity s of the drift with a Lepski-type method. For some maximal regularity
r > 1, let the integers 0 < Jmin < Jmax be given by
2Jmin ∼

 n 1/(2r+3)
,
log n

2Jmax ∼

n1/4
.
(log n)2

Note that Jmin , Jmax depend on the sample size n, which is suppressed in the notation. If we knew in advance
that b has regularity r, then we would choose the resolution level Jmin . The upper bound Jmax is chosen such
that Jmax + Un satisfies the third condition in Theorem 2.7. The set in which we will adaptively choose the
optimal resolution level for regularities s ∈ [1, r] is defined by
Jn := [Jmin , Jmax ] ∩ N.
Similar to Giné and Nickl [18, Lem. 2], we show under the following assumption on b that the optimal truncation
level can be consistently estimated up to a fixed integer.
Assumption C. Let b ∈ C s (D), s > 1, satisfy for constants 0 < d1 < d2 < ∞ and an integer J0 > 0 that
d1 2−Js 6 kπJ (b) − bkL∞ ([a,b]) 6 d2 2−Js ,

∀J > J0 .

(28)

The second inequality in (28) is the well known Jackson inequality which is satisfied for all usual choices of
wavelet basis. The first inequality is the main condition here, called self-similarity assumption. It excludes the
cases where the bias would be smaller than the usual order 2−Js . Although the estimator bbJ,U would profit
from a smaller bias, we cannot hope for a consistent estimation of the optimal projection level and the resulting
regularity index s if (28) (or a slight generalisation by Bull [5]) is violated. Indeed, Hoffmann and Nickl [26]
have shown that this kind of condition is necessary to construct adaptive and honest confidence bands. On

18

TITLE WILL BE SET BY THE PUBLISHER

the other hand, it has been proved by Giné and Nickl [18] that the set of functions that do not satisfy the
self-similarity assumption is nowhere dense in the Hölder norm topologies. In that sense, the self-similarity
assumption is satisfied by “typical” functions. We will give an illustrative example next. Probabilistic examples
for self-similar functions are those Gaussian processes which can be represented as stochastic series expansions
like the Karhunen-Loève expansion for Brownian motion or typical examples of Bayesian priors. Naturally,
more regular functions b ∈ C r (D) for some r > s cannot satisfy Assumption C. For a further discussion and
examples we refer to Giné and Nickl [18, Section 3.5] as well as Bull [5].
Example 3.1. Let b be a smooth function on R except for some point x0 ∈ (a, b) where the sth order derivative
b(s) has a jump for some integerP
s > 1. Locally around x0 the function b can be approximated by a Taylor
s−1
polynomial b(x) = βs± (x − x0 )s + l=0
βl (x − x0 )l + O(|x − x0 |s+1 ) with coefficients β0 , . . . , βs−1 ∈ R and where
the coefficient of order s is some βs+ ∈ R or βs− ∈ R depending on x > x0 and x < x0 , respectively. Due to the
jump of b(s) we have βs+ 6= βs− .
Choose kj as the nearest integer of 2j x0 , implying that x0 is in the middle of the support of ψj,kj . Using the
elementary estimate |hf, ψj,k i| 6 kf kL∞ (supp ψj,k ) 2−j/2 kψkL1 , we obtain
kπJ (b) − bkL∞ ([a,b]) =

X

j>J,k

hb, ψj,k iψj,k

L∞ ([a,b])

& sup 2j/2 |hb, ψj,kj i|.
j>J

For sufficiently large j the regularity of the wavelet basis, being thus orthogonal to polynomials, and their
compact support yield
2j/2 |hb, ψj,kj i| > 2j

ˆ

b(x) − βs− (x − x0 )s +

> 2j |βs+ − βs− |

ˆ

s−1
X
l=0


βl (x − x0 )l ψ(2j x − kj )dx + O(2−j(s+1) )

(x − x0 )s ψ(2j x − kj )dx + O(2−j(s+1) )
ˆ
y s ψ(y + ε)dy + O(2−j(s+1) ).
min

x>x0

> 2−js |βs+ − βs− |

ε∈[−1/2,1/2]

y>0

We conclude for J sufficiently large that kπJ (b) − bkL∞ ([a,b]) & 2−Js .

The oracle choice Jn∗ which balances the bias kπJ (b) − bkL∞ ([a,b]) and the main stochastic error is given by
Jn∗

:=

Jn∗ (s)

n
K
= min J ∈ Jn : (d2 + 1)2−Js 6
4

r

23J J o
n

for some suitable constant K > 0 depending only on ψ, inf{µ(x) : x ∈ ∪l∈L supp ϕj0 ,l } and the maximal
e = sup Σj,k where the latter two quantities can be replaced by the consistent estimators
asymptotic variance Σ
j,k
∗
which we have discussed in Section 1.3. We see easily that 2Jn ∼ ( logn n )1/(2s+3) . Following Lepski’s approach,
we define the estimator for Jn∗ by
r
n
o
23j j
b
b
b
Jn = min J ∈ Jn : kbJ − bj kL∞ ([a,b]) 6 K
(29)
∀j > J, j ∈ Jn .
n
Lemma 3.2. Grant Assumptions B and C for s ∈ [1, r] with some r > 1 and σ = 1. Let w be admissible. Then
there are a constant K > 0 depending only on ψ, inf{µ(x) : x ∈ ∪l∈L supp ϕj0 ,l } and the maximal asymptotic
e = supj,k Σj,k , an integer M > 0 depending only on d1 , d2 , K and for any τ ∈ (0, 1) there are
variance Σ
constants C, c > 0 depending on τ, K, ψ such that


τ
/ [Jn∗ − M, Jn∗ ] 6 C n−cJmin + e−cJmin → 0.
P Jbn ∈

TITLE WILL BE SET BY THE PUBLISHER

19

The proof of this lemma relies on the concentration result in Proposition 1.2 and is postponed to Section 5.2.
Applying that Jbn is a reasonable estimator of Jn∗ , we obtain a completely data-driven estimator
bb := bb b
Jn ,Un ,

with Jbn from (29) and 2Un = log n.

(30)

Corollary 3.3. In the situation of Lemma 3.2 the adaptive estimator bb defined by (30) satisfies kbb − bkM =

∗ p
OP (n−1/2 ) and kbb − bkL∞ ([a,b]) = OP (n/ log n)−s/(2s+3) un with un := wJn∗ 2−Jn / Jn∗ . Further for every
m ∈ {0, 1, . . . , M } we have
√
d
n(bbJn∗ −m,Un − b) −→ Gb in M0 (w)
as n → ∞ for the tight Gaussian random variable Gb in M0 (w) given by (26).
Proof. Combining Lemma 3.2 and Theorem 2.7, there is for any δ > 0 a constant C > 0 such that for n large
enough
∗
Jn
X


√
√
P nkbbJ − bkM > C + o(1) 6 (M + 1)δ.
P nkbb − bkM > C 6
∗ −M
J=Jn

∗
b
Since M is a finite constant, we havekbb − bkM = OP (n−1/2 ). Using that 2Jn ∼ 2Jn , a calculation similar to
(12) yields the bound for the uniform norm. For the second claim notice that the estimators bbJn∗ −m,Un satisfy
the conditions of Theorem 2.7.


The bound for the uniform risk is slightly suboptimal because un diverges arbitrary slowly (depending on
the choice of w) to infinity. Using direct estimates of the k · k∞ -norm in the proofs in Section 4, this additional
factor could be circumvented. However, it can be interpreted as an additional factor that corresponds to a slight
undersmoothing which is often used to have a negligible bias in the construction of confidence bands.
Another consequence of Lemma 3.2 is that we can consistently estimate the regularity s of b. For a sequence
of random variables (vn ) with vn−1 = oP (1) we define the estimator


vn 
log n − log log n
3
.
(31)
1+
sbn := max 1,
−
Jbn
2(log 2)(Jbn + vn ) 2
∗

Using that 2Jn ∼ (n/ log n)1/(2s+3) , we derive from Lemma 3.2 the following corollary. The proof can be found
in Section 5.3.

Corollary 3.4. In the situation of Lemma 3.2 the estimator sbn given by (31) satisfies for any sequence of
random variables (vn ) with vn−1 = oP (1)
v 
n
P (b
sn 6 s) → 1 and s − sbn = OP
.
Jn∗

With the estimator bb from (30) we can now construct our adaptive confidence bands as follows. By a
Bonferroni correction we take care of the possible dependence between the estimators bbJn∗ −m,Un and the adaptive
choice Jbn of the resolution level. In this way sample splitting can be avoided, which was also used by Bull [5].
For any level α ∈ (0, 1) let β = α/(M + 1) and define
o
n
ζbβ,n
Een := Een (ζbβ,n , sbn , tn ) := f : kf − bbkM < √ , kf kC sbn 6 tn ,
n

(32)

b
where (tn ) is a sequence of random variables with t−1
n = oP (1) and ζβ,n is an (over-)estimator of the critical
value ζβ given by P (kGb kM < ζβ ) > 1 − β similarly to the construction in Section 1.3. Now we can state our
final theorem:

20

TITLE WILL BE SET BY THE PUBLISHER

Theorem 3.5. Grant Assumptions B and C for σ = 1, s ∈ [1, r] with some r > 1. Let w = (wj ) be admissible
∗ p
b −1/2
bn := wJbn 2−Jn Jbn . For α ∈ (0, 1) set β := α/(M + 1) with M
and define un := wJn∗ 2−Jn / Jn∗ as well as u
as in Lemma 3.2. Let ζβ > 0 be given by P (kGb kM > ζβ ) 6 β and let ζbβ,n be an (over-)estimator satisfying
P (ζbβ,n√> ζβ − ε) → 1 for all ε > 0. Then the confidence set Een = Een (ζbβ,n , sbn , tn ) given by (32), where we choose
tn := u
bn and sbn according to (31) with vn = oP (log u
bn ) and vn−1 = oP (1), satisfies
lim inf P (b ∈ Een ) > 1 − α
n→∞

and

|Een |∞ = OP

 n −s/(2s+3) 
un .
log n

Proof. We will adapt the proof of Proposition 2.8 to the estimated quantities Jbn , sbn and ζbβ . By Corollary 3.4
the probability of the event {b
sn 6 s} converges to one. Due to vn−1 = oP (1), we thus have b ∈ C bsn (vn ) with
probability tending to one. Using Lemma 3.2 and Corollary 3.3, we infer
lim sup P (b ∈
/ Een ) = lim sup
n→∞

n→∞

M
X

m=0

P


√
nkbbJn∗ −m,Un − bkM > ζbβ,n 6 (M + 1)P (kGb kM > ζβ ) 6 α.

We conclude that lim inf n→∞ P (b ∈ Een ) > 1 − α.
To estimate the diameter, we proceed as in (12). Applying additionally Corollary 3.4, we obtain for any
f, g ∈ Een
kf − gk∞ .

X

∗
j6Jn

2j/2 max |hf − g, ψj,k i| +
k

6 kf − bbkM + kg − bbkM

 X

∗
j6Jn

X

∗
j>Jn

2j/2 max |hf − g, ψj,k i|
k

2j/2 wj + kf − gkC sbn


∗
∗
= OP n−1/2 2Jn /2 wJn∗ + OP (tn 2−Jn s+OP (vn ) )


∗
∗
bn ,
= OP n−1/2 23Jn /2 (Jn∗ )1/2 un + 2−Jn s u

X

2−jbsn

∗
j>Jn

where we have plugged in the choices of tn , un and vn and u
bn . un with probability converging to one. Since
∗
2Jn ∼ (n/ log n)1/(2s+3) , the assertion follows.

The confidence bands are constructed explicitly and this helps to verify that the confidence bands are honest,
i.e. the coverage is achieved uniformly over some set of the unknown parameter. The general philosophy being
that uniformity in the assumptions leads to uniformity in the statements, the detailed derivation of honesty is
tedious so that we only sketch it here. The main ingredients of the proof are the central limit theorem and the
concentration inequality for Markov chains. In the original version of the concentration inequality, Theorem 9
by Adamczak and Bednorz [1], the constants are given explicitly in terms of the assumptions and thus the
concentration inequality is uniform in the underlying Markov chain Z. It is also to be expected that the central
limit theorem holds uniformly in the bounded-Lipschitz metric with respect to Z although this is not explicitly
contained in the statement. With these uniform ingredients a uniform version of Theorem 1.5 can be proved,
where the convergence in distribution is again metrised in the bounded-Lipschitz metric. In combination with
a uniform bound on the Lebesgue densities of kGµ kM this leads to honest confidence bands in Proposition 1.6.
Thanks to the explicit derivation of Assumption A from Assumption B, uniformity in the diffusion model carries
over to uniformity in the Markov chain and we see that the confidence bands in Proposition 2.3 are honest.
Likewise a uniform version of Theorem 2.7 can be proved. Provided the random variables kGb kM have uniformly
bounded Lebesgue densities this uniform version entails honest and adaptive confidence bands for the drift in
Theorem 3.5.

21

TITLE WILL BE SET BY THE PUBLISHER

4. Proof of Theorem 2.7
In the sequel we use the notation
J+ = J + U
for the projection level of µ
bJ + and we define
S :=

[

l∈L

supp ϕj0 ,l ⊆ [a − 2−j0 (2N − 1), b + 2−j0 (2N − 1)].

To analyse the estimation error of the wavelet coefficients hbbJ,U , ψj,k i, we apply the following linearisation
lemma:
Lemma 4.1. Grant Assumption B with σ = 1. For j ∈ {−1, j0 , . . . , J} and k ∈ Kj we have
hbbJ,U − b, ψj,k i = − µ
bJ + − µ,
where the remainder is given by RJ + = −

µ
bJ + −µ
2b
µJ +

+

2−Jn (s+1) = o(1),
then



µ
bJ + −µ
µ

′

′
ψj,k
+ hRJ + , ψj,k i,
2µ

. If J + = Jn+ satisfies for some τ ∈ (0, 1)
+

(log n)2/τ n−1 2Jn Jn+ = O(1),


+
+
kRJn+ kL∞ (S) = OP n−1 Jn+ 22Jn + 2−Jn (2s+1) .

Proof. Writing η := (b
µJ + − µ)/µ, the chain rule yields

′
η′
bJ + − µ ′
1
1µ
1
log(1 + η) =
+ RJ + ,
(log µ
bJ + )′ − b =
=
2
2
2(1 + η)
2
µ

where the remainder is given by

RJ + := −

bJ + − µ ′
µ
b + − µµ
ηη ′
=− J
.
2(1 + η)
2b
µJ +
µ

Using integration by parts with vanishing boundary terms, the wavelet coefficients corresponding to the linear
term can be written as
1
1
′
h((b
µJ + − µ)µ−1 )′ , ψj,k i = − hb
µ + − µ, ψj,k
µ−1 i.
2
2 J
Let us bound the remainder, starting with kb
µ′J + − µ′ kL∞ (S) . Decomposing the uniform error into a bias and
n
a stochastic error term, we obtain
kb
µ′J + − µ′ kL∞ (S) 6
n

X

+
j6Jn
,k

′
hµn − µ, ψj,k iψj,k

L∞ (S)

+

X

+
j>Jn
,k

′
hµ, ψj,k iψj,k

L∞ (S)

=: Vn + Bn .
P
Using the localisation property of the wavelet function k k |ψ(• − k)|k∞ . 1 (which holds for ψ ′ as well) and
the regularity of µ ∈ C s+1 (D), implying supj,k:supp ψj,k ∩S6=∅ 2j(s+3/2) |hµ, ψj,k i| < ∞, the bias can be estimated
by
X
X
X
+
′
Bn .
max
|hµ, ψj,k i|
|ψj,k
|
.
2−js . 2−Jn s .
+
j>Jn

k:supp ψj,k ∩S6=∅

k

L∞ (S)

+
j>Jn

22

TITLE WILL BE SET BY THE PUBLISHER

For the stochastic error term we obtain similarly
X
Vn .
23j/2
+
j6Jn

max

k:supp ψj,k ∩S6=∅

|hµn − µ, ψj,k i|.

√
The maximum of 2j subgaussian random variables is of order OP ( j). More precisely, Proposition 1.2 and the
assumptions on Jn+ yield for any j0 6 j 6 Jn+ and τ ∈ (0, 1), similarly to (10),


p


√
P
max
n|hµn − µ, ψj,k i| > jt . 2j exp − c1 (log n)j τ tτ + 2j exp − c2 j(t ∧ t2 )
k:supp ψj,k ∩S6=∅


6 exp j(log 2 − c1 j τ −1 (log n)tτ ) + exp j(log 2 − c2 (t ∧ t2 )) .

Using Jn+ . log n, the right-hand side of the previous display is arbitrarily small for large enough t. An analogous
estimate holds for the scaling functions ψ−1,· . Therefore,


q
 X

p
+
+ 
+ 
+
3j/2
′
′
3Jn
/2
−Jn
s
2
kb
µJ + − µ k∞ = OP
= OP 2
j/n + O 2
Jn /n + O 2−Jn s .
n

+
j0 6j6Jn

Analogously, we have
kb
µJn+ − µk∞



q

+
+
+
Jn
/2
= OP 2
Jn /n + O 2−Jn (s+1) .

Since µ is bounded away from zero on S, the choice of Jn+ yields in particular that we have limn→∞ P (inf x∈S µ
bJn+ (x) >
1
inf
µ(x))
=
1.
We
conclude
x∈S
2

µ′J + − µ′ kL∞ (S) kb
µJn+ − µkL∞ (S) + kb
µJn+ − µk2L∞ (S)
kRJn+ kL∞ (S) = OP kb
n
q
q


+
+ 
+
+
= OP 23Jn /2 Jn+ /n + 2−Jn s 2Jn /2 Jn+ /n + 2−Jn (s+1) ,
which shows the asserted bound for kRJn+ kL∞ (S) .



The linearised stochastic error term can be decomposed into
′
′
X
ψj,k
ψj,k
1
′
+ (Id −πJ + )µ,
− hb
µJ + − µ, ψj,k
µ−1 i = −
hµn − µ, ψl,m i ψl,m ,
2
2µ
2µ
+
l6J ,m

= − µn − µ,

′
′
 ψ′ 
ψj,k
ψj,k
j,k
+ µn − µ, (Id −πJ + )
+ (Id −πJ + )µ,
.
2µ
2µ
2µ

(33)

Roughly for j 6 J 6 J + , Theorem 1.5 (or an analogous result for ill-posed problems) applies for the first term
in the above display, the second term should converge to zero by the localisation of the ψj,k in the Fourier
domain and the third term is a bias that can be bounded by the P
smoothness of µ. If Un → ∞ this “µ-bias”
term is of smaller order than the “b-bias” which is determined by j>J,k hb, ψj,k iψj,k .
Let us make these considerations precise. We will need the following lemma, which relies on the localisation
of the wavelets in Fourier domain. More precisely, ψ can be chosen such that for some γ > 1 we have
ˆ
ϕ, ψ ∈ C γ (R) and
xk ψ(x)dx = 0, for k = 0, . . . , ⌈γ⌉.
In the Fourier domain we conclude by the compact support of ψ
|F ϕ(u)| .

1
,
(1 + |u|)γ

|F ψ(u)| .

|u|γ
,
(1 + |u|)2γ

u ∈ R.

(34)

23

TITLE WILL BE SET BY THE PUBLISHER

Lemma 4.2. Grant Assumption B and let the compactly supported father and mother wavelet functions ϕ and
ψ satisfy (34) for some γ > 1. Then for any m ∈ Z, j < l and k ∈ Kj
X

′
|hψl,m , ψj,k
µ−1 i| . 2l−(l−j)(γ−1/2) + 2−l+j ,

m∈Z

X

m∈Z

′
|hψl,m , ψj,k
µ−1 i| . 2l−(l−j)(γ−3/2) + 1

and

′
|hψl,m , ψj,k
µ−1 i|2 . 22l−(l−j)(2γ−2) + 2−(l−j) ,

where we have to replace j by j0 on the right-hand side for ψ−1,k = ϕj0 ,k .
Proof. Let Γ > 0 be large enough such that supp ϕ ∪ supp ψ ⊆ [−Γ, Γ]. Noting that the following scalar product
can only be nonzero if the support of ψl,m is contained in D, a Taylor expansion of µ−1 yields for j > j0
′
i|
|hψl,m µ−1 , ψj,k
′
6 |µ(2−l m)|−1 |hψl,m , ψj,k
i| + 2−l Γ

max

x:|x−m2−l |62−l Γ

′
6 kµ−1 kL∞ (D) |hψl,m , ψj,k
i| + 2−l+j Γ

We conclude

|(µ−1 )′ (x)|

max

x:|x−m2−l |62−l Γ

ˆ

|ψl,m (x)||ψ ′j,k (x)|dx

|(µ−1 )′ (x)|kψkL2 kψ ′ kL2 .

′
′
|hψl,m , ψj,k
µ−1 i| . kµ−1 kL∞ (D) |hψl,m , ψj,k
i| + 2−l+j k(µ−1 )′ kL∞ (D) .

(35)

−l

Using Plancherel’s identity, F ψl,m (u) = F [2l/2 ψ(2l • − m)](u) = 2−l/2 eimu2 F ψ(2−l u) and (34), we obtain
′
|hψl,m , ψj,k
i| 6

2−(j+l)/2
2π

ˆ

|F ψ(2−j u)uF ψ(2−l u)|du

2−(j+l)γ |u|2γ+1
du
(1 +
+ 2−l |u|)2γ
ˆ
ˆ
|u|du
|v|dv
l−(l−j)(γ−1/2)
6 2−(j+l)/2−(j+l)γ
=
2
,
−2γj
−l
2γ
2
(1 + 2 |u|)
(1 + |v|)2γ

. 2−(j+l)/2

ˆ

2−j |u|)2γ (1

where we have substituted v = 2−l u in the last line. Due to γ > 1, the integral in the last display is finite so
that combining this bound with (35) yields the assertions, noting that by the compact support of ψ only for
′
O(2l−j ) many m the scalar products hψl,m , ψj,k
µ−1 i are nonzero.
−l
For j = −1 we substitute again v = 2 u and obtain analogously
′
|hψl,m , ψ−1,k
i| .

ˆ

2−(j0 +l)/2−lγ |u|γ+1 du
6 2l−(l−j0 )(γ−1/2)
(1 + 2−l |u|)2γ (1 + 2−j0 |u|)γ

ˆ

|v|dv
(1 + |v|)2γ

and
′
|hψl,m , ψ−1,k
µ−1 i| . kµ−1 kL∞ (D) |hψl,m , ϕ′j0 ,k i| + 2−l+j0 k(µ−1 )′ kL∞ (D) .



Now we can bound the bias in (33) in the multi-scale space M0 .
Lemma 4.3. Let the weighting sequence w be admissible and grant Assumption B and (34) for some γ >
3/2 + δ, δ ∈ (1, 2]. Then we have
′
(Id −πJ+U )µ, ψj,k
/(2µ)



j6J,k M

. 2−J(s+1/2) 2−U(s+3/2) wJ−1 .

24

TITLE WILL BE SET BY THE PUBLISHER

Proof. Recall that we have by definition
′
(Id −πJ + )µ, ψj,k
/(2µ)



′
/(2µ) .
= sup max wj−1 (Id −πJ + )µ, ψj,k

j6J,k M

j6J k∈Kj

As in the proof of Theorem 1.5 we have
sup
l,m:supp ψl,m ∩S6=∅

2j(s+3/2) |hψl,m , µi| . kµkC s+1 (D) .

Hence, for all j 6 J,
′
(Id −πJ + )µ, ψj,k
/(2µ)

X

=

′
hµ, ψl,m ihψl,m , ψj,k
/(2µ)i

l>J + ,m

6 sup 2l |hµ, ψl,m i|
l>J + ,m

.2

−J + (s+1/2)

X

l>J + ,m

X

l>J + ,m

′
2−l |hψl,m , ψj,k
/(2µ)i|

′
2−l |hψl,m , ψj,k
/(2µ)i|.

Now Lemma 4.2 yields
X
X
X
+
+
′
2−l |hψl,m , ψj,k
/(2µ)i| .
2−(l−j)(γ−3/2) +
2−l . 2−(J −j)(γ−3/2) + 2−J .
l>J + ,m

l>J +

l>J +

Due to the monotonicity of j 7→ 2jδ wj−1 , we conclude for γ > 3/2 + δ
′
/(2µ)
sup max wj−1 (Id −πJ + )µ, ψj,k

. 2−J

j6J k∈Kj

+

(s+1/2)

sup wj−1 (2−(J

+

−j)(γ−3/2)

+

+ 2−J )

j6J

. 2−J

+

(s+1/2) −(J + −J)

2

wJ−1 .



The second term in (33) can be bounded by the following lemma.
√
Lemma 4.4. Let the weighting sequence w satisfy j2j /wj = O(1) and grant Assumption B and (34) for some
γ > 5/2. If Jn+ = Jn + Un satisfies for some τ ∈ (0, 1)
+

(log n)2/τ n−1 2Jn Jn+ = O(1)
then we have

Un → ∞,

and

 ψ′   
j,k
µn − µ, (Id −πJn+ )
= oP (n−1/2 ).
j,k
2µ
M
′
Proof. In order to apply Proposition 1.2, we need to calculate the L2 -norm and the L∞ -norm of (Id −πJn+ )(ψj,k
/µ).
For j ∈ {j0 , . . . , Jn } Parseval’s identity and Lemma 4.2 yield
πJn



 ψ′ 
j,k
(Id −πJn+ )
µ

2
L2

=

X

+
l>Jn
,m

.

X

′
|hψl,m , ψj,k
/µi|2

22l−(l−j)(2γ−2) +

+
l>Jn

. 2j(2γ−2)

X

X

2−2(l−j)+(l−j)+

+
l>Jn
+

2−l(2γ−4) + 2−(Jn −j)

+
l>Jn
+

+

+

. 2−(Jn −j)(2γ−4)+2j + 2−(Jn −j) . 2−(Jn −j) 22j .

25

TITLE WILL BE SET BY THE PUBLISHER

Another application of Lemma 4.2 yields
 ψ′ 
j,k
(Id −πJn+ )
2µ

∞

X

6

′
2l/2 max |hψl,m , ψj,k
/µi|
n

+
l>Jn

.

X

23l/2−(l−j)(γ−1/2) +

+
l>Jn

.2

X

2−l/2+j

+
l>Jn

+
−j)(γ−2)+3j/2
−(Jn

+

+

+ 2−(Jn −j)/2+j/2 . 2−(Jn −j)/2 23j/2 .

The concentration inequality, Proposition 1.2, yields for positive constants ci > 0, i = 1, 2, . . . ,
P



6

 ψ′ 

√ −1
j,k
>t
nwj µn − µ, (Id −πJn+ )
2µ
16j6Jn k
′
√
ψ 

X
j,k
P
n µn − µ, (Id −πJn+ )
> twj
2µ
sup max

16j6Jn ,k

.

Jn
X
j=1


√
+
2j exp(−c1 (2(Jn −j)/2 2−3j/2 nwj t)τ )


+ exp −

+

c2 2(Jn −j)/2 t2
√
.
(2j /wj )2 + t max(23j/2 (log n)1/τ , 2j )/(wj n)

Since jwj−1 23j/2 (log n)1/τ n−1/2 . 2j/2 j 1/2 (log n)1/τ n−1/2 . 1 and Jn+ . log n by the assumptions on wj and
Jn+ , we conclude for any t > 0 and n sufficiently large
P
.



sup max
16j6Jn

Jn
X
j=1

2

j



k

 ψ′ 

√ −1
j,k
>t
nwj µn − µ, (Id −πJn+ )
2µ

exp(−c3 (log n)2



2(Jn −j)/2 jt2 
j t ) + exp − c4
1+t

+
τ (Jn
−j)/2 τ τ

+

Jn


 X
t2 
exp j log 2 − c5 2Un /2
exp j log 2 − c3 (Jn+ )τ −1 tτ log n +
1+t
j=1
j=1

+ τ −1 τ
+ τ −1 τ
1 − eJn log 2−c3 (Jn ) t log n
. elog 2−c3 (Jn ) t log n
+
1 − elog 2−c3 (Jn )τ −1 tτ log n

.

Jn
X

+e

log 2−c6 2Un /2 (t2 ∧t) 1
+ τ −1 τ

. e−c3 (Jn )

t log n

Un /2

2

(t ∧t))
− eJn (log 2−c6 2
U
/2
1 − elog 2−c6 2 n (t2 ∧t)
Un /2 2
(t ∧t)

+ e−c6 2

→ 0.

Finally note that all bounds hold true for the scaling function ϕj0 ,· if j is replace by j0 .
Now we have all pieces together to prove the multi-scale central limit theorem.
Proof of Theorem 2.7. Since b has Hölder regularity s > 0 on S ⊆ D, the bias can be bounded by
kb − πJn (b)kM = sup max wj−1 |hψj,k , bi| . sup wj−1 2−j(s+1/2) = o(n−1/2 ).
j>Jn k∈Kj

j>Jn



26

TITLE WILL BE SET BY THE PUBLISHER

Using that the M0 -norm is weaker than the L∞ (S)-norm, Lemma 4.1 and decomposition (33) together with
Lemmas 4.3 and 4.4 yield
′

ψj,k

eJn ,Un
hbbJn ,Un − b, ψj,k i j6Jn ,k = − µn − µ,
+R
2µ j6Jn ,k

eJn ,Un )kM = oP (n−1/2 ).
with kπJn (R

Therefore, it remains to show that


√
′
/(2µ)i)j,k , L(Gb ) → 0.
βM0 L πJn (− nhµn − µ, ψj,k
This follows exactly as in Theorem 1.5, where we use that the factor 2j , by which the norms
′
kψj,k
/(2µ)kL2 . 2j ,

′
kψj,k
/(2µ)k∞ . 23j/2

(36)

are larger than kψj,k kL2 and kψj,k k∞ , respectively, is counterbalanced through the additional growth of the
admissible weighting sequence w.


5. Remaining proofs
5.1. Proof of Theorem 2.4
Step 1: For δ ∈ (0, 1/2) and 0 < c < C < ∞ define
Vξ := {µ ∈ C 7/4+δ/2 (D) : 0 < c < µ and kµkC 7/4+δ/2 < C},
V := M0 (w) and W := M0 (w)
e with wj 6 2δj and w
e > 2(1+δ)j . We first establish the Hadamard differentiability
of
ξ : Vξ ⊆ V → W, µ 7→

µ′
2µ

with derivative given by (23).
To this end let h ∈ M0 (w) and ht → h as t → 0. For all ht such that µ + tht is contained in Vξ for small
t > 0 we have
ξ(µ + tht ) − ξ(µ)
− ξµ′ (h)
t
µh′t − µ′ ht
µh′ − µ′ h
=
−
2(µ + tht )µ
2µ2

=
M(w)
e

M(w)
e

µ(µ′ + th′t ) − µ′ (µ + tht ) µh′ − µ′ h
−
2(µ + tht )µt
2µ2

using w
ej > 2j(1+δ) for δ ∈ (0, 1/2), this is bounded by
(h′t − h′ )
2(µ + tht )

+
−3/2−δ

B∞∞

M(w)
e

µ2 (h′t − h′ ) + µµ′ (h − ht ) + tht (µ′ h − µh′ )
=
2µ2 (µ + tht )

µ′ (h − ht )
2µ(µ + tht )

+
−3/2−δ

B∞∞

tht (µ′ h − µh′ )
2µ2 (µ + tht )

.
M(w)
e

,
M(w)
e

27

TITLE WILL BE SET BY THE PUBLISHER

ρ
Applying a pointwise multiplier theorem [43, Thm. 2.8.2] and the continuous embedding C ρ → B∞∞
, we obtain
up to constants the upper bound

1
µ′
kh′t − h′ kB −3/2−δ +
∞∞
2(µ + tht ) C 7/4+δ/2
2µ(µ + tht )
′
′
ht (µ h − µh )
+t
2µ2 (µ + tht ) M(w)
e

C 7/4+δ/2

kh − ht kB −1/2−δ
∞∞

. kht − hkB −1/2−δ + kh − ht kB −1/2−δ + t . kht − hkM(w) + t,
∞∞

∞∞

where we have used wj 6 2jδ in the last step. The last expression tends to 0 as t → 0 and this shows the
Hadamard differentiability of ξ : Vξ → W.
Step 2: To apply the delta method it is now important that µ
bJn maps into Vξ . Theorem 1.5 gives conditions
such that kb
µJn − µkM(w) = O(n−1/2 ). Provided that 2(9/4+δ/2)Jn wJn n−1/2 = o(1) we deduce from the fact that
µJn − µkC 7/4+δ/2 = o(1).
µ
bJn is developed until level Jn only and from the ratio of the weights at level Jn that kb
We conclude that with probability tending to one µ
b ∈ Vξ . By modifying µ
b on events with probability tending
to zero we can achieve that always µ
b ∈ Vξ . On the above assumptions we obtain the weak convergence
√

n(b
µJn − µ) → Gµ in M0 (w) by Theorem 1.5 and application of the delta method yields the assertion.

5.2. Proof of Lemma 3.2
We will prove that:
(i) for any τ ∈ (0, 1) there are constants 0 < c, C < ∞ depending only on τ, K, ψ such that for any J ∈ Jn
satisfying J > Jn∗ and for all n ∈ N large enough
τ
P (Jbn = J) 6 C(n−cJ + e−cJ ),

(ii) there is an integer M > 0 depending only on d1 , d2 , K and constants 0 < c′ , C ′ < ∞ depending on
τ, K, ψ such that for any J ∈ Jn satisfying J < Jn∗ − M and for all n ∈ N large enough
′ τ
′ 
P (Jbn = J) 6 C ′ n−c J + e−c J .

Given (i) and (ii), we obtain, for a constant c′′ > 0,

P Jbn ∈
/ [Jn∗ − M, Jn∗ ] 6

∗
Jn
−M−1

X

J=Jmin

P (Jbn = J) +

JX
max

∗ +1
J=Jn

P (Jbn = J)



′′
τ
′′
= O (Jmax − Jmin ) n−c (Jmin ) + e−c Jmin


′′
′′ τ
= O log n n−c Jmin + e−c Jmin .

′′

τ

′′

Since n−c Jmin + e−c Jmin decays polynomially in n, the assertion of the lemma follows.
To show (i) and (ii), recall J + = J + Un = J + log2 log n. For notational convenience we define
V (n, j) := (23j j/n)1/2 ,
which is the order of magnitude of the stochastic error for projection level j. Recall that for any f ∈ M ∩ VJ
we can bound
X
X
kf kL∞ ([a,b]) .
2j/2 max |hf, ψj,k i| 6 kf kM
2j/2 wj . kf kM 2J/2 wJ .
j6J

k∈Kj

j6J

28

TITLE WILL BE SET BY THE PUBLISHER
+

+

Since any J ∈ Jn satisfies 2−J (s+1) = o(1) and (log n)2/τ n−1 2J J + = O(1), we conclude from decomposition
(33) as well as from Lemma 4.1 and Lemma 4.4 (applied to wj = j 1/2 2j ) that
bbJ − b = −

X D

j6J,k∈Z

µn − µ,

′ E
ψj,k
ψj,k +
2µ

X

j6J,k∈Z

|

D

′ E
X
ψj,k
ψj,k +
hb, ψj,k iψj,k +RJ
2µ
j>J,k∈Z
} |
{z
}

(Id −πJ + )µ,
{z

=:B µ (J)

for some remainder RJ ∈ VJ with
kRJ kL∞ ([a,b]) = OP n−1 J + 22J

+

Moreover, Lemma 4.3 and Assumption C yield



+ O 2−J

+

(2s+1)

=:B b (J)


+ oP n−1/2 J 1/2 2−3J/2 .



kB µ (J) + B b (J)kL∞ ([a,b]) 6 (d2 + o(1))2−Js .

(37)

+

Using n−1/2 22J J + = o(1) for all J ∈ Jn , we conclude
kbbJ − bkL∞ ([a,b]) 6

X

j6J,k∈Kj

D

µn − µ,

′ E
ψj,k
ψj,k
2µ

L∞ ([a,b])

+ (d2 + o(1))2−Js + oP (V (n, J)).

With this preparation at hand we can proceed similarly as in [18, Lem. 2].
Part (i): For any fixed J > Jn∗ we have
X


P Jbn = J 6
P kbbJ−1 − bbL kL∞ ([a,b]) > K V (n, L) .
L∈Jn ,L>J

As in the derivation of (38) we obtain for n sufficiently large
kbbJ−1 − bbL kL∞ ([a,b]) 6

X

J<j6L,k

′ E
D
ψj,k
µn − µ,
ψj,k
2µ

L∞ ([a,b])


1
+ d2 + 1 (2−(J−1)s + 2−Ls ) + (V (n, J − 1) + V (n, L)).
4

By definition of Jn∗ we have for any L > J > Jn∗ that


∗
K
K
d2 + 1 (2−(J−1)s + 2−Ls ) 6 2(d2 + 1)2−Jns 6 V (n, Jn∗ ) 6 V (n, L).
2
2
Therefore,

P Jbn = J 6

X

P

L∈Jn ,L>J



X

J<j6L,k

D

µn − µ,

′ E
ψj,k
ψj,k
2µ

L∞ ([a,b])

>


K −1
V (n, L) .
2

Analogously to (10) and using (36), Proposition 1.2 yields for any τ ∈ (0, 1) and constants c1 , ..., c4 > 0:


′ E
D
ψj,k
µn − µ,
ψj,k
2µ


K −1
V
(n,
L)
2
L∞ ([a,b])
J<j6L,k∈Kj
r
′ E
 X
D
ψj,k
K − 1 23L L 
>
,
2j/2 max µn − µ,
6P
k∈Kj
2µ
2
n
P

X

J<j6L

>

(38)

29

TITLE WILL BE SET BY THE PUBLISHER

using that

P∞

k=0

2−k/2 6 7/2, we obtain the upper bound
′ E
D
ψj,k
K − 1 1/2 
max n1/2 2−L µn − µ,
>
L
J<j6L,k
2µ
7
′ E

D
X
ψj,k
K − 1 1/2 
6
>
P n1/2 2−j µn − µ,
j
2µ
7
J<j6L,k
X

τ
τ
6 c1
e−c2 j log n + e−c3 j 6 c1 (n−c4 J + e−c4 J ),

P



(39)

J<j6L

′
where we require that K is chosen sufficiently large, depending on kψj,k
/µkL∞ (S) and Σj,k . It remains to sum
this upper bound over all L ∈ Jn with L > J which yields the claim since Jn contains no more than log n
elements.
Part (ii): Let J < Jn∗ − M for some M ∈ N to be specified below. We have



P Jbn = J 6 P kbbJ − bbJn∗ kL∞ ([a,b]) 6 K V (n, Jn∗ ) .

Using Assumption C and the triangle inequality, we obtain similarly to (38), for sufficiently large n,

∗
d1
kbbJ − bbJn∗ kL∞ ([a,b]) > 2−Js − d2 + 1 2−Jn s
2
′ E
D
X
ψj,k
ψj,k
µn − µ,
−
2µ
∗
J<j6Jn ,k∈Kj

L∞ ([a,b])

1
− (V (n, Jn∗ ) + V (n, J)).
4

Owing to J < Jn∗ − M , s > 1 and the definition of Jn∗ , we can bound


∗
1  −(Jn∗ −1)s
d1 −Js
d1
2
2
− d2 + 1 2−Jn s > (d2 + 1)
2M−1 −
2
2(d2 + 1)
2
K
d1
1
>
2M−1 −
V (n, Jn∗ − 1)
4 2(d2 + 1)
2
d1
1
K
V (n, Jn∗ ),
2M−1 −
>
16 2(d2 + 1)
2
where we have used in the last inequality that (Jn∗ − 1)/Jn∗ > 1/2 for n sufficiently large. We conclude
e V (n, J ∗ ) −
kbbJ − bbJn∗ kL∞ ([a,b]) > K
n

X

∗ ,k∈K
J<j6Jn
j

′ E
D
ψj,k
ψj,k
µn − µ,
2µ

L∞ ([a,b])

e := Kd1 2M−1 − K − 1 . Since K
e > K for M large enough, we obtain similarly as in (39) for any
with K
32(d2 +1)
32
2
′
′
τ ∈ (0, 1) and some c , C > 0


P Jbn = J 6 P

X

∗ ,k
J<j6Jn

D

µn − µ,

′ τ
′ 
6 C ′ n−c J + e−c J .

′ E
ψj,k
ψj,k
2µ

L∞ ([a,b])


e − K)V (n, J ∗ )
> (K
n


30

TITLE WILL BE SET BY THE PUBLISHER

5.3. Proof of Corollary 3.4
∗

Owing to (cn/ log n)1/(2s+3) 6 2Jn 6 (Cn/ log n)1/(2s+3) for constants 0 < c < C, we find
log n − log log n
log c
3
log C
3
log n − log log n
+
− 6s6
+
− .
2(log 2)Jn∗
2(log 2)Jn∗
2
2(log 2)Jn∗
2(log 2)Jn∗
2
Since P (Jbn 6 Jn∗ ) → 1 by Lemma 3.2 and due to vn−1 = oP (1), we obtain with probability converging to one
that

v  3
log n − log log n
n
−
> sbn .
− oP
s > max 1,
b
b
2
Jn
2(log 2)(Jn + vn )
Moreover, since P (Jn∗ − M 6 Jbn 6 Jn∗ ) → 1 and vn−1 = oP (1), we have with probability converging to one
s − sbn 6


log n − log log n 
1
log C
3vn
vn
1
−
+
+
. ∗.
2(log 2)Jn∗
1 + vn /Jn∗
2(log 2)Jn∗
2(Jn∗ − M )
Jn



References

[1] Adamczak, R. and Bednorz, W. (2015). Exponential concentration inequalities for additive functionals of
Markov chains. ESAIM Probab. Stat., 19:440–481.
[2] Aı̈t-Sahalia, Y. (2010). Econometrics of Diffusion Models. John Wiley & Sons, Ltd.
[3] Bakry, D., Cattiaux, P., and Guillin, A. (2008). Rate of convergence for ergodic continuous Markov processes:
Lyapunov versus Poincaré. J. Funct. Anal., 254(3):727–759.
[4] Bass, R. F. (1998). Diffusions and elliptic operators. Springer, New York.
[5] Bull, A. D. (2012). Honest adaptive confidence bands and self-similar functions. Electron. J. Stat., 6:1490–
1516.
[6] Castellana, J. V. and Leadbetter, M. R. (1986). On smoothed probability density estimation for stationary
processes. Stochastic Process. Appl., 21(2):179–193.
[7] Castillo, I. and Nickl, R. (2013). Nonparametric Bernstein–von Mises theorems in Gaussian white noise.
Ann. Statist., 41(4):1999–2028.
[8] Castillo, I. and Nickl, R. (2014). On the Bernstein–von Mises phenomenon for nonparametric Bayes procedures. Ann. Statist., 42(5):1941–1969.
[9] Chauveau, D. and Diebolt, J. (2003). Estimation of the asymptotic variance in the CLT for Markov chains.
Stoch. Models, 19(4):449–465.
[10] Chen, X. (1999). Limit theorems for functionals of ergodic Markov chains with general state space. Mem.
Amer. Math. Soc., 139(664).
[11] Chernozhukov, V., Chetverikov, D., and Kato, K. (2014). Anti-concentration and honest, adaptive confidence bands. Ann. Statist., 42(5):1787–1818.
[12] Chorowski, J. and Trabs, M. (2016). Spectral estimation for diffusions with random sampling times.
Stochastic Process. Appl., doi 10.1016/j.spa.2016.03.009.
[13] Comte, F., Genon-Catalot, V., and Rozenholc, Y. (2007). Penalized nonparametric mean square estimation
of the coefficients of diffusion processes. Bernoulli, 13(2):514–543.
[14] Dalalyan, A. (2005). Sharp adaptive estimation of the drift function for ergodic diffusions. Ann. Statist.,
33(6):2507–2528.
[15] Galtchouk, L. and Pergamenshchikov, S. (2014). Geometric ergodicity for classes of homogeneous Markov
chains. Stochastic Process. Appl., 124(10):3362–3391.
[16] Geyer, C. J. (1992). Practical Markov chain Monte Carlo. Statistical Science, 7(4):473–483.
[17] Gihman, I. I. and Skorohod, A. V. (1972). Stochastic differential equations. Springer, Heidelberg.
[18] Giné, E. and Nickl, R. (2010). Confidence bands in density estimation. Ann. Statist., 38(2):1122–1170.
[19] Giné, E. and Nickl, R. (2015). Mathematical foundations of infinite-dimensional statistical models. Cambridge Series in Statistical and Probabilistic Mathematics. Cambridge University Press.

TITLE WILL BE SET BY THE PUBLISHER

31

[20] Gobet, E., Hoffmann, M., and Reiß, M. (2004). Nonparametric estimation of scalar diffusions based on low
frequency data. Ann. Statist., 32(5):2223–2253.
[21] Gugushvili, S. and Spreij, P. (2014). Nonparametric Bayesian drift estimation for multidimensional stochastic differential equations. Lith. Math. J., 54(2):127–141.
[22] Hansen, L. P. and Scheinkman, J. A. (1995). Back to the future: generating moment implications for
continuous-time Markov processes. Econometrica, 63(4):767–804.
[23] Hansen, L. P., Scheinkman, J. A., and Touzi, N. (1998). Spectral methods for identifying scalar diffusions.
J. Econometrics, 86(1):1–32.
[24] Härdle, W., Kerkyacharian, G., Picard, D., and Tsybakov, A. (1998). Wavelets, approximation, and
statistical applications, volume 129 of Lecture Notes in Statistics. Springer-Verlag, New York.
[25] Hoffmann, M. (1999). Adaptive estimation in diffusion processes. Stochastic Process. Appl., 79(1):135–163.
[26] Hoffmann, M. and Nickl, R. (2011). On adaptive inference and confidence bands. Ann. Statist., 39(5):2383–
2409.
[27] Kristensen, D. (2010). Pseudo-maximum likelihood estimation in two classes of semiparametric diffusion
models. Journal of Econometrics, 156(2):239–259.
[28] Kutoyants, Y. A. (2004). Statistical inference for ergodic diffusion processes. Springer Series in Statistics.
Springer-Verlag London, Ltd., London.
[29] Lacour, C. (2008). Nonparametric estimation of the stationary density and the transition density of a
Markov chain. Stochastic Process. Appl., 118(2):232–260.
[30] Ledoux, M. (2001). The concentration of measure phenomenon, volume 89 of Mathematical Surveys and
Monographs. American Mathematical Society, Providence, RI.
[31] Löcherbach, E., Loukianova, D., and Loukianov, O. (2011). Penalized nonparametric drift estimation for
a continuously observed one-dimensional diffusion process. ESAIM Probab. Stat., 15:197–216.
[32] Low, M. G. (1997). On nonparametric confidence intervals. Ann. Statist., 25(6):2547–2554.
[33] Masuda, H., Negri, I., and Nishiyama, Y. (2011). Goodness-of-fit test for ergodic diffusions by discrete-time
observations: an innovation martingale approach. J. Nonparametr. Stat., 23(2):237–254.
[34] Meyn, S. P. and Tweedie, R. L. (2009). Markov chains and stochastic stability. Cambridge University
Press, Cambridge.
[35] Negri, I. and Nishiyama, Y. (2010). Goodness of fit test for ergodic diffusions by tick time sample scheme.
Stat. Inference Stoch. Process., 13(1):81–95.
[36] Nickl, R. and Söhl, J. (2015). Nonparametric Bayesian posterior contraction rates for discretely observed
scalar diffusions. arXiv preprint arXiv:1510.05526.
[37] Robert, C. P. (1995). Convergence control methods for Markov chain Monte Carlo algorithms. Statistical
Science, 10(3):231–253.
[38] Rosenblatt, M. (1970). Density estimates and Markov sequences. In Nonparametric Techniques in Statistical
Inference (Proc. Sympos., Indiana Univ., Bloomington, Ind., 1969), pages 199–213. Cambridge Univ. Press,
London.
[39] Roussas, G. G. (1969). Nonparametric estimation in Markov processes. Ann. Inst. Statist. Math., 21:73–87.
[40] Schmisser, E. (2013). Nonparametric estimation of the derivatives of the stationary density for stationary
processes. ESAIM Probab. Stat., 17:33–69.
[41] Spokoiny, V. G. (2000). Adaptive drift estimation for nonparametric diffusion model. Ann. Statist.,
28(3):815–836.
[42] Szabó, B., van der Vaart, A. W., and van Zanten, J. H. (2015). Frequentist coverage of adaptive nonparametric Bayesian credible sets (with discussion). Ann. Statist., 43(4):1391–1428.
[43] Triebel, H. (1983). Theory of function spaces, volume 78 of Monographs in Mathematics. Birkhäuser Verlag,
Basel.
[44] van der Meulen, F. and van Zanten, H. (2013). Consistent nonparametric Bayesian inference for discretely
observed scalar diffusions. Bernoulli, 19(1):44–63.

32

TITLE WILL BE SET BY THE PUBLISHER

[45] van der Vaart, A. and van Zanten, H. (2005). Donsker theorems for diffusions: necessary and sufficient
conditions. Ann. Probab., 33(4):1422–1451.
[46] van der Vaart, A. W. (1998). Asymptotic statistics, volume 3 of Cambridge Series in Statistical and
Probabilistic Mathematics. Cambridge University Press, Cambridge.
[47] Yakowitz, S. (1989). Nonparametric density and regression estimation for Markov sequences without mixing
assumptions. J. Multivariate Anal., 30(1):124–136.

