ON FREQUENTIST COVERAGE ERRORS OF BAYESIAN CREDIBLE SETS
IN HIGH DIMENSIONS

arXiv:1803.03450v1 [] 9 Mar 2018

KEISUKE YANO AND KENGO KATO
Abstract. In this paper, we study frequentist coverage errors of Bayesian credible sets for an
approximately linear regression model with (moderately) high dimensional regressors, where the
dimension of the regressors may increase with but is smaller than the sample size. Specifically, we
consider Bayesian inference on the slope vector by fitting a Gaussian distribution on the error term
and putting priors on the slope vector together with the error variance. The Gaussian specification
on the error distribution may be incorrect, so that we work with quasi-likelihoods. Under this
setup, we derive finite sample bounds on frequentist coverage errors of Bayesian credible rectangles.
Derivation of those bounds builds on a novel Berry–Esseen type bound on quasi-posterior distributions and recent results on high-dimensional CLT on hyper-rectangles. We use this general result to
quantify coverage errors of Castillo–Nickl and L∞ -credible bands for Gaussian white noise models,
linear inverse problems, and (possibly non-Gaussian) nonparametric regression models. In particular, we show that Bayesian credible bands for those nonparametric models have coverage errors
decaying polynomially fast in the sample size, implying advantages of Bayesian credible bands over
confidence bands based on extreme value theory.

1. Introduction
Bayesian inference for high or nonparametric statistical models is an active research area in
the recent statistics literature. Posterior distributions provide not only point estimates but also
credible sets. In a classical regular statistical model with a fixed finite dimensional parameter
space, it is well known that the Bernstein–von Mises (BvM) theorem holds under mild conditions
and the posterior distribution can be approximated (under the total variation distance) by a normal
distribution centered at an efficient estimator (e.g. MLE) and with covariance matrix identical to
the inverse of the Fisher information matrix as the sample size increases. The BvM theorem implies
that a Bayesian credible set is typically a valid confidence set in the frequentist sense, namely, the
coverage probability of a (1 − α)-Bayesian credible set evaluated under the true parameter value
is approaching (1 − α) as the sample size increases. There exists a literature on the frequentist
behavior of Bayesian credible sets in a nonparametric statistical model. Freedman [22] gave the
negative result for the BvM theorem in infinite Gaussian sequence models with Gaussian priors;
Johnstone [31], Leahu [33], and Bontemps [7] discovered the conditions under which the BvM
theorem holds when using Gaussian priors. Recently, Castillo and Nickl [9] and [10] established the
BvM theorem using weaker topologies than ℓ2 .
Date: March 12, 2018.
Key words and phrases. Castillo-Nickl band, credible rectangle, sieve prior.
K. Yano is supported by the Grant-in-Aid for Research Activity start-up (17H06570) from the JSPS. K. Kato is
supported by the Grant-in-Aid for Scientific Research (C) (15K03392) from the JSPS.
1

This paper aims at studying frequentist coverage errors of Bayesian credible rectangles in an
approximately linear regression model with an increasing number of regressors. We provide finite
sample bounds on frequentist coverage errors of (quasi-)Bayesian credible rectangles based on sieve
priors, where the model includes a unknown bias term, the error variance is also unknown, and
the distribution of the error term may not be Gaussian. Sieve priors are prior distributions on the
slope vectors where the dimension of the regressors may increase. We allow sieve priors to be nonGaussian or not to be an independent product. We go through “quasi-Bayesian” approach because
we fit a Gaussian distribution on the error term but we do not specify Gaussian distributions as
the distribution of the error term. The resulting posterior distribution is called “quasi-posterior.”
An important application of our results is finite sample quantification of Bayesian nonparametric
credible bands based on sieve priors. We presents finite sample bounds of Castillo–Nickl and L∞ credible bands 1 in Gaussian white noise models, in linear inverse problems, and in (possibly nonGaussian) nonparametric regression models. There is a literature on nonparametric credible bands.
Studies of frequentist approaches go back to Bickel, Rosenblatt, and Smirnov [6, 42]. More recent
studies are available in [13, 18, 26]. Studies of Bayesian approaches are relatively few. In particular,
there is a limited literature on quantification of frequentist coverage errors of nonparametric credible
bands. Castillo and Nickl [10] showed that for Gaussian white noise models Castillo–Nickl credible
bands based on product priors have asymptotically optimal frequentist coverage. See also [39] for
adaptive Castillo–Nickl credible bands based on a spike and slab prior. Yoo and Ghosal [49] showed
that Castillo–Nickl credible bands based on Gaussian series priors have the asymptotically optimal
frequentist coverage in (possibly sub-Gaussian) nonparametric regression models. All there work
built upon asymptotic results. Recently, Yang et al. [48] obtained a non-asymptotic result using
Gaussian process priors. Yet, quantification of frequentist coverage errors of nonparametric credible
bands based on general priors had been limited.
Our results have an implication supporting Bayesian approach to constructing nonparametric
confidence bands. In the literature on nonparametric confidence bands, it is known that confidence
bands based on extreme value theory are not satisfactory due to the slow convergence of Gaussian
maxima. Hall [28] showed that bootstrap confidence bands have coverage errors decalying polynomially fast in the sample size. A more general discussion on the fast convergence of coverage errors
of bootstrap confidence bands is available in [13]. An alternative to bootstrap approach is Bayesian
approach. Castillo and Nickl [10] vaguely discussed the advantage of Bayesian credible bands over
confidence bands based on extreme value theory. Our result shows that Bayesian credible bands
have also coverage errors comparable to bootstrap confidence bands, yielding the advantage over
confidence bands based on extreme value theory. See Remark 3.2.
The main ingredients of the derivation are (i) a novel Berry–Esseen type bound for the BvM
theorem in case with sieve priors, that is, a finite sample bound on the total variation distance
between the quasi-posterior distribution based on sieve priors and the Gaussian distribution and
(ii) recent results on high dimensional CLT on hyper-rectangles. Our Berry–Esseen type bound
1In the present paper, a nonparametric credible band whose L∞ -diameter is bounded both below and above is

called an L∞ -credible band. A nonparametric credible band whose L∞ -diameter is bounded only above is called a
Castillo–Nickl-credible band.
2

opens the door to non-asymptotic treatment of posteriors going beyond conjugate analyses. The
high dimensional CLT on hyper-rectangles is developed by Chernozhukov et al. [12, 16] and is used
to approximate in hyper-rectangle regions the sampling distribution by the Gaussian distribution.
This enables us to get rid of any assumption on the error distribution except assumptions on the
moment of the error distribution.
1.1. Literature review and contributions. Closely related are [48, 49]. For nonparametric
regression model, Yang et al. [48] obtained finite sample bounds on frequentist coverage errors of
Bayesian credible bands based on Gaussian process priors. They worked with (a) Gaussian process
priors, (b) the assumption that the error terms follows a sub-Gaussian distribution, and (c) the
assumption that the error variance is known. The present paper differs from [48] in that
• we work with possibly non-Gaussian priors;
• we allow a more flexible assumption on the distribution of the error terms;
• we allow the error variance to be unknown.
To deal with non-Gaussian priors, we develop novel Berry–Esseen type bounds on quasi-posterior
distributions. To weaken the moment assumption on error terms, we introduce a recently-developed
probabilistic tool “high-dimensional CLT on hyper rectangles.” In the case with the unknown error
variance, the quasi-posterior contraction for a prior on a variance affects on the coverage error and
thus a more careful treatment is conducted. Yoo and Ghosal [49] worked with (a) Gaussian series
priors, (b) the assumption that the error terms follows a sub-Gaussian distribution, and (d) the
requirement that the nominal coverage level (1 − α) tends to 1 as the sample size grows. The
requirement (d) indicates that their result is available only in the large sample asymptotics.
The present paper works with the BvM theorem in nonparametric statistics. There exists a vast
literature on the BvM theorem. The BvM theorem in Gaussian white noise model is studied by
[9, 10, 22, 31, 33, 39]. The BvM theorem in linear regression with high dimensional regressors is
studies by [7, 24]. The BvM theorem in nonparametric regression with Gaussian process prior is
studies by [48, 49]. For the BvM theorem in the other nonparametric models, see also [8, 11, 23, 25,
40]. The present paper goes through quasi-Bayesian approach because we do not use the Gaussian
assumption on error terms. The BvM theorem for quasi-posterior distribution was developed by
[3, 17, 21, 32, 34].
Importantly, our Berry–Esseen type bound produces a substantially better result in a critical
dimension under which the BvM theorem holds. Ghosal [24], Bontemps [7], and Spokoiny [43]
investigated critical dimensions when using sieve priors. The result in Bontemps [7] did not cover
the case with an unknown error variance; the results in [24, 43] covered the case with the unknown
error variance. Our result is consistent to the result in [7] in the case with the known error variance.
Our result substantially improves the results in [24, 43] in the case with the unknown error variance.
The results in [24, 43] typically indicated that the critical dimension is p3 = o(n) in the case with the
unknown error variance, while our result indicates that the critical dimension is p2 (log n)3 = o(n),
where p is the number of the regressor and n is the sample size. For the detailed comparison, see
Remark 2.2. Our obtained critical dimension is sufficient for the use in nonparametric regression
model with the unknown error variance; see Section 3.3.
3

1.2. Organization and notation. Let k · k denote the Euclidean norm. Let k · k∞ denote the
max or supremum norm for vectors and functions. For a set S of vectors or functions , let kSk∞
denote the supremum of the max or supremum norms of differences between any two elements in
S. Let N (µ, Σ) denote the Gaussian distribution with mean vector µ and covariance matrix Σ. For
any x ∈ R, let x+ = max{x, 0}. For two sequences {an } and {bn } depending on n, the notation
an . bn signifies that an 6 cbn for some universal constant c > 0. The notation an ∼ bn signifies
that an . bn and bn . an . For any positive semi-definite matrices A and B, the notation A  B
signifies that B − A is positive semi-definite. Throughout the paper, constants c1 , c2 , . . ., c, and
e
c1 , e
c2 do not depend on sample size n and dimension p. The values of c1 , c2 , . . ., c, and e
c1 , e
c2 are
different in each theorem, proposition, and proof.
2. Bayesian credible rectangles in high dimensions
Consider an approximately linear regression model
Y = Xβ0 + r + ε,

(1)

where Y = (Y1 , . . . , Yn )⊤ ∈ Rn is a vector of outcome variables, X is an n × p design matrix,
β0 ∈ Rp is an unknown coefficient vector, r = (r1 , . . . , rn )⊤ ∈ Rn is a deterministic (i.e., nonrandom) bias term, and ε = (ε1 , . . . , εn )⊤ ∈ Rn is a vector of i.i.d. error terms with mean zero and
variance 0 < σ02 < ∞. We are primarily interested in the situation where the number of regressors p
increases with the sample size n, i.e., p = pn → ∞ as n → ∞, but we often suppress the dependence
on n for the sake of notational simplicity. In addition, we allow the error variance σ02 to depend
2 , which allows us to include Gaussian white noise models in the subsequent
on n, i.e., σ02 = σ0,n
analysis as a special case. In the general setting, the error variance σ02 is also unknown. In the
present paper, we work with the dense model with moderately high-dimensional regressors where
β0 need not be sparse and p = pn may increase with the sample size n but p < n. To be precise,
we will maintain the assumption that the design matrix X is of full column rank, i.e., rank X = p.
The approximately linear model (1) is flexible enough to cover various nonparametric models such
as Gaussian white noise models, linear inverse problems, and nonparametric regression models, via
series expansions of functions of interest in those nonparametric models; see Section 3.
We consider Bayesian inference on the slope vector β0 . To this end, we fit a Gaussian distribution
on the error ε, but we allow the Gaussian specification on the error distribution to be incorrect.
Namely, we work with the quasi -likelihood of the form
2 /(2σ 2 )

(β, σ 2 ) 7→ (2πσ 2 )−n/2 e−kY −Xβk

.

We assume independent priors on β and σ 2 , i.e.,
β ∼ Πβ , σ 2 ∼ Πσ2 , β ⊥
⊥ σ2,

(2)

where we assume that Πβ is absolutely continuous with density π, i.e., Πβ (dβ) = π(β)dβ, and Πσ2
is supported in (0, ∞). Then the resulting quasi-posterior distribution for (β, σ 2 ) is
2 /(2σ 2 )

Π(d(β, σ 2 ) | Y ) ∝ (2πσ 2 )−n/2 e−kY −Xβk
4

π(β)dβΠσ2 (dσ 2 ),

and the marginal quasi-posterior distribution for β is Πβ (dβ | Y ) = π(β | Y )dβ, where
Z
2
2
e−kY −Xβk /(2σ )
π(β | Y ) = π(β) R
Π 2 (dσ 2 | Y ),
e 2 /(2σ2 )
e βe σ
e−kY −X βk
π(β)d

and Πσ2 (dσ 2 | Y ) denotes the quasi-posterior distribution for σ 2 . We will assume that Πσ2 may be
b2 of σ 2 (in that case, Πσ2 (· | Y ) = δσb2 ), but
data-dependent, e.g., Πσ2 = δσb2 for some estimator σ
Πβ is data-independent.
We will derive finite sample bounds on frequentist coverage errors of Bayesian credible rectangles
for the approximately linear model (1) under a prior of the form (2). For given c = (c1 , . . . , cp )⊤ ∈
Rp , R > 0, and positive sequence {wj }pj=1 , let I(c, R) denote the hyper-rectangle of the form


⊤
p |βj − cj |
6 R, 1 6 ∀j 6 p .
I(c, R) := β = (β1 , . . . , βp ) ∈ R :
wj

b ) = (X ⊤ X)−1 X ⊤ Y . For given α ∈ (0, 1), we
Let βb denote the OLS estimator for β0 , i.e., βb = β(Y
b R
bα ), where the radius R
bα is chosen in such a
consider a (1 − α)-credible rectangle of the form I(β,
b R
b R
bα ) is 1 − α, i.e., Πβ {I(β,
bα ) | Y } = 1 − α.
way that the posterior probability of the set I(β,
We make the following conditions on Πβ and Πσ2 . For R > 0, let
B(R) := {β ∈ Rp : kX(β − β0 )k 6 Rσ0 } and

φΠβ (R) := 1 −

Condition 2.1. There exists a positive constant C1 such that
q
−p
π(β0 ) > σ0
det(X ⊤ X)e−C1 p log n .

inf

e
β,β∈B(R)

e
{π(β)/π(β)}.

(3)

Condition 2.2. There exist non-negative constants δ1 , δ2 , δ3 ∈ [0, 1) such that with probability at


least 1 − δ3 , Πσ2 σ 2 : σ 2 /σ02 − 1 > δ1 | Y 6 δ2 .
√
Condition 2.3. The inequality φΠβ (1/ n) 6 1/2 holds.

Condition 2.1 imposes Πβ to put a sufficient mass around β0 . Condition 2.2 is a marginal
posterior contraction of Πσ2 . Condition 2.3 is a preliminary flatness condition on Πβ . The detailed
discussion on these conditions are provided after the main theorem.
We also make the assumptions on the model.
√
Assumption 2.1. There exists a positive constant C2 such that kX(X ⊤ X)−1 X ⊤ rk 6 C2 σ0 p log n.
Assumption 2.2. There exists a positive constant C3 for which either one of the following conditions holds.
(a) E[|ε1 /(σ0 C3 )|q ] 6 2 for some integer q > 2;
(b) E[exp{ε21 /(σ0 C3 )2 }] 6 2.
Assumption 2.1 controls the norm of the bias term. Assumption 2.2 is a moment assumption
on the error terms. These assumptions are substantially weak and satisfied in all applications we
present.
5

The following theorem, which is the main result of this section, provides bounds on frequentist
b R
bα ) together with bounds on the maxcoverage errors of the Bayesian credible rectangle I(β,
b R
bα ). In what follows, let λ and λ denote the maximum and minimum eigenvalues
diameter of I(β,
of (X ⊤ X)−1 , respectively, and let w := max{w1 , . . . , wp } and w := min{w1 , . . . , wp }.
Theorem 2.1 (Coverage errors of credible rectangles). Suppose that Conditions 2.1–2.3, Assumption 2.1, and either of Assumption 2.2 (a) or (b) hold. Then, there exist positive constants c1 , . . . , c4
depending only on C1 , C2 , C3 and q appearing in Condition 2.1 and Assumptions 2.1 and 2.2 (a)
such that the following hold. For every n > 2, we have that
n
o
bR
bα ) − (1 − α)
P β0 ∈ I(β,


(4)

 p
τ p
log p + ωn
6 φΠβ c1 p log n + c1 δ1 p log n + δ2 + δ3 +
σ0 λ1/2
where τ := k(X ⊤ X)−1 X ⊤ rk∞ and

1/6 
1/3

7
3
λ p log (pn)
1−q/2 (log n)−q/2 + λ p log (pn)

+
p

1−2/q

λ
n
λ n

1/6

7
p
log
(pn)
λ
−c
p
log
n
ωn = e 2
+ λ
n



 −c2 p log n
e

under Assumption 2.2 (a),
under Assumption 2.2 (b),
if εi ’s are Gaussian;

In addition, provided that the right hand side on (4) is smaller than α/2, for sufficiently large p
bR
bα ) is bounded as
depending only on α, the max-diameter of I(β,
p
p
b R
bα )|∞ 6 c4 σ0 λ1/2 w log p
c3 σ0 λ1/2 w log p 6 |I(β,

with probability at least

1 − c p1−q/2 (log n)−q/2 − δ
1
3
−c
p
log
n
1 − c1 e 2
− δ3

under Assumption 2.2 (a),
under Assumption 2.2 (b).

Theorem 2.1 indicates that the frequentist coverage error of the Bayesian credible rectangle is
controlled by the flatness function φΠβ up to terms depending on the prior Πβ on β. The discussions
below provide a typical bound on φΠβ .
2.1. Discussions on conditions. First, we verify that a locally log-Lipschitz prior meets Conditions 2.1 and 2.3, providing an upper bound of φΠβ .
Definition 2.1. A locally log-Lipschitz prior is defined as a prior distribution on β that there
exists L = Ln > 0 for which the inequality | log π(β) − log π(β0 )| 6 Lkβ − β0 k holds for β such that
1/2 √
p log n.
kβ − β0 k 6 σ0 λ
The following proposition shows that a locally log-Lipschitz prior meets Condition 2.3. It also
provides an upper bound of φΠβ with a locally log-Lipschitz prior.
Proposition 2.1. For a locally log-Lipschitz prior Πβ with a log-Lipschitz constant L, the inequality
√
1/2 √
φΠβ (c p log n) 6 cLσ0 λ
p log n holds for any c > 0. Further, the prior Πβ satisfies Condition
1/2 √
2.3 provided that σ0 Lλ / n 6 1/2.
6

For the verification of Condition 2.1, we focus on the following two subclasses of locally logLipschitz priors. Let B := kβ0 k.
R
(Isotropic prior) An isotropic prior is a prior of the form π(β) = ρ(kβk)/ ρ(kβk)dβ with a proba1/2 √
bility density ρ on R+ such that ρ is strictly positive on [0, B + σ0 λ
p log n] and
1/2 √
continuously differentiable on [0, B + σ0 λ
p log n], and such that there exists a
R∞ k
positive constant m for which 0 x ρ(x)dx is bounded above by emk log k for any
k ∈ N;
Q
(Product prior) A product prior of log-Lipschitz priors is of the form π(β) = pi=1 πi (βi ) of which
1/2 √
e
for
log πi for each i is strictly positive on [0, B + σ0 λ
p log n] and L-Lipschitz
e > 0.
some L

For expositional simplicity, we use the following assumption to verify that isotropic or product
priors meet Condition 2.1.
Assumption 2.3. There exists a universal positive constant c for which we have, for every n > 2,
p
log{ det(X ⊤ X)/σ0p } 6 c log n.

This assumption ensures that X ⊤ X/σ02 is not much expanded in n. The assumption is satisfied
in case that the column vectors of X/σ0 are nearly orthonormal in Rp . The assumption is also
satisfied in all applications we present.
The following proposition ensures that isotropic or product priors are locally log-Lipschitz priors
satisfying Condition 2.1.
Proposition 2.2. Under Assumption 2.3, an isotropic prior and a product prior of log-Lipschitz
priors satisfy Condition 2.1. An isotropic prior is a locally log-Lipschitz prior with locally logLipschitz constant L such that
L 6 c1 B

max

1/2 √

x:06x6B+σ0 λ

p log n

|d log ρ/dx(x)|

for some positive constant c1 depending only on m and c appearing in the definition of ρ and
Assumption 2.3. In particular, for a standard Gaussian distribution, L 6 c1 B 2 . A product prior of
e is a locally log-Lipschitz with L = Lp
e 1/2 .
log-Lipschitz priors with a log-Lipschitz constant L
Next, we provide discussions on Condition 2.2. We consider following two cases:

(Plug-in) Πσ2 = Πσbu2 with σ
bu2 (Y ) := kY − X(X ⊤ X)−1 X ⊤ Y k2 /(n − p);
(Full Bayes) Πβ is a standard Gaussian distribution and Πσ2 is the inverse Gamma distribution
IG(µ1 , µ2 ) with shape parameter µ1 > 1/2 and scale parameter µ2 > 1/2.
The following two propositions provide possible choices of δ1 , δ2 , and δ3 . For simplicity, we
assume that σ0 is a constant independent of n. Let δe1 := δ1 − 2krk2 /{σ02 (n − p)} − 1/(n − p) > 0.

Proposition 2.3. Suppose that n > cp for some c > 1. Then, there exist positive constants c1 and
c2 depending only on c, C3 and q appearing in Assumption 2.2 such that

c max{n−4/q δ−q/2 , n1−q/2 δe−q } under Assumption 2.2 (a),

1
1
1
P |b
σu2 /σ02 − 1| > δ1 6
c1 exp(−c2 n max{δ2 , δe2 })
under Assumption 2.2 (b).
1

7

1

Proposition 2.4. Suppose that n > cp for some c > 1. Then, there exist positive constants c1 and
c2 depending only on c, µ1 , µ2 , C3 and q appearing in Assumption 2.2 such that the inequality
Πσ2 (σ 2 : |σ 2 /σ02 − 1| > δ1 | Y ) 6 c1 (nδe1 )−1

holds with probability at least

1 − c max{n−4/q δ−q/2 , n1−q/2 δe−q }
1
1
1
1 − c1 exp{−c2 n max{δ2 , δe2 }}
1

1

under Assumption 2.2 (a),
under Assumption 2.2 (b).

For a better understanding, Table 1 summarizes the results using the asymptotics in n in both
cases when n > cp for some c > 0 and krk2 /n = o(n−1/2 ).
Table 1. Possible orders of δ1 , δ2 , δ3 with respect to n: κ is arbitrary
Assumption 2.2 and prior
(a) and plug-in
(a) and full Bayes
(b) and plug-in
(b) and full Bayes

δ1
−1/2+κ/q
n

δ2
0

δ3
−κ/2
max{n
, n1−κ }

n−1/2+κ/q
n−1/2−κ/q
max{n−κ/2 , n1−κ }
√
n−1/2 log n
0
n−1
√
n−1/2 log n n−1/2 (log n)−1/2
n−1

Remark 2.1 (Compatibility with the result in [49]). In the case with the Gaussian prior for β and
a sub-Gaussian distributon for ε, the possible covergence rate of δ1 was investigated by Yoo and
Ghosal [49]; See Proposition 4.1 in [49]. For the choice of δ1 , our results in Propositions 2.3 and
2.4 are compatible with their result up to a logarithmic factor.
2.2. Berry–Esseen type bounds on posterior distributions. Before providing several applications of the main theorem, we present an important ingredient for the proof of Theorem 2.1,
namely, the Berry–Esseen type bound on posterior distributions. This bound substantially improves
on the previous work related to the BvM theorem, as discussed below. For R > 0, let
√


p log nσ0
n
b
∩ {Y ∈ Rn : Πσ2 (|σ 2 /σ02 − 1| > δ1 | Y ) 6 δ2 }.
H(R) := Y ∈ R : kX(β(Y ) − β0 )k 6 R
4
For two probability measures P and Q, kP − QkTV denotes the total variation between P and Q.

Proposition 2.5 (Berry–Esseen type bounds on posterior distributions). Under Conditions 2.1–
2.3, there exist positive constants c1 and c2 depending only on C1 , C2 , C3 such that for every n > 2
and for Y ∈ H(c1 ), the inequality
 p

b σ 2 (X ⊤ X)−1 )
p
log
n
6
φ
Πβ (· | Y ) − N (β,
c
+ c1 (δ1 p log n + δ2 + e−c2 p log n )
Π
1
0
β
TV

holds.

Proposition 2.6. Assume that Assumption 2.1 holds. Then, there exist positive constants c1 and
c2 depending only on C2 , C3 , and q appearing in Assumptions 2.1-2.2 such that

c p1−q/2 (log n)−q/2 + δ
under Assumption 2.2 (a),
1
3
P(Y ∈
/ H(c1 )) 6
c1 exp(−c2 p log n) + δ3
under Assumption 2.2 (b).
8

Remark 2.2 (Critical dimension of the Bernstein–von Mises theorem). The previous propositions
clarify the critical dimension under which the BvM theorem holds. We compare our result with
the results on the critical dimension by [7, 25, 43]. In the case that kβ0 k is independent of n and
1/2
σ0 λ
∼ n−1/2 , the comparison is demonstrated using a locally log-Lipschitz prior with locally
log-Lipschitz constant L independent of n. The followings are the summary of the previous works:
• Ghosal [25] showed that when the distribution of i.i.d. error terms εi s has a smooth density
function with known variance, the BvM theorem holds if p4 log p = o(n) and some additional
assumptions hold;
• Bontemps [7] showed that when the distribution of i.i.d. error terms is Gaussian with known
variance, the BvM theorem holds if p log n = o(n) 2;
• Spokoiny [43] showed that when the high-dimensional local asymptotic normality holds, the
BvM theorem holds if p3 = o(n). See also [37].
Our result (Propositions 2.1, 2.3, 2.5, and 2.6) improves the previous work from the following
viewpoints:
• In the case with known variance, our result shows that when the distribution of i.i.d. error
terms has the third order moment, the BvM theorem (for quasi-posterior distributions)
holds if p log n = o(n). Comparing our result with [25] indicates that borrowing the Gaussian likelihood conducts a substantial improvement on the critical dimension. When the
distribution of error terms is Gaussian, our result is consistent to [7];
• Our result covers the case with unknown variance. This makes the substantial difference
between our result and [7]. Taking the care of the case with unknown variance is important
in application including nonparametric regression models. In the case with unknown variance, our result shows that when the distribution of i.i.d. error terms is sub–Gaussian, the
BvM theorem holds for β if p2 (log n)3 = o(n). Comparing our result with [43], our result
provides a substantial improvement on the critical dimension.
3. Applications
In this section, we consider applications of the general results developed in the previous sections
to quantifying coverage errors of Bayesian credible sets in Gaussian white noise models, linear
inverse problems, and (possibly non-Gaussian) nonparametric regression models.
3.1. Gaussian white noise model. We first consider a Gaussian white noise model and analyze
coverage errors of Castillo-Nickl and L∞ -credible bands. Consider a Gaussian white noise model
1
dY (t) = f0 (t)dt + √ dW (t), t ∈ [0, 1],
n
where dW is a canonical white noise and f0 is an unknown function. We assume that f0 is in the
s
Hölder–Zygmund space B∞,∞
with smoothness level s for some s > 0. It will be convenient to
2Bontemps [7] uses the condition that there exists M such that M = O(n) and p log p = o(M ) and it seems to
n
n
n

exclude the setting that p log n = o(n) if p is of polynomial order with respect to n because we cannot set Mn = p log n.
However, replacing the asymptotic evaluation at the final step in the proof of Proposition 9 in [7] by the finite sample
evaluation and multiplying Mn by a sufficiently large positive constant, the setting that p log n = o(n) is allowed.
9

s
define the Hölder–Zygmund space B∞,∞
using a wavelet basis. Let S > s be an integer and fix
sufficiently large J0 = J0 (S). Let {φJ0 ,k : 0 6 k 6 2J0 − 1} ∪ {ψl,k : J0 6 l, 0 6 k 6 2l − 1} be
an S-regular Cohen–Daubechies–Vial (CDV) wavelet basis of L2 [0, 1]. Then the Hölder–Zygmund
s
space B∞,∞
is defined by
(
)
s
B∞,∞
=

s
f : kf kB∞,∞
:=

max

06k62J0 −1

|hφJ0 ,k , f i| +

sup

J0

6l<∞,06k62l −1

2l(s+1/2) |hψl,k , f i| < ∞ ,

R
where h·, ·i denotes the L2 [0, 1] inner product, i.e., hf, gi := [0,1] f (t)g(t)dt. For the notational
convention, let ψJ0 −1,k := φJ0 ,k for 0 6 k 6 2J0 − 1 and let I(J) := {(l, k) : J0 6 l 6 J − 1, 0 6 k 6
2l − 1} ∪ {(l, k) : l = J0 − 1, 0 6 k 6 2J0 − 1} for J > J0 .
3.1.1. Castillo–Nickl credible bands. The Castillo–Nickl credible band is defined as
(
)
b, ψl,k i|
|hf
−
f
√
C(fb, R) := f : max
6R
for R > 0,
(l,k)∈I(J)
l

where J = Jn > J0 is taken in such a way that 2Jn = (n/ log n)1/(2s+1) un for a divergent sequence un
R
P
bα ) the (1 − α)-Castillo–
and fb :=
ψl,k ψl,k dY. For a given prior Πf on f , we call C(fb, R
(l,k)∈I(J)

bα is chosen in such a way that Πf {C(fb, R
bα ) | Y } = 1 − α.
Nickl credible band, where the radius R
J
∞
2
We consider a sieve prior Πf on L [0, 1] induced from a prior Πβ on R via the map
X
(βJ0 ,0 , βJ0 ,1 , . . . , βJ−1,2J −1 −1 ) 7→
ψl,k (·)βl,k .
(l,k)∈I(J)

The following theorem establishes bounds on coverage errors of Castillo–Nickl credible bands.
P
Let τ∞ := kf0 − (l,k)∈I(J) ψl,k β0,lk k∞ with β0,lk = hf0 , ψl,k i for (l, k) ∈ I(J).

Proposition 3.1. Under Conditions 2.1 and 2.3 for Πβ that corresponds to Πf and under the
p
assumption that τ∞ 6 C2′ 2J (log n)/n for some C2′ > 0, there exist positive constants c1 , c2 , c3
depending only on C1 appearing in Condition 2.1 and C2′ such that the following hold: For n > 2
s
satisfying kf0 kB∞,∞
6 un , we have

√
p
nτ∞ 1/2
−c2 2J log n
J
b
b
|P{f0 ∈ C(f , Rα )} − (1 − α)| 6 φΠβ (c1 2 log n) + c1
J
+e
.
2J/2

In addition, provided that the right hand side above is smaller than α/2, for sufficiently large n
bα ) is bounded above as
depending only on α and un , the L∞ -diameter of C(fb, R
q
bα )k∞ :=
kC(fb, R
sup
kf − gk∞ 6 c3 2J (log n)/n
bα )
f,g∈C(fb,R

with probability at least 1 − c1 exp(−c2 2J log n).

Remark 3.1 (Rate of convergence). We discuss asymptotic forms of the result using a locally
P
log-Lipschitz prior with locally log-Lipschitz constant L = Ln . Since (l,k)∈I(J) |β0,lk |2 . u2n and
P
τ∞ . l>J 2−ls supk 2l(s+1/2) |β0,lk | . 2−Js un , we have

−s/(2s+1)
log n
n
b
b
|P{f0 ∈ C(f , Rα )} − (1 − α)| . Ln
(5)
u1/2
n + s+1/2 .
log n
un
10

In particular, for the standard Gaussian prior, we have

−s/(2s+1)
n
log n
bα )} − (1 − α)| .
|P{f0 ∈ C(fb, R
u3/2
n + s+1/2 ,
log n
un

since Ln . un from Proposition 2.2.

Remark 3.2 (Coverage errors for the surrogate function). Consider coverage errors for the surrogate function E[fb]. In this case, we can set τ∞ = 0 and thus we have

−s/(2s+1)
n
b
b
b
|P{E[f ] ∈ C(f , Rα )} − (1 − α)| . Ln
u1/2
n .
log n
From this, we see that the Bayesian credible band has the polynomial decay of the coverage error
with respect to E[fb]. Hall [28] showed that the bootstrap confidence band has the polynomial decay
of the coverage error with respect to E[fb]. Our result shows that the Bayesian credible band is
comparable to the bootstrap confidence band.
In [10], Castillo and Nickl also consider multi-scale sets using an admissible sequence w =
(w1 , w2 , . . .):
)
(
|hf − fb∞ , ψl,k i|
6R
for R > 0,
f : sup
wl
(l,k)∈I∞

where I∞ := {(l, k) : J0 6 l < ∞, 0 6 k 6 2l − 1} ∪ {(l, k) : l = J0 − 1, 0 6 k 6 2J0 − 1}
√
R
P
and fb∞ := (l,k)∈I∞ ψl,k ψl,k dY. Here we call a sequence such that wl / l ր ∞ an admissible
sequence. In what follows, we will bound the coverage error and the L∞ -diameter of Bayesian
credible sets of the form
(
)
b∞ , ψl,k i|
|hf
−
f
bα ) := f : sup
bα ,
Cw (fb∞ , R
6R
w
l
(l,k)∈I∞
bα is taken in such a way that Πf {Cw (fb∞ , R
bα ) | Y } = 1 − α.
where the radius R
The following proposition provides the coverage error of multi-scale credible bands using a sieve
√
′
J′
prior on R2 , where J ′ is taken in a way that 2J = (n/ log n)1/(2s+1) . Let u′n := wJ ′ / J ′ ,
√
w := inf l>J ′ wl , and w := maxl<J ′ wl . For simplicity, we assume that maxl<J ′ { l/wl } 6 1.

Proposition 3.2. Under Conditions 2.1 and 2.3 for Πβ that corresponds to Πf , there exist positive constants c1 , c2 , c3 depending only on C1 such that the followings hold: For n > 2 satisfying
s
kf0 kB∞,∞
6 u′n and for any δ > 0, we have
p
p
′
√
bα )) − (1 − α)| 6 φΠ (c1 2J ′ log n) + c1 (e−c2 2J log n + nwδ log n + e−c2 nw2 δ2 ).
|P(f0 ∈ Cw (fb∞ , R
β

Further, provided that the right hand side above is smaller than α/2, for sufficiently large n debα ) is bounded as
pending only on α, L∞ -diameter of Cw (fb∞ , R
q
bα )k∞ 6 c3 2J ′ (log n)/n max{w, u′n }
kCw (fb, R
′

with probability at least 1 − c1 exp(−c2 2J log n) − c1 exp −c2 nw2 δ2 .
11

Remark 3.3 (Choices of δ and w). From proposition 3.2, if nw2 δ2 log n → ∞ and nw2 δ2 log n → 0,
then the coverage error vanishes, which suggests that an admissible sequence should depend on n
when using a sieve prior. Given a divergent sequence {ul : J0 − 1 6 l} in priori, typical choices of
δ and w are
√
 l
for l < J ′ ;
√ s+1/2
δ = 1/( nuJ ′ ) and wl =
√
ul l for l > J ′ .
Using these choices, the same asymptotic result as that of Proposition 3.1 is obtained.

3.1.2. L∞ -credible bands. Focusing on the simple case in which the smoothness s is in (0, 1], L∞ credible bands are constructed using Haar scaling functions: for l ∈ N, let {φl,0 , . . . , φl,2l −1 } be
φl,k (·) := 2l/2 1(k/2l ,(k+1)/2l ] (·).
Let C(fb, R) be given as
(
)
b, φJ,k i|
|hf
−
f
√
C(fb, R) := f : max
6 R , R > 0,
06k62J −1
J
R
P2J −1
where J = (n/ log n)1/(2s+1) un with a divergent sequence un and fb := k=0
φJ,k φJ,k dY . For
bα is chosen in the way that Πf (C(fb, R
bα ) | Y ) = 1 − α. We
a prior Πf of f and for α ∈ (0, 1), R
J
consider a sieve prior Πf on L∞ [0, 1] induced from a prior Πβ on R2 via the same map as in the
previous subsection.
The following theorem provides the coverage error of L∞ -credible bands. In the following,
P2J −1
τ∞ := kf0 − k=0
β0,Jk φJ,k k∞ , where β0 = (β0,J0 , . . . , β0,J(2J −1) ) with β0,Jk := hφJ,k , f0 i for
J
k = 0, . . . , 2 − 1.
Proposition 3.3. Under Conditions 2.1 and 2.3 for Πβ that corresponds to Πf and under the
p
assumption that τ∞ 6 C2′ 2J (log n)/n for some C2′ > 0, there exist positive constants c1 , . . . , c4
s
6 un ,
depending only on C1 and C2′ such that the followings hold: For n > 2 satisfying kf0 kB∞,∞
we have

√
p
nτ∞ 1/2
−c2 2J log n
J
b
b
J
+e
.
|P(f0 ∈ C(f , Rα )) − (1 − α)| 6 φΠβ (c1 2 log n) + c1
2J/2
Further, provided that the right hand side above is smaller than α/2, for sufficiently large n debα ) is bounded as
pending only on α, the L∞ -diameter of C(fb, R
q
q
bα )k∞ 6 c4 2J (log n)/n
c3 2J (log n)/n 6 kC(fb, R
with probability at least 1 − c1 exp(−c2 2J log n).

Remark 3.4 (Comparison to Proposition 3.1). Compared to Proposition 3.1, Proposition 3.3
provides the lower bound of the L∞ -diameter.
3.2. Linear inverse problem. The second application is the frequentist evaluation of the coverage
error of the credible bands based on an indirect observation in Gaussian white noise model:
1
dY (t) = K(f0 )(t)dt + √ dW (t),
n
12

where K is a known linear operator and f0 is included in the s-Hölder–Zygmund space as described in the previous section. To this end, we introduce the wavelet-vaguelette decomposition
(1) (2)
{ψl,k , vl,k , vl,k , κl : (l, k) ∈ I∞ } of K, where recall that I∞ := {(l, k) : J0 6 l < ∞, 0 6 k 6
2l − 1} ∪ {(l, k) : l = J0 − 1, 0 6 k 6 2J0 − 1}: {ψl,k } is the wavelet basis (with the same nota(1)
(2)
tional convention used in the previous subsection), {vl,k } and {vl,k } are near-orthogonal functions,
(2)

and {κlk } is the quisi-singular values such that K(ψl,k ) = κl,k vl,k , (l, k) ∈ I∞ . For details, see
[1, 20, 32, 30] and references therein. Our results cover both the mildly ill-posed and the severely
l
ill-posed cases for {κl,k }: κl,k ∼ 2−rl (mildly ill-posed); κl,k ∼ 2−r2 (severely ill-posed).
We use the Castillo–Nickl credible band for f . Let C(fb, R) be
(
)
b, ψl,k i|
κ
|hf
−
f
l,k
√
C(fb, R) := f : max
6 R , R > 0,
(l,k)∈I(J)
l

P
where fb := (l,k)∈I(J) ψl,k κ−1
l,k

R

(1)

vl,k dY . The choice of J is as follows:

2J = (n/ log n)1/(2s+2r+1) un for an positive sequence un
2J = c log n for some 1/(2r) < c < 1/r

in the mildly-ill posed case;
in the severely-ill posed case.

J
(1)
bα is chosen in the way that Πf (C(fb, R
bα ) |
We use a prior Πf induced from Πβ on R2 via {vl,k }, and R
Y ) = 1 − α.
The following theorem provides the coverage error of Castillo–Nickl credible band in linear inverse
P
′ := kK(f ) − K(
problems. In the following, τ∞
0
(l,k)∈I β0,lk ψl,k )k∞ , where β0,lk := hf, ψl,k i.

Proposition 3.4. Under Conditions 2.1 and 2.3 for Πβ that corresponds to Πf and under the
p
′ 6 C′
′
J
assumption that τ∞
2 2 (log n)/n for some C2 > 0, there exist positive constants c1 , c2 , c3
depending only on C1 appearing in Condition 2.1 and C2′ such that the followings hold: For n > 2
s
satisfying kf0 kB∞,∞
6 un , we have

√ ′
n
o
 p

nτ∞ 1/2
−c2 2J log n
J
b
b
P f0 ∈ C(f , Rα ) − (1 − α) 6 φΠβ c1 2 log n + c1
J
+e
.
2J/2
Further, provided that the right hand side above is smaller than α/2, for sufficiently large n debα ) is bounded as
pending only on α, L∞ -diameter of C(fb, R
q
bα )k∞ 6 c3 κ−1 J −1
2J (log n)/n
kC(fb, R
J−1,2
−1
with probability at least 1 − c1 exp(−c2 2J log n).

Remark 3.5 (Rate of convergence). The asymptotic form is demonstrated using a locally logLipschitz prior with locally log-Lipschitz constant L = Ln . We have
n
o
bα ) − (1 − α)
P f0 ∈ C(fb, R
 
−s/(2s+2r+1)
1/2

log n
Ln n
, in the mildly ill-posed case;
un + s+r+1/2
log n
un
.
−s/(2s+2r+1)


Ln n
+ (log n)−s ,
in the severely ill-posed case,
log n
13

and
bα )k∞ .
kC(fb, R




n
log n

−s/(2s+2r+1)

r+1/2

un

, in the mildly ill-posed case;

(log n)−s ,

in the severely ill-posed case,

with probability at least 1 − c1 exp(−c2 2J log n).
3.3. Nonparametric regression model. The third application of the main theorem is the frequentist evaluation of coverage errors of credible bands in nonparametric regression models:
Yi = f0 (Ti ) + εi , i = 1, . . . , n,
where ε = (ε1 , . . . , εn ) is a vector of i.i.d. error terms with mean zero and variance σ02 and {Ti : i =
1, . . . , n} are i.i.d. samples on [0, 1]. For simplicity, ε and {Ti : i = 1, . . . , n} are independent and
σ0 does not depend on n.
Using p basis functions {ψjp (·) : 1 6 j 6 p}, we consider the credible bands for f of the form
)
(
b(·)
f
(·)
−
f
bα ,
bα ) = f :
6R
C(fb, R
kψ p (·)k
Pp

∞

Pn

P
bα is taken in a way
where fb(·) := j=1 ψjp (·)βbj with βb := argminβ i=1 |Yi − pj=1 ψjp (Ti )βj |2 , R
bα ) | Y } = 1 − α, and ψ p (·) := (ψ p (·), . . . , ψpp (·))⊤ . We consider a prior Πf of f
that Πf {C(fb, R
1
induced from a sieve prior Πβ on Rp by the map
(β1 , . . . , βp ) 7→

p
X

βj ψjp (·).

j=1

Due to the randomness of {Ti }, it is necessary to develop bounds of the coverage error and
L∞ -diameter taking the randomness of {Ti } into consideration. To this end, we modify Conditions
2.1 and 2.3 and add conditions on the basis function. Let ψep (·) := ψ p (·)/kψ p (·)k, and ξp :=
supt∈[0,1] kψ p (t)k. Let β0 := argminβ E|f0 (T1 ) − ψ p (T1 )⊤ β|2 . For R > 0, let
e
B(R)
:= {β : kβ − β0 k 6 n−1/2 R}, and φeΠβ (R) := 1 −

inf

e B(R)
e
β,β∈

e
{π(β)/π(β)}.

Condition 3.1. There exists a positive constant C1 such that π(β0 ) > e−C1 p log n .
√
Condition 3.2. The inequality φeΠβ (1/ n) 6 1/2 holds.

Condition 3.3. There exist strictly positive constants b and b such that the eigenvalues of the p × p
2
matrix (Eψip (T1 )ψjp (T1 )) are included in [b2 , b ].
Condition 3.4. There exist positive constants C4 and C5 such that the inequalities
log ξp 6 C4 log p and log

sup {kψep (t) − ψep (t′ )k/|t − t′ |} 6 C5 log p

t6=t′ ∈[0,1]

hold.

14

Remark 3.6 (Comments on the assumptions). Conditions 3.1 and 3.2 are the versions of Conditions 2.1 and 2.3 in the case that {Ti }s are random. Condition 3.3 is a standard assumption.
Condition 3.4 is substantially weak; for example, the assumption holds for Fourier series, Spline
series, CDV wavelets, and local polynomial partition series; see [5].
bα ).
The following proposition provides both the coverage error and the L∞ -diameter of C(fb, R
p
p
⊤
p
⊤
2
Let τ2 := E|f0 (T1 ) − ψ (T1 ) β0 | and τ∞ := kf0 (·) − ψ (·) β0 k∞ . Further, let
τ :=

f0 (·) − ψ p (·)⊤ β0
kψ p (·)k

.
∞

Proposition 3.5. Under Conditions 3.1-3.4 and 2.2, there exists positive constants c1 , c2 , c3 depending only on C1 , . . . , C5 , b, b, and q appearing in Conditions3.1-3.4 and 2.2 and Assumption 2.2
such that the followings hold: For n > 2 and any sufficiently small δ > 0, we have
bα )) − (1 − α)| 6 φeΠ (c1
|P(f0 ∈ C(fb, R
β

where

γn :=
and

p

p log n) + δ2 + δ3 + c1 (n−2δ + δ1 p log n + ωn + γn ),

(6)

n
1/2 o
√ p
n τ22
τ∞ nδ log p + nτ log p
+ max 1, pξp2 /n
log n p


o
n
nδ (log n)1/2 max ξ 2 /n1/2 n1/q log n, ξ 2 /n1/6 (log n)2/3
p
p
ωn :=
nδ (log n)7/6 ξ 2 /n1/6
p

under Assumption 2.2 (a)
under Assumption 2.2 (b).

Further, provided that the right hand side in (6) is smaller than α/2, for sufficiently large p de√ √
pending only on α, with probability at least 1 − δ3 − c1 { nτ log p + exp(−c2 p log n)}, we have
q
sup
kf − gk∞ 6 c3 ξp2 (log p)/n.
bα )
f,g∈C(fb,R

Remark 3.7 (Choices of ξp , τ2 , τ∞ , and τ ). For typical basis functions including Fourier series,
√
spline series, and CDV wavelets, ξp . p; see Section 3 in [5]. For S(> s)-regular CDV wavelets,
in the case that f0 is in the Hölder–Zygmund space with smoothness level s, τ2 ∼ τ∞ ∼ p−s .
For the other series and the other function classes, bounds on τ2 and τ∞ are available from the
√
approximation theorem; see [19] and Section 3 in [5]. Typical choice of τ is τ∞ / p; For the Haar
√
wavelet, we have τ ∼ τ∞ / p, since τ 6 τ∞ / inf t∈[0,1] kψ p (t)k. For periodic S-regular wavelets, we
√
also have τ ∼ τ∞ / p as shown in Appendix E.3.
Remark 3.8 (Rate of convergence). Consider the case with an unknown variance. Assume that
√
there exists a constant s > 1/2 such that τ2 ∼ τ∞ ∼ p−s , τ ∼ p−s−1/2 , and ξp . p. Assume
also that the error distribution is sub-Gaussian. Note that the assumption that s > 1/2 is usual
in nonparametric regression with an unknown variance; see Assumption A.1 in [49]. Consider that
we put a locally log-Lipschitz prior with locally log-Lipschitz constant L = Ln on β and use an
15

estimate σ
b2 = σ
bu2 . Then, taking p ∼ (n/ log n)1/(2s+1) un with a divergent sequence un , we have
bα )) − (1 − α)|
|P (f0 ∈ C(fb, R


−s/(2s+1)
−(s−1/2)/(2s+1)
n
n
log n
1/2
. Ln
un +
un log n + s+1/2
log n
log n
un

(7)
(8)

and
sup
bα )
f,g∈C(fb,R

kf − gk∞ .



n
log n

−s/(2s+1)

u1/2
n

with probability 1 − c1 /n.
One of the important aspects of this result is that it admits a general sieve prior for β. From (8),
the diminishing rate of the coverage error with respect to the prior distribution, that is, the first
√
and second terms on the right hand side in (8) is unchanged whenever Ln . p up to a logarithmic
factor and un .
4. Proof of Theorem 2.1
In this section, we provide the proof of the main theorem.
4.1. Technical lemmas. Before the proof, we state pivotal ingredients of the proof except the
Berry–Esseen type bound on posterior distributions: the high-dimensional CLT on hyper-rectangles,
the anti-concentration inequality on hyper-rectangles, Anderson’s lemma, and concentration inequality for Gaussian maxima.
The high-dimensional CLT on hyper-rectangles is stated as follows: let Z1 , . . . , Zn be independent
p-dimensional random vectors with mean zero. We denote the j-th coordinate of Zi by Zij . Let
f1 , . . . , Z
fn be independent centered p-dimensional Gaussian vectors such that each Zei has the same
Z
covariance matrix as Zi . Let Are be the class of all hyper-rectangles in Rp : for any A ∈ Are, A is of
the form A = {β ∈ Rp : ai 6 βi 6 ai , 1 6 ∀i 6 p} for (a1 , . . . , ap )⊤ ∈ Rp and for (a1 , . . . , ap )⊤ ∈ Rp .
Assume that the following three conditions hold:
P
H1. There exists b > 0 such that n−1 ni=1 E|Zij |2 > b for 1 6 ∀j 6 p;
P
H2. There exists a sequence Bn > 1 such that n−1 ni=1 E|Zij |2+k 6 Bn4 for 1 6 ∀j 6 p and for
k = 1, 2;
H3. We assume either one of the following two conditions:
(a) There exists q > 0 such that E[{maxj=1,...,p |Zij |/Bn }q ] 6 2 for 1 6 ∀i 6 n;
(b) E[exp{|Zij |/Bn }] 6 2 for 1 6 ∀i 6 n and for 1 6 ∀j 6 p.
Lemma 4.1 (High Dimensional CLT for Hyperrectangles; Proposition 2.1 in [16]). Assume that
Conditions H1 and H2 hold. Let
!
!
n
n
X
X
√
√
Zei / n ∈ A .
Zi / n ∈ A − P
ρ = ρn := sup P
A∈Are

i=1

i=1

16

Then, there exists a positive constant e
c1 for which we have
 
1/6
 2 3

7
2

(pn) 1/3
e
c1 Bn logn (pn)
+e
c1 Bnnlog
under Condition H3 (a);
1−2/q
ρ6
1/6
 2 7

e
under Condition H3 (b).
c1 Bn logn (pn)

Here, e
c1 depends only on b appearing in Condition H1 and q appearing in Condition H3.

We use the anti-concentration inequality on hyper-rectangles. Let σj2 := EZj2 > 0 for 1 6 ∀j 6 p
and let σ := min{σj }, σ := max{σj }.

Lemma 4.2 (Anti-concentration inequality on hyper-rectangles; Theorem in [35]). There exists a
universal positive constant e
c2 for which we have, for every z = (z1 , . . . , zp )⊤ ∈ Rp and R > 0,
R
ej 6 zj for 1 6 ∀j 6 p) 6 e
γ := γ(R) := P(Zej 6 zj + R for 1 6 ∀j 6 p) − P(Z
c2
σ

p

log p.

Remark 4.1 (Comment on the anti-concentration inequality). The point here is that the above
inequality allows for general hyper-rectangles. Remark that when we focus on max-rectangles, the
anti-concentration inequality above is less sharp than the anti-concentration inequality on max√
rectangles obtained by [15]; see also [13] in the sense that the former uses log p while the latter
uses E[maxi=1,...,p Zej ].
The followings are Anderson’s lemma and the concentration inequality for Gaussian maxima.

e be two nonnegative definite
Lemma 4.3 (Anderson’s lemma; Collorary 3 in [2]). Let Σ and Σ
e
and symmetric p × p matrices. Let Y and Ye be random vectors from N (0, Σ) and from N (0, Σ),
p
e
respectively. Let C be a convex symmetric set in R . If Σ − Σ is nonnegative definite, then P(Y ∈
C) 6 P(Ye ∈ C).
Lemma 4.4 (Concentration inequality for Gaussian maxima; Theorem 2.5.8. in [27]). Let {Ni }pi=1
be i.i.d. random variables from the standard Gaussian distribution. Then, we have
2 /2

P(| max |Ni | − E max |Ni || > R) 6 e−R
i=1,...,p

i=1,...,p

, R > 0.

4.2. Proof of Theorem 2.1. We only prove the theorem under Assumption 2.2 (a). The proof
under Assumption 2.2 (b) is completed replacing Lemma 4.1 (a) by Lemma 4.1 (b).
The proof is divided into two parts: The former part is to present an upper bound of the
b ), R
bα )) − (1 − α)|. The latter part is to evaluate the max-diameter of
coverage error |P(β0 ∈ I(β(Y
b
b
I(β(Y ), Rα ).
bα concentrates on the (1 − α)Upper bound for the coverage error. At the first step, we show that R
quantile of some distribution with a high probability. From Proposition 2.5, we have
b ), R
b ), R
b ), σ 2 (X ⊤ X)−1 )| 6 ω for Y ∈ H,
bα ) | Y ) − N (I(β(Y
bα ) | β(Y
|Πβ (I(β(Y
0

where ω is the upper bound in Proposition 2.5 and recall that
p
b ) − β0 )k 6 c1 p log nσ0 /4} ∩ {Y : Πσ2 (|σ 2 /σ 2 − 1| > δ1 | Y ) 6 δ2 }.
H = {Y : kX(β(Y
0
17

⊤
−1 ⊤
Let G be the cumulative distribution function of σ0 max{|e⊤
(p),i (X X) X N |/wi }, where e(d),i is
the d-dimensional unit vector whose i-th component is 1, and N is the random vector from the
standard p-dimensional Gaussian distribution. Since

b ), R
b ), σ 2 (X ⊤ X)−1 ) = P(σ0 max {|e⊤ (X ⊤ X)−1 X ⊤ N |/wi } 6 R
bα ) | β(Y
bα | Y ),
N (I(β(Y
0
(p),i
i=1,...,p

bα )| 6 ω for Y ∈ H. Letting G−1 be the quantile function of G yields
we have |(1 − α) − G(R
bα 6 G−1 (1 − α + ω) for Y ∈ H,
G−1 (1 − α − ω) 6 R

(9)

which completes the first step.
b ), R
bα )) − (1 − α). The lower
At the second step, we derive an upper bound of P(β0 ∈ I(β(Y
bα 6 G−1 (1 − α + ω) holds for Y ∈ H,
bound is obtained in the same way. Because the inequality R
we have
b ), R
bα )) − (1 − α)
P(β0 ∈ I(β(Y

⊤
−1 ⊤
b
6 P(Y ∈ {Y : max {|e⊤
(p),i (X X) X Y |/wi } 6 Rα } ∩ H) − (1 − α) + P(Y 6∈ H)
i=1,...,p


⊤
⊤
−1 ⊤
−1
6 P max {|e(p),i (X X) X (ε + r)|/wi } 6 G (1 − α + ω) − (1 − α) + ρ + P(Y 6∈ H)
i=1,...,p

6 γ(k(X ⊤ X)−1 X ⊤ rk∞ ) + ρ + P(Y 6∈ H),

where both ρ and γ = γ(k(X ⊤ X)−1 X ⊤ rk∞ ) are constants appearing in Lemmas 4.1 and 4.2 in
the case that Z := (X ⊤ X)−1 X ⊤ ε. From Proposition 2.6, we have the upper bound of P(Y 6∈ H).
Noting that ρ is independent of rescaling of Z and replacing Z by Z/σ0 λ1/2 , we can take b = 1 and
√
Bn = p(E|ε1 /σ0 |q )1/q (λ/λ)1/2 , since we have
n−1

n
X
i=1

E|Zij |2 := n−1

and since it follows from

n
X
i=1

⊤
−1 ⊤
2
2
E|e⊤
(p),j (X X) X e(n),i εi | > σ0 λ

p
√
⊤
−1
1/2
|Zij /(εi /σ0 )| 6 ke⊤
)kX ⊤ e(n),i k 6 (λ/λ) p
(n),i X(X X) k/ λ 6 (λ/λ

that for 1 6 ∀j 6 p,
n−1

n
X
i=1

n

X
√
√
E|Zij |4 6 ( pλ/λ)4 E(ε1 /σ0 )4 .
E|Zij |3 6 ( pλ/λ)3 E(ε1 /σ0 )3 and n−1
i=1

Thus, from Lemmas 4.1 and 4.2, we obtain the following bounds on ρ and γ: For some e
c1 > 0
depending only on q,
(
1/6 
1/3 )
p log3 (pn) λ
p log7 (pn) λ
+
,
ρ6e
c1
n
λ
n1−2/q λ
γ6e
c1

k(X ⊤ X)−1 X ⊤ rk∞ p
log p,
σ0 λ

which completes the second step and thus completes the evaluation of the coverage error.
18

Estimate of the max-diameter. At the first step, we bound the max diameter using the quantile
function F −1 of maxi=1,...,p |Ni |. From the triangle inequality, we have, for Y ∈ H,
G−1 (1 − α − ω) 6

max |β1,i − β2,i | 6 2G−1 (1 − α + ω).

sup

bR
bα ) i=1,...,p
β1 ,β2 ∈I(β,

Lemma 4.3 yields, for R > 0,




wR
ei | 6 R 6 P
6
P
max
|
N
P max |Ni | 6
i=1,...,p
i=1,...,p
σ0 λ1/2

max |Ni | 6

i=1,...,p

wR
σ0 λ

1/2

!

,

e := σ0 W (X ⊤ X)−1 X ⊤ N and W = diag(w1 , . . . , wp ). Therefore, for any β ∈ (0, 1) we have
where N
!


w
w
−1
−1
G−1 (1 − β),
(10)
G (1 − β) 6 F (1 − β) 6
1/2
1/2
σ
λ
0
σ0 λ
which completes the first step.
At the second step, we will show that for sufficiently large p depending only on α, F −1 (1−α±ω) ∼
√
√
log p with probability at least P(Y ∈ H). First, we will show that F −1 (1 − α + ω) . log p. From
Lemma 4.4, taking sufficiently large p depending only on α yields
p
c22 log p/2) < α − α/2 < α − ω,
P( max |Ni | − E max |Ni | > e
c2 log p) 6 exp(−e
i=1,...,p

i=1,...,p

for some positive constant e
c2 . Therefore, noting that

F −1 (1 − α + ω) := inf{R : P( max |Ni | > R) 6 α − ω}
i=1,...,p

= inf{R : P( max |Ni | − E max |Ni | > R − E max |Ni |) 6 α − ω},
i=1,...,p

i=1,...,p

i=1,...,p

√
we have F −1 (1 − α + ω) . log p.
√
Second, we show that F −1 (1 − α − ω) & log p. From the Paley–Zygmund inequality, we have,
for θ ∈ [0, 1],
P( max |Ni | > θE max |Ni |) > (1 − θ)2 (E max |Ni |)2 /E( max |Ni |)2 .
i=1,...,p

i=1,...,p

i=1,...,p

i=1,...,p

Here, it follows that
E{ max |Ni |}2 6 {E max |Ni |}2 +
i=1,...,p

i=1,...,p

√
2πE max |Ni | + 2
i=1,...,p

because we have, for any δ in (0, 1),
E[{ max |Ni |}2 ]
i=1,...,p
Z
=

[0,E maxi=1,...,p |Ni |+δ]

P( max |Ni | > t)dt +

6 (E max |Ni | + δ)2 +
i=1,...,p

6 (E max |Ni | + δ)2 +
i=1,...,p

Z

2

i=1,...,p

Z

[E maxi=1,...,p |Ni |+δ,∞)

2te−(t−E maxi=1,...,p |Ni |

[E maxi=1,...,p |Ni |+δ,∞)

√
2πE max |Ni | + 2,
i=1,...,p

19

P( max |Ni |2 > t)dt

2 /2

i=1,...,p

dt

(11)

where the second inequality follows from the concentration inequality of the maxima of a Gaussian
process. Since it follows that (E maxi=1,...,p |Ni |)2 /E(maxi=1,...,p |Ni |)2 → 1 as p → ∞ from (11), we
have, for sufficiently large p and for some θ depending only on α,
P( max |Ni | > θE max |Ni |) > α + ω
i=1,...,p

and thus we obtain F −1 (1 − α − ω) &
the proof.

√

i=1,...,p

log p, which completes the second step and hence completes


Appendix A. Proof of Proposition 2.5
A.1. Technical Lemmas. We present here some technical lemmas that will be used to prove
Proposition 2.5.
Lemma A.1 (Scheffé’s lemma). Let Q1 and Q2 be probability measures on a measurable space with
a common dominating measure µ. Let q1 = dQ1 /dµ and q2 = dQ2 /dµ. Then
Z
Z
1
kQ1 − Q2 kTV =
|q1 (x) − q2 (x)|dµ(x) = (q1 (x) − q2 (x))+ dµ(x),
2
Proof. See, e.g., p.84 in [45].



Lemma A.2 (Posterior contraction of a marginal prior distribution). Recall that B(R) = {β ∈
Rp : kX(β −β0 )k 6 σ0 R} for R > 0. Under Conditions 2.1 and 2.3, there exist positive constants e
c1
and e
c2 depending only on C1 in Condition 2.1 such that for a sufficiently large R > 0, the inequality
Πβ (β 6∈ B(R) | Y, σ 2 ) 6 4 exp{e
c1 p log n − e
c2 (σ02 /σ 2 )R2 }

(12)

holds for Y ∈ H, where recall that

b ) − β0 )k 6 Rσ0 /4} ∩ {Y : Πσ2 (|σ 2 /σ 2 − 1| > δ1 | Y ) 6 δ2 }.
H := {Y : kX(β(Y
0

Proof. We use the following lower bounds on the small ball probability of a prior distribution:
Lemma A.3 (Lower bounds on the small ball probability of a prior distribution). Let Πβ be a
probability measure with a density π with respect to the p-dimensional Lebesgue measure. Recall
e for R > 0. Then, we have, for every R > 0,
that φΠβ (R) = 1 − inf β,β∈B(R)
{π(β)/π(β)}
e
Πβ (β ∈ B(R)) >

Proof of Lemma A.3. Observe that
Z
Πβ (β ∈ B(R)) =

{1 − φΠβ (R)}(πeR)p/2
2(p/2 + 1)p/2+1/2

π(β)dβ >

B(R)

inf

β∈B(R)



p

π(β)
π(β0 )

π(β0 )σ0p

det(X ⊤ X)



π(β0 )

Z

.

dβ.
B(R)

Changing variables, we have that
Z
Z
(σ02 R2 )p/2 π p/2
(σ 2 R2 )p/2
dβ = p
,
dβ = p 0
det(X ⊤ X) kβk61
det(X ⊤ X)Γ(p/2 + 1)
B(R)
20

where Γ(·) is the Gamma function. Using the bound
√
2π
Γ(p/2 + 1) 6
(p/2 + 1)p/2+1/2 e−p/2 e1/18
e
(see section 5.6.1. in [36]), we have that
Z
(σ 2 πeR2 )p/2 e17/18
dβ > √ p 0
.
2π det(X ⊤ X)(p/2 + 1)p/2+1/2
B(R)
√
Since e17/18 / 2π > 1/2, we obtain the desired inequality.



Return to the proof of Lemma A.2. Letting P := X(X ⊤ X)−1 X ⊤ , we have
R
−hP (ε+r),X(β−β0 )i/σ2 −kX(β−β0 )k2 /(2σ2 ) π(β)dβ
c e
2
Πβ (β ∈ B | Y, σ ) = RB −hP (ε+r),X(β−β )i/σ2 −kX(β−β )k2 /(2σ2 )
.
0
0
e
π(β)dβ

Since cx2 + c−1 y 2 > 2xy for x, y, c > 0, we have, for any c > 1,
Z
2
2
2
e−hP (ε+r),X(β−β0 )i/σ −kX(β−β0 )k /(2σ ) π(β)dβ
Bc
Z
2
2
2
ekP (ε+r)kkX(β−β0 )k/σ −kX(β−β0 )k /(2σ ) π(β)dβ
6
c
ZB
2
−1
2
2
2
2
e{ckP (ε+r)k +c kX(β−β0 )k }/(2σ )−kX(β−β0 )k /(2σ ) π(β)dβ
6

(13)

Bc

6 exp{ckP (ε + r)k2 /(2σ 2 ) − (1 − c−1 )(σ02 /σ 2 )R2 /2}.

(14)

e = 1/√πen, we have
Letting R
Z
2
2
2
e−hP (ε+r),X(β−β0 )i/σ −kX(β−β0 )k /(2σ ) π(β)dβ
Z
2
2
2
e−hP (ε+r),X(β−β0 )i/σ −kX(β−β0 )k /(2σ ) π(β)dβ
>
>

Z

e
B(R)

e
B(R)

2 +c−1 kX(β−β

e−{ckP (ε+r)k

2
2
2
2
0 )k }/(2σ )−kX(β−β0 )k /(2σ )

π(β)dβ

e2 /2}Πβ (B(R)).
e
> exp{−ckP (ε + r)k2 /(2σ 2 ) − (1 + c−1 )(σ02 /σ 2 )R

(15)

We have
Z
2
2
2
e−hP (ε+r),X(β−β0 )i/σ −kX(β−β0 )k /(2σ ) π(β)dβ

e
1 − φΠβ (R)
2
2
−1 e2
2
2
ep log n/2−p log p−C1 p log n e−ckP (ε+r)k /(2σ )−(1+c )(R /2)(σ0 /σ )
2
e2 /2}, (16)
> 4−1 exp{p log n/2 − p log p − C1 p log n − ckP (ε + r)k2 /(2σ 2 ) − (1 + c−1 )(σ02 /σ 2 )R
>

where the first inequality follows from (15) and from Lemma A.3 and the second inequality follows
from Condition 2.3.
Combining (14) and (16) with (13), we have, for Y ∈ H,
Z
Z

2
2
−kY −Xβk2 /(2σ2 )
e
π(β)dβ
e−kY −Xβk /(2σ ) π(β)dβ
Bc

6 4 exp[(C1 + 1/2)p log n + {(1 + c−1 )/(2n)}(σ02 /σ 2 ) − {(1 − c−1 )/2 − c/16}(σ02 /σ 2 )R2 ].
21

(17)

Taking c = 3 completes the proof.



Lemma A.4. Let A be an n × n symmetric positive semidefinite matrix such that kAkop 6 1 and
rank(A) < n. Let ε = (ε1 , . . . , εn )⊤ be a vector of i.i.d. random variables with mean zero and unit
variance.
(a) If in addition Assumption 2.2 (a) holds for an integer q > 2 and C3 > 0, then there exists
p
a positive constant e
c1 depending only on q and C3 such that, for every R > rank(A),


p
c1 rank(A)/(R − rank(A))q .
P ε⊤ Aε > R2 6 e

(b) If instead Assumption 2.2 (b) holds for C3 > 0, then there exists a positive constant e
c1
depending only on C3 such that, for every R > 0,



P |ε⊤ Aε − E[ε⊤ Aε]| > R2 6 2 exp{−e
c1 min R4 /kAk2HS , R2 },
where k · kHS denotes Hilbert–Schmidt norm.

Proof. For Case (a), see Corollary 5.1 in [4]. The inequality in Case (b) is called the Hanson-Wright
inequality; for a proof, we refer to [29] and [41].

A.2. Proof of Proposition 2.5. Before the proof, we prepare additional notations for the sake
b ), σ 2 (X ⊤ X)−1 ). Let B := B(c1 √p log n) and H := H(c1 )
e := N (β(Y
of notational simplicity. Let N
0
for a sufficiently large c1 > 0 depending on C1 and C2 . Let ΠB
β (dβ | Y ) be the probability measure
defined by
Z
B
Πβ (dβe | Y )
Πβ (dβ | Y ) := 1β∈B Πβ (dβ | Y )
B

e B be the probability measure defined by
and let N
Let Πβ (· |

Y, σ 2 )

Z
B
e (dβ).
e
e
N
N (dβ) := 1β∈B N (dβ)

be the distribution defined by
2

−kY −Xβk2 /(2σ2 )

Πβ (dβ | Y, σ ) := e

π(β)dβ

2
and let ΠB
β (· | Y, σ ) be the distribution defined by

ΠB
β (dβ

2

−kY −Xβk2 /(2σ2 )

| Y, σ ) := 1β∈B e

B

Z

π(β)dβ

e

2 /(2σ 2 )

e−kY −X βk

Z

e βe
π(β)d

e 2 /(2σ2 )

e−kY −X βk
B

e β.
e
π(β)d

In the proof, e
c1 , e
c2 , . . . are positive constants depending only on C1 , C2 , and c1 .

Proof outline :
First of all, we present a brief outline of the proof. From the triangle inequality, we have

e kTV 6 kΠβ (dβ | Y ) − Πβ (dβ | Y, σ 2 )kTV + kΠβ (dβ | Y, σ 2 ) − N
e kTV .
kΠβ (dβ | Y ) − N
(18)
0
0
 2
Consider the first term on the right hand side of (18). Letting S = S(δ1 ) := σ : |σ 2 /σ02 − 1 6 δ1 },
it follows that
Z
2
kΠβ (dβ | Y, σ 2 ) − Πβ (dβ | Y, σ02 )kTV Πσ2 (dσ 2 | Y ) + δ1
kΠβ (dβ | Y ) − Πβ (dβ | Y, σ0 )kTV 6
S

22

with probability at least 1 − δ2 , from the application of Jensen’s inequality to the function x → |x|
and from Condition 2.2. For the bound of the first term on the rightmost hand in the above
inequality, the triangle inequality yields
Z
(19)
kΠβ (dβ | Y, σ 2 ) − Πβ (dβ | Y, σ02 )kTV Πσ2 (dσ 2 | Y ) 6 A1 + A2 + A3 ,
S

where
A1 :=
A2 :=
A3 :=

Z

Z

S

ZS
S

2
2
kΠβ (dβ | Y, σ 2 ) − ΠB
β (dβ | Y, σ )kTV Πσ2 (dσ | Y ),
2
2
B
2
kΠB
β (dβ | Y, σ ) − Πβ (dβ | Y, σ0 )kTV Πσ2 (dσ | Y ),
2
2
2
kΠB
β (dβ | Y, σ0 ) − Πβ (dβ | Y, σ0 )kTV Πσ2 (dσ | Y ).

Upper bounds of A1 , A2 , A3 will be presented in (21),(22), and (23). Consider the second term on
the right hand side of (18). From the triangle inequality, we have
e kTV 6 A4 + A5 + A6 ,
kΠβ (dβ | Y, σ02 ) − N

(20)

e −N
e B kTV , A5 := kN
e B − ΠB (dβ | Y, σ 2 )kTV , and A6 := kΠB (dβ | Y, σ 2 ) − Πβ (dβ |
where A4 := kN
0
0
β
β
Y )kTV . Upper bounds of A4 , A5 , A6 will be presented in (25), (26), and (27).
Upper bound of (19):
Consider A1 in (19). From Lemmas A.1 and A.2, taking a sufficiently large c1 depending only
on C1 yields
Z
Πβ (β 6∈ B | Y, σ 2 )Πσ2 (dσ 2 | Y ) 6 4e−ec1 p log n .
(21)
A1 =
S

Consider A2 in (19). From Lemma A.1, we have
Z Z

2
2
1 − φΠβ ,2 (β, σ 2 ) + ΠB
A2 =
β (dβ | Y, σ0 )Πσ2 (dσ | Y ),
S

where

2

2 /(2σ 2 )

φΠβ ,2 (β, σ ) := R

π(β)e−kY −Xβk
e
−kY −X βk
Be

2 /(2σ 2 )

e βe
π(β)d

R

B

e 2 /(2σ2 )
e βe
0 π(β)d

e−kY −X βk

2 /(2σ 2 )
0

π(β)e−kY −Xβk

.

From Cauchy–Schwarz’s inequality and Assumption 2.1, we have
e−hP (ε+r),Xβ0 −Xβi/σ

2 −kXβ
2

2
2
0 −Xβk /(2σ )
2 /(2σ 2 )
0

> e−hP (ε+r),Xβ0 −Xβi/σ0 −kXβ0 −Xβk

2

e−C2 c1 δ1 p log n/{4(1−δ1 )}−c1 δ1 p log n/(1−δ1 ) .

Likewise, we have
Z
2
2
e
e 2
e βe
e−hP (ε+r),Xβ0 −X βi/σ −kXβ0 −X βk /(2σ ) π(β)d
B
Z
2
2
2
C2 c1 δ1 p log n/{4(1−δ1 )}+c21 δ1 p log n/(1−δ1 )
e β.
e
6e
e−hP (ε+r),Xβ0 −Xβi/σ0 −kXβ0 −Xβk /(2σ0 ) π(β)d
B

23

Therefore, we have, for β ∈ B, Y ∈ H, and σ 2 ∈ S,
φΠβ ,2 (β, σ 2 ) > exp(−e
c2 δ1 p log n)
and thus it follows that
A2 6 e
c2 δ1 p log n

(22)

A3 6 Πβ (β 6∈ B | Y, σ02 ) 6 4 exp(−e
c3 p log n).

(23)

since (1 − e−x )+ 6 x for x > 0.
Consider A3 in (19). From Lemmas A.1 and A.2, taking a sufficiently large c1 depending only
on C1 yields, for Y ∈ H,
Therefore, inequalities (21), (22), and (23) yield
Z
c4 e−ec5 p log n + e
c4 δ1 p log n.
kΠβ (dβ | Y, σ 2 ) − Πβ (dβ | Y, σ02 )kTV Πσ2 (dσ 2 | Y ) 6 e

(24)

S

Upper bound of (20):
Consider A4 in (20). From Lemmas A.1 and A.4, we have
p
e (B c ) 6 exp{−(3c1 p log n/4 − √p)2 /2}.
A4 = N

(25)

Consider A5 in (20). From Lemma A.1, we have
!
Z
eB
dN
2
1−
A5 =
(β)
ΠB
β (dβ | Y, σ0 ).
2)
dΠB
(·
|
Y,
σ
0
β
+

e Since we have, for β ∈ B,
e with respect to the Lebesgue measure by φ.
We denote the density of N
2
e
e
eB
dΠB
dN
φ(β)
π(β)φ(β)
β (· | Y, σ0 )
(β) = R
(β) = R
and
,
e e e
e e e e
dβ
dβ
B φ(β)dβ
B π(β)φ(β)dβ

applying Jensen’s inequality to x → (1 − x)+ yields
!
!
Z
Z
Z
e
e β)
e
eB
π(β)
φ(
dN
2
B
1−
dβe
ΠB
(β | Y, σ0 )
Πβ (dβ | Y ) =
1−
R
β (dβ | Y )
′
′
e
π(β)
dΠB
φ(β
)dβ
B
β
B
+
+
!
Z Z
e
e β)
e
π(β)
φ(
e B (dβ | Y ).
1−
6
dβΠ
R
β
′
′
e
π(β)
φ(β )dβ
B
+

B

Therefore, we obtain

A5 6 φΠβ (c1

p

p log n).

(26)

Consider A6 in (20). From Lemmas A.1 and A.2, taking a sufficiently large c1 > 0 yields
A6 = Πβ (β 6∈ B | Y, σ02 ) 6 4 exp(−e
c6 p log n).

(27)

Therefore, inequalities (25), (26), and (27) yield
e kTV 6 φΠ (c1
kΠβ (dβ | Y, σ02 ) − N
β
24

p

p log n) + e
c7 exp(−e
c8 p log n).

(28)

Combining (24) and (28) with (19) provides the upper bound of the target total variation and
thus completes the proof.

A.3. Proof of Proposition 2.6. Let c be any positive number. Under Assumption 2.2 (a), it
√
follows from Lemma A.4 (a) with R = c p log n that the inequality
P(Y 6∈ H(c)) 6 e
c1 p1−q/2 (log n)−q/2 + δ3

holds for some e
c1 > 0 depending only on c, C3 , and q. Under Assumption 2.2 (b), it follows from
Lemma A.4 (b) with R = (c2 + 1)p log n, that the inequality
2 ,p log n}

P(Y 6∈ H(c)) 6 2e−ec2 min{p(log n)

+ δ3

holds for some e
c2 > 0 depending only on c, C3 , and q. Thus, we completes the proof.



Appendix B. Proofs of Propositions 2.1–2.4

1/2
e
Proof of Proposition 2.1. Let B(R)
:= {β : kβ − β0 k 6 σ0 λ R} for R > 0. The inequality
p
1/2 p
φΠβ (c p log n) < cLσ0 λ
p log n,

(29)

holds for any c > 0, since
p
1/2 p
e
6 cσ0 Lλ
p log n,
φΠβ (c p log n) 6 sup [1 − exp{− log{π(β)/π(β)}}]
e B
e
β,β∈

−1/2

kβ − β0 k and the second inequality
where the first inequality follows because kX(β − β0 )k > λ
√
√
−x
follows because 1 − e
6 x. Substituting c = 1/( pn log n) into (29), we have φΠβ (1/ n) 6
√
1/2

Lλ σ0 / n. Therefore, we complete the proof.
Proof of Proposition 2.2. First, consider an isotropic prior. We have
Z
log π(β0 ) = log ρ(kβ0 k) − log ρ(kβk)dβ


Z ∞
p−1
p/2
x ρ(x)dx
= log ρ(kβ0 k) − log {pπ /Γ(p/2 + 1)}
0

> log inf ρ(x) − e
c1 p log p
x∈[0,B]

q
> log inf ρ(x) − e
c1 p log p − e
c1 log n + log{ det(X ⊤ X)/σ0p }
x∈[0,B]

for some positive constant e
c1 depending only on m and c appealing in the definition of an isotropic
prior and Assumption 2.3. Thus, we see that an isotropic prior satisfies Condition 2.1. To see the
locally log-Lipschitz continuity, Taylor’s expansion yields
| log π(β0 + s1 ) − log π(β0 + s2 )| 6

sup
√

x:06x6B+

σ02 λp log n

|d log ρ/dx(x)|(kβ0 + s1 k − kβ0 + s2 k).

This completes the proof for the case of an isotropic prior.
25

Second, consider a product prior π(β) =

Qp

i=1 πi (βi ).

We have

e 1/2 kβ0 k
log π(β0 ) > p log min πi (0) − Lp
i

e
> p log min πi (0) − LBp
log n
i

q
e
> −LBp(1
+ o(1)) log n − e
c2 log n + log{ det(X ⊤ X)/σ0p }

for some positive constant e
c2 depending only on c appearing in Assumption 2.3. Thus, we see that
a product prior satisfies Condition 2.1. To see the locally log-Lipschitz continuity, the Lipschitz
continuity of log π(β) yields
| log π(β) − log π(β0 )| 6
which completes the proof.

p
X
i=1

e 1/2 kβ − β0 k,
| log πi (βi ) − log πi (β0,i )| 6 Lp



Proof of Proposition 2.3. We present the proof only for the case under Assumption 2.2 (a). The
proof for the case under Assumption 2.2 (b) is completed replacing Lemma A.4 (a) by Lemma A.4
(b).
c1 /(n − p)q/2−1 δe1q for some positive
First, from Lemma A.4 (a), we have P(b
σu2 /σ02 − 1 > δ1 ) 6 e
constant e
c1 depending only on q, because it follows that
σ
bu2 =
6

=
where

kY − X(X ⊤ X)−1 X ⊤ Y k2
σ02 (n − p)σ02

kε − X(X ⊤ X)−1 X ⊤ εk2 + 2kr − X(X ⊤ X)−1 X ⊤ rk2 + |ε⊤ u|2
σ02 (n − p)
e + 2kr − X(X ⊤ X)−1 X ⊤ rk2
ε⊤ Aε
σ02 (n − p)


{I − X(X ⊤ X)−1 X ⊤ }r/k{I − X(X ⊤ X)−1 X ⊤ }rk
u :=
 arbitrary

if {I − X(X ⊤ X)−1 X ⊤ }r 6= 0,
if otherwise,

e := I − X(X ⊤ X)−1 X ⊤ + uu⊤ .
and A
Next, we will show that


max{nq/4 , n}
pq/2+1
c2 q/2
P σ
bu2 (Y )/σ02 − 1 6 −δ1 6 e
+e
c2
(n − p)q δ1q
δ1 (n − p)q/2

(30)

for some positive constant e
c2 depending only on q. Letting Pe be the projection onto the linear
space spanned by {X, (I − X(X ⊤ X)−1 X ⊤ )r}, we have

P σ
bu2 (Y )/σ02 − 1 6 −δ1


6 P {kεk2 − kPeεk2 }/{σ02 (n − p)} 6 1 − δ1



6 P kεk2 /σ02 (n − p) − n/(n − p) 6 −δ1 /2 + P kPeεk2 /σ02 (n − p) > p/(n − p) + δ1 /2 . (31)
26

For the upper bound of the first term on the rightmost hand side in (31), we use Rosenthal’s
inequality:
Lemma B.1 (Rosenthal’s inequality; see [38] and [47].). For some positive constant e
c3 depending
q/2
q/4
2
6e
c3 max{n , n}.
only on q, we have E kε/σ0 k − n
We have, from Markov’s inequality and Rosenthal’s inequality,

q/2
P kεk2 /{σ02 (n − p)} − n/(n − p) 6 −δ1 /2 6 e
c4 max{nq/4 , n}/{δ1 (n − p)q/2 }

(32)

for some e
c4 > 0 depending only on q. For the upper bound of the second term on the rightmost
p
hand side in (31), we use Lemma A.4 (a) with R = p + (n − p)δ1 /2. We thus obtain


q/2
P kPeεk2 /{σ02 (n − p)} > p/(n − p) + δ1 /2 6 e
c4 n1−q/2 /δ1 .
(33)

Combining (32) and (33) with (31) yields (30), which completes the proof for the case under
Assumption A.4 (a).

Proof of Proposition 2.4. The marginal posterior distribution of σ 2 is given by the inverse Gamma
distribution IG(a∗ , b∗ ), where a∗ = µ1 + n/2 − p/2 and b∗ = µ2 + kY − P Y k2 /2. The mean of
this marginal posterior is {2µ2 + k(I − X(X ⊤ X)−1 X ⊤ )Y k2 }/{2µ1 + n − p − 2}; while the variance
is 2{2µ2 + k(I − X(X ⊤ X)−1 X ⊤ )Y k2 }2 /{2µ1 + n − p − 2}2 {2µ1 + n − p − 4}. From Chebyshev’s
inequality,
c1 k(I − X(X ⊤ X)−1 X ⊤ )Y k2 /{n2 (δ1 − |E[σ 2 /σ02 | Y ] − 1|)2 }
Πσ2 (σ 2 : |σ 2 /σ02 − 1| > δ1 | Y ) 6 e
(34)

for some positive constant e
c1 depending only on µ1 and µ2 . We have, from the proof of Proposition
2.3, the upper bound of P(k(I − X(X ⊤ X)−1 X ⊤ )Y k2 /(n − p) − 1 > δ1 /2) and thus complete the
proof.


Appendix C. Proofs for Section 3.1
C.1. Proof of Proposition 3.1. First, we transform a white noise model
dW (t)
dY (t) = f0 (t)dt + √
n
into a Gaussian sequence model
Yl,k = β0,lk + rl,k + εl,k , (l, k) ∈ I,
√
is N (0, 1/n) and rl,k = τ∞ / 2J . This transformation is done via a

where the distribution of εl,k
mapping
Z

Z
Z
f→
ψ(J0 −1)0 (t)f (t)dt, ψ(J0 −1)1 (t)f (t)dt, . . . , ψ(J−1)(2J −1 −1) (t)f (t)dt .

Therefore, if the estimate of L∞ -diameter is provided, Theorem 2.1 will complete the proof.
27

bα ). For f, g ∈ C(fb, R
bα ), let
Next, we derive the upper bound of the L∞ -diameter of C(fb, R
h := f − g. It follows from the triangle inequality and from the approximation ability of the
S-regular wavelet that ||h||∞ 6 e
c1 (A1 + A2 + A3 ) for some e
c1 > 0, where
A1 := 2J0 /2

A2 :=

max

06k62J0 −1

X

2l/2

J0 6l6J−1

A3 :=

X

2l/2

J6l

|hh, ψ(J0 −1)k i|,

max

06k62l −1

max

06k62l −1

|hh, ψl,k i|,

|hh, ψl,k i|.

bα , the quantities A1 and A2 on the right hand side are bounded as max{A1 , A2 } 6
Using the radius R
√
bα for some e
c2 > 0. There exist positive constants e
c3 , e
c4 , e
c5 such that, for sufficiently large
e
c2 2J/2 J R
n depending only on α,
max{A1 , A2 } 6 e
c3

q

2J /n

p
log n

bα 6 e
with probability at least 1 − e
c4 exp(−e
c5 2J log n), since it follows from Theorem 2.1 that R
c6 σ0
on H for some e
c6 > 0. The quantity A3 on the right hand side is bounded as
A3 6

e
c7 2J/2 2−J(s+1/2) u1/2
n

for some e
c7 > 0. This complete the proof.

=e
c7

q

2J /n

p
log n/usn



C.2. Proof of Proposition 3.2. The proof consists of the upper bound of the coverage error and
the estimate of the L∞ -diameter.
Upper bound of the coverage error. We apply Theorem 2.1 for the Gaussian sequence model
Yl,k = β0,lk + rl,k + εl,k , (l, k) ∈ I(J ′ )
and the corresponding credible band by setting X = I2J ′ , wl,k = wl for (l, k) ∈ I(J ′ ), and σ0 =
√
1/ n.
However, an additional treatment for the term supl>J ′ ,06k62l −1 |β0,lk −Yl,k |/wl is required because
R
bα ) depends on (YJ ′ +1,0 , YJ ′ +1,1 , . . .), where Yl,k := ψl,k dY for J ′ 6 l < ∞, 0 6 k 6 2l − 1.
C(fb∞ , R
Let Y := (YJ0 −1,0 , . . .).
e := {Y : supJ ′ 6l,06k62l−1 |Yl,k −β0,lk |/wl 6 δ}.
The treatment that we conduct is as follows. Let H
Note that we have
bα )) 6 P(
P(f0 ∈ C(fb∞ , R

sup

(l,k)∈I(J ′ )
28

bα )
|β0,lk − Yl,k |/wl 6 R

and
bα ))
P(f0 ∈ C(fb∞ , R
> P(Y ∈ {Y :

(l,k)∈I(J ′ )

> P(Y ∈ {Y :

(l,k)∈I(J ′ )

= P(Y ∈ {Y :

sup
sup
sup

(l,k)∈I(J ′ )

|β0,lk − Yl,k |/wl +

sup
J ′ 6l,06k62l −1

bα } ∪ H)
e
|β0,lk − Yl,k |/wl 6 R

bα }) − P(Y 6∈ H)
e
|β0,lk − Yl,k |/wl + δ 6 R

bα }) + P(Y 6∈ H).
e
|β0,lk − Yl,k − sgn(Yl,k − β0,lk )δwl |/wl 6 R

Therefore, setting r := (±wl δ) enables us to apply Theorem 2.1 for this case. Two remarks are in
order. Remark that the sign of r does not affect on the result. Remark also that Assumption 2.1 is
always satisfied because Assumption 2.1 is on a bias term in the model and because there is no bias
e to the upper bound will complete
term in the Gaussian sequence model. Thus, adding P(Y 6∈ H)
the evaluation of the coverage error.
e Let {Nl,k : J ′ < l, 0 6 k 6 2l − 1} be i.i.d. random variConsider the upper bound of P(Y 6∈ H).
ables from the standard Gaussian distribution. It follows from Lemma 4.4 and from the inequality
√
E maxk62l −1 |Nl,k | 6 2l that for some e
c1 , e
c2 > 0,
X
√
e 6
P(Y 6∈ H)
P(max |Nl,k | − E max |Nl,k | > wl (δ/σ0 − 2/un ))
l>J ′

k

6e
c1 e−ec2 nw

2 δ2

.

This completes the evaluation of the coverage error.
bα ). For
Estimate of the L∞ -diameter. We derive the upper bound of the L∞ -diameter of C(fb∞ , R
bα ), we have
f, g ∈ C(fb∞ , R
X
X
kf − gk∞ 6 2
2l/2 max |hh, ψl,k i| + 2
2l/2 max |hh, ψl,k i|,
l<J ′

k62l −1

l>J ′

k62l −1

where h := f − g. The former term on the right hand side in the above inequality is bounded as
X
X
√
√
√
√
′
bα
2l/2 max |hh, ψl,k i| 6
2l/2 l(wl / l){ max |hh, ψl,k i/wl |} 6 e
c3 2J /2 J ′ max′ {wl / l}R
l<J ′

k62l −1

k62l −1

l<J ′

l<J

for some e
c3 > 0. There exist e
c4 , e
c5 , e
c6 > 0 such that for sufficiently large n depending only on α,
q
X
wl
l/2
2
max |hh, ψl,k i| 6 e
c4 2J ′ (log n)/n max′ √
l
l<J
k62 −1
l
l6J ′

′
bα 6
with probability at least 1 − e
c5 exp(−e
c6 2J log n), since it follows from (9) and (10) that R
p
(log n)/n on H for some e
c7 > 0,
The latter term on the right hand side is bounded as
X
′
2
2l/2 max |hh, ψl,k i| 6 e
c8 2−J (s+1/2) un

l>J ′

k62l −1

for some e
c8 > 0 because of the approximation property of the S-regular wavelet. This completes
the proof.

29

C.3. Proof of Proposition 3.3. The proof follows essentially the same line as that of Proposition
3.1. The only difference is to provide the lower bound of the L∞ -diameter. It follows that there
exists e
c1 > 0 for which we have, for sufficiently large n such that the coverage error is bounded by
p
α/2, supf,g∈C(fb,Rbα ) kf − gk∞ > e
c1 2J /n with a high probability, since supf,g∈C(fb,Rbα ) kf − gk∞ >
√
bα . This completes the proof.
2J R

Appendix D. Proofs for Section 3.2
Proof of Prosposition 3.4. The proof follows the line of that of Proposition 3.3. The only difference
(1)
is to deal with the non-orthogonality of {vl,k }.
(1)
When denoting by Ye = (Yel,k )(l,k)∈I(J) the coefficients of dY with respect to {v : (l, k) ∈ I(J)},
l,k

Ye follows

Yel,k = κl,k β0,lk + re + εl,k , (l, k) ∈ I(J),

where β0 = (β0,lk )(l,k)∈I(J) is the coefficient vector of f0 with respect to {ψl,k : (l, k) ∈ I(J)}, re :=
√
′ / 2J , and ε = (ε )
τ∞
l,k (l,k)∈I(J) follows the Gaussian distribution with mean zero and covariance
matrix Σ.
(1)
From the near-orthogonality of {vl,k }, there exist b and b such that bI2J /n  Σ  bI2J /n , where
(1)

(2)

constants b, b depend only on the frame constants of {vl,k } and {vl,k }. Let κ := diag(κl,k )(l,k)∈I(J) .
√
Thus, setting Y = Σ−1/2 Ye , X = Σ−1/2 κ, r = Σ−1/2 re, σ0 = 1, and wl,k = lκ−1 , Theorem 2.1
l,k

completes the proof.



Appendix E. Proof for Section 3.3
First of all, we transform the model into the following regression model based on the approximation by p basis functions {ψjp : 1 6 j 6 p}:
Y = Xβ0 + r + ε,
where Y = (Y1 , . . . , Yn )⊤ , X = (X1 , . . . , Xn )⊤ with each component Xi of which the j(∈ {1, . . . , p})th component is ψjp (Ti ), and r = (r1 , . . . , rn )⊤ with each component ri = f0 (Ti ) − ψ p (Ti )⊤ β0 . Here,
P
recall that β0 ∈ argminE|f0 (T1 ) − pj=1 ψjp (T1 )βj |2 .

E.1. Technical lemmas. Before the proof, we state four technical tools used in the proof. Let
N(n) be a random n-vector from N (0, σ02 In ), and N(p) be a random p-vector from N (0, σ02 Ip ). Let
B = (Bij ) := (Eψip (T1 )ψjp (T1 )) and recall ψep (·) := ψ p (·)/kψ p (·)k and ξp := kkψ p (·)kk∞ .

Lemma E.1 (Matrix Chernoff inequality; [44]). Let {Ai : i = 1, . . . , n} be an i.i.d. sequence of
positive semi-definite and self-adjoint p × p matrices of which the maximum eigenvalues are almost
surely bounded by R. Then,
X
P(λmin (
Ai /n) 6 (1 − δ)λmin (E[A1 ])) 6 p{e−δ /(1 − δ)1−δ }nλmin (E[A1 ])/R for δ ∈ [0, 1] and
X
P(λmax (
Ai /n) 6 (1 − δ)λmax (E[A1 ])) 6 p{eδ /(1 + δ)1−δ }nλmin (E[A1 ])/R for δ > 0,
30

where λmin (·) and λmax (·) are the maximum and the minimum eigenvalues.
Lemma E.2 (Lemma 4.2 in [5]). Under Conditions 3.3-3.4, the equality
√
B −1 X ⊤ ε
ψep (·)⊤ n(βb − β0 ) − ψep (·)⊤ √
n

6 R1 + R2
∞

holds, where R1 and R2 are random variables such that there exist positive constants e
c1 and e
c2
depending only on q appearing in Assumption 2.2 (a) for which we have
q

√
ξp2 log p 1/q √
e
2
c1 η
log p + pτ∞ )
under Assumption 2.2 (a),
n (n
q
R1 6
2 log p √
√
√
ξ
e
p
c1 η 2
n ( log n log p + pτ∞ ) under Assumption 2.2 (b),
p
R2 6 ce2 η log pτ∞
with probability at least 1 − e
c2 /η, for any η > 1.

Remark E.1. Belloni et al. [5] provides the proof under Assumption 2.2 (a). The proof under
Assumption 2.2 (b) is almost the same noting that n1/q comes from E[maxi=1,...,n |εi |].

Lemma E.3 (Corollary 2.2 and Proposition 3.3 in [14]). Under Conditions 3.3-3.4, for any η > 0,
d
e=
there exists a random variable Z
kψep (·)⊤ B −1 N(p) k∞ such that the inequality
√ B −1 X ⊤ ε
− Ze
ψep (·)⊤ n
n
∞

1/2
1/3
1/q log n ξ
n)3/4 ξp
n)2/3 ξp
p
e
+ (logη1/2
+ (logη1/3
c1 n η1/2
n1/2
n1/4
n1/6
6
3/4 ξ 1/2
2/3 ξ 1/3
ξ
e
p
p
+ (log n)
c log n p + (log n)
1 η1/2 n1/2

η1/2

n1/4

η1/3

n1/6

under Assumption 2.2 (a),
under Assumption 2.2 (b)

holds with probability at least 1 − e
c2 {η + (log n)/n} for some e
c1 , e
c2 > 0 not depending on n and p.

√
Lemma E.4. Under Condition 3.4, the inequality Ekψep (·)⊤ B −1 N(p) k∞ 6 e
c1 log p holds for some
positive constant e
c1 depending only on C5 appearing in Condition 3.4.
Proof. From Dudley’s entropy integral (e.g.,see Corollary 2.2.8 in [46]),
E[kψep (·)⊤ B −1/2 N(p) k∞ ]

6 E[|ψep (0)⊤ B −1/2 N(p) |] + E[ sup

6b+

Z

θ

0

p

t6=t′ ∈[0,1]

log N ([0, 1], dX , δ)dδ,

|ψep (t)⊤ B −1/2 N(p) − ψep (t′ )⊤ B −1/2 N(p) |]

where N ([0, 1], dX , δ) is a δ-covering number of [0, 1] with respect to
dX (t, t′ ) := {E[ψep (t)⊤ B −1/2 N(p) − ψep (t′ )⊤ B −1/2 N(p) ]2 }1/2

and θ := supt∈[0,1] dX (t, 0). Since θ is bounded by 2b,
Z 2b p
Z θp
log N ([0, 1], dX , δ)dδ 6
log N ([0, 1], dX , δ)dδ.
0

31

0

Since it follows from the bound on covering numbers of functions Lipschitz in one parameter

(Theorem 2.7.11 in [46]) that we have, for some e
c2 > 0, N ([0, 1], dX , δ) 6 e
c2 pC5 /δ , the inequality
Z 2b p
Z 2b p
p
log N ([0, 1], dX , δ)dδ 6 C5 log p +
log(e
c2 b/δ)dδ
0

0

holds. Thus, we obtain the desired inequality.



Lemma E.5. Under Conditions 3.3-3.4, there exists a positive constant e
c1 not depending on n and
p for which we have
p

sup P kψep (·)⊤ B −1/2 N(p) k∞ − x 6 R 6 e
c1 R log p, R > 0.
x∈R

Proof. From Theorem 2.1 in [14],


sup P kψep (·)⊤ B −1/2 N(p) k∞ − x 6 R 6 e
c1 RE[kψep (·)⊤ B −1/2 N(p) k∞ ]
x∈R

and thus Lemma E.4 completes the proof.



E.2. Proof of Proposition 3.5. We only provide the proof under Assumption 2.2 (a). We mention
that although the proof is not a direct consequence of Theorem 2.1, we can follow the same line as
the proof of Theorem 2.1.
Modification of the test set H. Take c1 > 0 sufficiently large. Before providing the coverage error
and the L∞ -diameter, we modify the test set
p
b ) − β0 )k 6 c1 p log n} ∩ {Y : Πσ2 (|σ 2 /σ 2 − 1| > δ1 | Y ) 6 δ2 }
H = {Y : kX(β(Y
0

in Proposition 2.5 as

b ) − β0 )k 6 c1
H := {(X, Y ) :kX(β(Y

p

p log n, (b/2)2 Ip  X ⊤ X/n  (2b)2 Ip }

∩ {(X, Y ) : Πσ2 (|σ 2 /σ02 − 1| > δ1 | Y ) 6 δ2 }.

The probability P((X, Y ) 6∈ H) is evaluated as follows:
P((X, Y ) 6∈ H) 6A1 + A2 + A3 + δ3 ,
where
p

p log n/2, (b/2)2 Ip  X ⊤ X/n  (2b)2 Ip ),
p
A2 := P(kX(X ⊤ X)−1 X ⊤ rk > c1 p log n/2),

A1 := P(kX(X ⊤ X)−1 X ⊤ εk > c1

A3 := P(X 6∈ {X : (b/2)2 Ip  (X ⊤ X)/n  (2b)2 Ip }).

It follows from Lemma A.4 that A1 6 e
c1 e−ec2 p log n for some e
c1 , e
c2 > 0. It follows from the Markov
inequality that
A2 6

E[r ⊤ X(X ⊤ X)−1 X ⊤ r]
n τ22
6
.
p log n
log n p

It follows from Lemma E.1 that A3 6 e
c1 e−ec2 p log n .
32

bα .
Upper bound for the coverage error. At the first step, we derive a high-probability bound of R
From Proposition 2.5, for (X, Y ) ∈ H,
bα | Y, X} − P(kψep (·)⊤ (X ⊤ X)−1 X ⊤ N(n) k∞ 6 R
bα | Y, X) 6 ω,
Πβ {kψep (·)⊤ (βb − β0 )k∞ 6 R

√
where ω := φeΠβ (c1 p log n) + c1 δ1 p log n + δ2 + δ3 + c1 e−c2 p log n . Here the constant c2 is determined
in Proposition 2.5. Letting G be the distribution function of kψep (·)⊤ (X ⊤ X)−1 X ⊤ N(n) k∞ , G−1 its
quantile function, we have
bα 6 G−1 (1 − α + ω) for (X, Y ) ∈ H.
G−1 (1 − α − ω) 6 R

(35)

Thus, we complete the first step.
At the second step, we derive approximation bounds of kψep (·)⊤ (X ⊤ X)−1 X ⊤ N(n) k∞ and kψep (·)⊤
√
(βb − β0 )k∞ by nkψep (·)⊤ B −1/2 N(p) k∞ in Kolmogorov distance. Let η = ηn be arbitrary divergent
sequence. Let
√
ρ1 := sup P(kψep (·)⊤ n(βb − β0 )k∞ 6 R) − P(kψep (·)⊤ B −1/2 N(p) k∞ 6 R) ,
R>0

√
ρ2 := sup P(kψep (·)⊤ n(X ⊤ X)−1 X ⊤ N k∞ 6 R) − P(kψep (·)⊤ B −1/2 N(p) k∞ 6 R) , and
R>0

γ(R) := sup P(|kψep (·)⊤ B −1/2 N(p) k∞ − x| 6 R).
x>0

and consider upper bounds of ρ1 , ρ2 , and γ(R).
There exist e
c3 , e
c4 > 0 for which we have four inequalities
P



√

n

ψep (·)⊤ (βb − β0 )
n

ξp2 /n

−1

⊤

>e
c3 η

6e
c4 /η 2 ,

P


√

ep

⊤

⊤

n kψ (·) (X X)

1/2 p

P

n

>e
c3 η

1/2 1/q
ξp2 /n
(n

6e
c4 (1/η 2 + log n/n),

log p(n1/q

∞

e
−Z

log n) +

∞

o
p
p
√
log p + pτ∞ ) + log pτ∞

⊤

X N(n) k∞ − kψ (·) B

ψep (·)⊤ B −1 X ⊤ ε/n
n

− ψep (·)⊤ B −1 X ⊤ ε/n

ep

6e
c4 /η 2 ,

√

∞

−1

⊤

X N(n) /nk∞ > e
c3 η

1/4
ξp2 /n
(log n)3/4
33

o

+e
c3 η

2/3

n



1/2 1/q
ξp2 /n
n

(36)

log p

1/6
ξp2 /n
(log n)2/3



(37)

o

(38)

and
P


√

n

ψep (·)⊤ B −1 X ⊤ N(n) /n

>e
c3 η

n

1/2 1/q
ξp2 /n
(n

∞

− Ze

log n) +

6e
c4 (1/η 2 + log n/n).

1/4
ξp2 /n
(log n)3/4

o

+e
c3 η

2/3

n

1/6
ξp2 /n
(log n)2/3

o

(39)

The first two inequalities follows from Lemma E.2 and the last two inequalities follows from Lemma
E.3.
It follows from inequalities (36) and (38) and from Lemma E.5 that for some e
c5 > 0,
ρ1 6e
c5 (A4 + A5 ),

(40)

where
o
n
1/2 1/q
1/6
log n
1
1/2
2
2/3
2
and
+
+
η(log
p)
max
ξ
/n
n
log
n,
ξ
/n
(log
n)
p
p
η2
n
o
n
1/2
.
A5 := η(log p)τ∞ max 1, pξp2 /n

A4 :=

(41)

Likewise, it follows from inequalities (37) and (39) and from Lemma E.5 that for some e
c5 > 0,
ρ2 6e
c5 A4 .

It follows from Lemma E.5 that for some e
c5 > 0,
Thus, we complete the second step.
Finally, we have

γ(R) 6 e
c5 R

p

log p.

(42)

(43)

bα )) 6 P{kψep (·)⊤ (βb − β0 )k∞ 6 G−1 (1 − α + ω) + τ } + P{(X, Y ) 6∈ H}
P(f0 ∈ C(fb, R
√
6 P{kψep (·)⊤ B −1/2 N(p) / nk∞ 6 G−1 (1 − α + ω) + τ } + ρ1 + P{(X, Y ) 6∈ H}
√
6 P{kψep (·)⊤ B −1/2 N(p) / nk∞ 6 G−1 (1 − α + ω)}
√
+ γ( nτ ) + ρ1 + P{(X, Y ) 6∈ H}
√
6 ω + ρ1 + ρ2 + γ( nτ ) + P((X, Y ) 6∈ H),
bα )) − (1 − α).
and thus from (40)-(43), taking η = nδ , we obtain the upper bound of P(f0 ∈ C(fb, R
bα )) − (1 − α), which provides the desired
Likewise, we obtain the lower bound of P(f0 ∈ C(fb, R
bound of the coverage error.

p
c6 (log p)/n for some e
c6 > 0.
Estimate of the L∞ -diameter. We will show that G−1 (1 − α + ω) 6 e
It follows from the concentration inequality for the suprema of the Gaussian process, from Lemma
E.4, by taking sufficiently large p depending only on α, that for sufficiently large e
c7 > 0,
p
2
P(kψep (·)⊤ B −1/2 N(p) k∞ − E[kψep (·)⊤ B −1/2 N(p) k∞ ] > e
c7 log p) 6 e−ec7 log p < α − ω − ρ2 .
34

Thus, observing
G−1 (1 − α + ω) := inf{R : P(kψep (·)⊤ (X ⊤ X)−1 X ⊤ N(n) k∞ > R) 6 α − ω}
√
6 inf{R : P(kψep (·)⊤ B −1/2 N(p) / nk∞ > R) 6 α − ω − ρ2 }


√
√
= inf R : P kψep (·)⊤ B −1/2 N(p) k∞ / n − E[kψep (·)⊤ B −1/2 N(p) / nk∞ ]


√
p
⊤
−1/2
> R − E[kψe (·) B
N(p) / nk∞ ] 6 α − ω − ρ2

and taking sufficiently large p depending only on α, we have G−1 (1 − α + ω) .

p

(log p)/n.



√
E.3. Proof of the bound on τ . In this subsection, we show that τ . τ∞ / p for periodic S > 2regular wavelets. We consider a wavelet pair (φ, ψ) satisfying the following three assumptions:
• There exists an integer N for which the support of φ is included in [0, N ] and the support
of ψ is included in [−N + 1, N ];
• φ and ψ are C S ;
P
• The inequality inf x∈R k∈Z {ψ(x − k)}2 > 0 holds.

We periodize the pair (φ, ψ) as follows:
X
X
(per)
(per)
φl,k (t) :=
2l/2 φ(2l t + 2l m − k) and ψl,k (t) :=
2l/2 ψ(2l t + 2l m − k)
m∈Z

m∈Z

(per)

(per)

for k = 0, . . . , 2l − 1 and l = 1, . . . , J. Taking J0 as 2J0 > 2N , {φJ0 ,k : k = 0, . . . , 2J0 − 1} ∪ {ψl,k :
k = 0, . . . , 2l − 1, l = J0 , . . . , J} forms p = 2J basis functions based on periodic S-regular wavelets.
√
It suffices to show that inf t∈[0,1] kψ p (t)k & p. Since 2J > 2N and since the support of ψ is
included in [−N + 1, N ], we have
p

2

J

kψ (t)k > 2

J −1
2X

{

X

k=0 m∈Z

J

J

2

J

ψ(2 t + 2 m − k)} = 2

J −1
2X

X

k=0 m∈Z

{ψ(2J t + 2J m − k)}2

and the rightmost quantity in the above inequality is bounded below by 2J inf x∈R
Thus we complete the proof.

P

2
k∈Z {ψ(x−k)} .

References
[1] F. Abramovich and B. Silverman. Wavelet decomposition approaches to statistical inverse problems. Biometrika,
85:115–129, 1998.
[2] T. Anderson. The integral of a symmetric unimodal function. Proc. Amer. Math. Soc., 6:170–176, 1955.
[3] Y. Atchadé. On the contraction properties of some high-dimensional quisi-posterior distributions. Ann. Statist.,
45:2248–2273, 2017.
[4] Y. Baraud. Model selection for regression on a fixed design. Probab. Theory Relat. Fields, 117:467–493, 2000.
[5] A. Belloni, V. Chernozhukov, D. Chetverikov, and K. Kato. Some new asymptotic theory for least squares series:
Pointwise and uniform results. J. Econometrics, 186:345–366, 2015.
[6] P. Bickel and M. Rosenblatt. On some global measures of the derivations of density function estimates. Ann.
Statist., 1:1071–1095, 1973.
35

[7] D. Bontemps. Bernstein–Von Mises theorems for Gaussian regression with increasing number of reguressors.
Ann. Statist., 39:2557–2584, 2011.
[8] S. Boucheron and E. Gassiat. A Bernstein–von mises theorem for discrete probability distributions. Electron. J.
Statist., 3:114–148, 2009.
[9] I. Castillo and R. Nickl. Nonparametric Bernstein–von Mises theorems in Gaussian white noise. Ann. Statist.,
41:1999–2028, 2013.
[10] I. Castillo and R. Nickl. On the Bernstein–von Mises phenomenon for nonparametric Bayes procedures. Ann.
Statist., 42:1941–1969, 2014.
[11] I. Castillo and J. Rousseau. A Bernstein–von mises theorem for smooth functionals in semiparametric models.
Ann. Statist., 43:2353–2383, 2015.
[12] V. Chernozhukov, D. Chetverikov, and K. Kato. Gaussian approximations and multiplier bootstrap for maxima
of sums of high-dimensional random vectors. Ann. Statist., 41:2786–2819, 2013.
[13] V. Chernozhukov, D. Chetverikov, and K. Kato. Anti-concentration and honest, adaptive confidence bands. Ann.
Statist., 42:1787–1818, 2014.
[14] V. Chernozhukov, D. Chetverikov, and K. Kato. Gaussian approximation of suprema of empirical processes.
Ann. Statist., 42:1564–1597, 2014.
[15] V. Chernozhukov, D. Chetverikov, and K. Kato. Comparison and anti-concentration bounds for maxima of
Gaussian random vectors. Probab. Theory Related Fields, 162:47–70, 2015.
[16] V. Chernozhukov, D. Chetverikov, and K. Kato. Central limit theorems and bootstrap in high dimensions. Ann.
Probab., 45:2309–2352, 2017.
[17] V. Chernozhukov and H. Hong. An MCMC approach to classical estimation. J. Econometrics, 115:293–346, 2003.
[18] G. Claeskens and I. Van Keilegom. Bootstrap confidence bands for regression curves and their derivatives. Ann.
Statist., 31:1852–1884, 2003.
[19] R. DeVore and G. Lorentz. Constructive Approximation. Springer, 1993.
[20] D. Donoho. Nonlinear solution of linear inverse problems by wavelet-vaguelette decomposition. Applied Comput.
Harmonic Analysis, 2:101–126, 1995.
[21] J.-P. Florens and A. Simoni. Nonparametric estimation of an instrumental regression: A quasi-Baysian approach
based on regularized posterior. J. Econometrics, 170:458–475, 2012.
[22] D. Freedman. On the Bernstein–von Mises theorem with infinite dimensional parameters. Ann. Statist., 27:1119–
1140, 1999.
[23] C. Gao and H. Zhou. Bernstein–von Mises theorems for functionals of the covariance matrix. Electron. J. Statist.,
10:1751–1806, 2016.
[24] S. Ghosal. Asymptotic normality of posterior distributions in high-dimensional linear models. Bernoulli, 5:315–
331, 1999.
[25] S. Ghosal. Asymptotic normality of posterior distributions for exponential families when the number of parameters tends to infinity. J. Multivariate Anal., 74:49–68, 2000.
[26] E. Giné and R. Nickl. Confidence bands in density estimation. Ann. Statist., 38:1122–1170, 2010.
[27] E. Gine and R. Nickl. Mathematical Foundations of Infinite-Dimensional Statistical Models. Cambridge Univ.
Press, 2016.
[28] P. Hall. On convergence rates of suprema. Prob. Theory Related Fields, 89:447–455, 1991.
[29] D. Hanson and E. Watson. A bound on tail probabilities for quadratic forms in independent random variables.
Ann. Math. Statist., 42:1079–1083, 1971.
[30] I. Johnstone. Gaussian estimation: Sequence and wavelet models. Unpublished manuscript.
[31] I. Johnstone. High dimensional Bernstein–von Mises: simple examples. IMS Collections, 6:87–98, 2010.
[32] K. Kato. Quasi-Bayesian analysis of nonparametric instrumental variables models. Ann. Statist., 41:2359–2390,
2013.
[33] H. Leahu. On the Bernstein–von Mises phenomenon in the Gaussian white noise model. Elec. J. Statist., 5:373–
404, 2011.
36

[34] Y. Liao and W. Jiang. Posterior consistency of nonparametric conditional moment restricted models. Ann.
Statist., 39:3003–3031, 2011.
[35] F. Nazarov. On the maximal perimeter of a convex set in Rn with respect to a gaussian measure. In Geometric
Aspects of Functional Analysis (2001-2002), pages 169–187. Springer, 2003.
[36] F. Olver, D. Lozier, R. Boisvert, and C. Clark. NIST Handbook of Mathematical Functions. Cambridge University
Press, 2010.
[37] M. Panov and V. Spokoiny. Finite sample Bernstein–von Mises theorem for semiparametric problems. Bayesian
Analysis, 10:665–710, 2015.
[38] V. Petrov. Limit theorems of probability theory. Sequences of independent random variables. Oxford University
Press, 1995.
[39] K. Ray. Adaptive Bernstein–von Mises theorems in Gaussian white noise. Ann. Statist., 45:2511–2536, 2017.
[40] V. Rivoirard and J. Rousseau. Bernstein–von Mises theorem for linear functionals of the density. Ann. Statist.,
40:1489–1523, 2012.
[41] M. Rudelson and Roman Vershynin. Hanson–Wright inequality and sub-gaussian concentration. Electron. Commun. Probab., 18:1–9, 2013.
[42] N. Smirnov. On the construction of confidence regions for the density of distribution of random variables. Doklady
Akad. Nauk SSSR, 74:189–191, 1950.
[43] V. Spokoiny. Bernstein–von Mises theorem for growing parameter dimension. arXiv:1302.3430.
[44] J. Tropp. User-friendly tail bounds for sums of random matrices. Found. Comput. Math., 12:389–434, 2012.
[45] A. Tsybakov. Introduction to Nonparametric Estimation. Springer, 2009.
[46] A. van der Vaart and J. Wellner. Weak Convergence and Empirical Processes : With Applications to Statistics.
Springer, 1996.
[47] B. von Bahr and C. Esseen. Inequalities for the rth absolute moment of a sum of random variables 1 6 r 6 2.
Ann. Math. Statist., 36:299–303, 1965.
[48] Y. Yang, A. Bhattacharya, and D. Pati. Frequentist coverage and sup-norm convergence rate in Gaussian process
regression. arXiv:1708.04753.
[49] W. Yoo and S. Ghosal. Supremum norm posterior contraction and credible set for nonparametric multivariate
regression. Ann. Statist., 44:1069–1102, 2016.
(K. Yano) Department of Mathematical Informatics, Graduate School of Information Science and
Technology, University of Tokyo, 7-3-1 Hongo, Bunkyo-ku, Tokyo 113-0033, Japan.
E-mail address: yano@mist.i.u-tokyo.ac.jp
(K. Kato) Graduate School of Economics, University of Tokyo, 7-3-1 Hongo, Bunkyo-ku, Tokyo
113-0033, Japan.
E-mail address: kkato@e.u-tokyo.ac.jp

37

