BAYESIAN INVERSE PROBLEMS WITH
NON-COMMUTING OPERATORS

arXiv:1801.09540v1 [] 29 Jan 2018

PETER MATHÉ
Abstract. The Bayesian approach to ill-posed operator equations in Hilbert space recently gained attraction. In this context,
and when the prior distribution is Gaussian, then two operators
play a significant role, the one which governs the operator equation, and the one which describes the prior covariance. Typically
it is assumed that these operators commute. Here we extend this
analysis to non-commuting operators, replacing the commutativity assumption by a link condition. We discuss its relation to the
commuting case, and we indicate that this allows to use interpolation type results to obtain tight bounds for the contraction of
the posterior Gaussian distribution towards the data generating
element.

1. Setup and Problem formulation
We shall consider the equation
y δ = Kx + δξ,

(1)

where δ > 0 prescribes a base noise level, and K : X → Y is a compact
linear operator between Hilbert spaces. The noise element ξ is a weak
random element. If this random element ξ has covariance Σ, then we
may pre-whiten Equation (1) to get
(2)

z δ = Σ−1/2 Kx + δΣ−1/2 ξ,

which is now a linear inverse problem under Gaussian white noise.
In the Bayesian framework we choose a prior for x. Since this is
2
assumed to be a tight and centered Gaussian measure N (0, δα C0 ), it
is equipped with a (scaled) covariance C0 which has a finite trace. As
calculations show, the relevant operator in the analysis will then be
1/2
B := Σ−1/2 KC0 , we refer to [2] for details.
Therefore, we have (at least) two operators to consider, the prior
covariance operator C0 as well as the operator H := B ∗ B. Both operators are non-negative compact self-adjoint operators in X. To simplify
the analysis we shall assume that the operator C0 is injective.
Within the present, very basic Bayesian context much is known, and
we refer to the recent survey [2] and references therein. In particular we
know that the posterior is (tight) Gaussian, and we find the following
Date: January 30, 2018.
1

2

PETER MATHÉ

representation for the posterior mean and covariance for the model
from (2):
1/2

(3)

xδα = C0 (αI + H)−1 B ∗ z δ , (posterior mean) and

(4)

Cαδ = δ 2 C0 (αI + H)−1 C0 (posterior covariance).

1/2

1/2

In the study [2] the authors highlight that the (square of the) contraction of the posterior towards the element x∗ , generating the data z δ is
driven by the squared posterior contraction (SPC), given as
(5)

δ

SPC(α, δ) := Ex Ezα kx∗ − xk2 ,
∗

α, δ > 0,

where the outward expectation is taken with respect to the data generating distribution, that is, the distribution generating z δ when x∗ is
given, and the inward expectation is taken with respect to the posterior distribution, given data z δ and having chosen a parameter α.
Moreover, the SPC has a decomposition
 
(6)
SPC(α, δ) = b2x∗ (α) + V δ (α) + tr Cαδ .
2

with the squared bias b2x∗ (α) := x∗ − Ex xδα , the estimation vari 
∗
∗
2
ance V δ (α) := Ex xδα − Ex xδα , and the posterior spread tr Cαδ .
Proposition 1 ibid asserts that the estimation variance V δ (α) is always
smaller than the posterior spread, thus we need to bound the bias and
posterior spread, only. Therefore we recall the form of the bias from
Lemma 1 ibid. as
∗

(7)

1/2

−1/2 ∗

bx∗ (α) = C0 sα (H)C0

x

,

α > 0,

where we abbreviate sα (H) = α (α + H)−1 . Plainly, if C0 and H commute, then we have that
(8)

bx∗ (α) = ksα (H)x∗ k .

Also, it is easily seen from the cyclic commutativity of the trace that
 


(9)
tr Cαδ = δ 2 tr (α + H)−1 C0 .

Here we aim at providing tight bounds, both for the bias and the posterior spread for non-commuting operators C0 and H. For Bayesian inverse problems we are aware of only one study [1], and we shall return to
more details, later. Within the ’classical’ theory of ill-posed problems
this was met earlier. The typical situation arises, when smoothness is
measured in some Hilbert scale, say Xa , where a ∈ R describes smoothness properties, generated by some (unbounded) operator. In order to
derive convergence rates for the reconstruction of the solution to the
equation (1) the operator K is then linked to the scale by assuming
that there is some µ > 0 such that
kKxkY ≍ kxk−µ ,

x ∈ X−µ .

BAYESIAN INVERSE PROBLEMS

3

This was first presented in [12] when considering regularization in
Hilbert scales, see also [7, Cor. 8.22] for a similar condition. Subsequently, in particular when measuring smoothness in a more general
sense in terms of variable Hilbert scales, such links were assumed in a
more general context. A comprehensive study is [11], which shows how
interpolation in variable Hilbert scales can be used in order to derive
error bounds under such link conditions. The present study may be
seen as an application of these techniques to Bayesian inverse problems.
We shall start in Section 2 with introducing the link condition, discuss its implications, and we relate this to the commuting case. Then
we shall use the decomposition of the SPC as in (6), and thus find
bounds for the bias in Section 3, and bounds for the posterior spread
in Section 4, respectively. We then summarize the results for giving
bounds for the squared posterior contraction in Section 5, and we conclude with a discussion in Section 6.
2. Linking operators and scales of Hilbert spaces
In order to introduce the fundamental link condition we need some
notions and auxiliary calculus.
2.1. Link condition. We first recall the following concepts from [2, 4].
Definition 1 (index function). A function ψ : (0, ∞) → R+ is called
an index function if it is a continuous non-decreasing function with
ψ(0) = 0.
Definition 2 (partial ordering for index functions). Given two index
functions g, h we shall write g ≺ h if the function t 7→ h(t)/g(t) is an
index function (h tends to zero faster than g).
The link condition which we are going to introduce now will be based
upon a partial ordering of self-adjoint operators, and we refer to [3] for
a comprehensive account. Although the monograph formally treats
matrices, only, most of the results transfer to (compact bounded) operators in Hilbert space.
Definition 3 (partial ordering for self-adjoint operators). Let G and
G′ be bounded self-adjoint operators in some Hilbert spaces X. We say
that G ≤ G′ if for all x ∈ X the inequality hGx, xi ≤ hG′ x, xi holds
true.
The following concept of ’concavity’ is the extension of concavity
from real functions to self-adjoint operators by functional calculus.
Definition 4. Let f : [0, a] → R+ be a continuous function. It is called
operator concave if for any pair G, H ≥ 0 of self-adjoint operators with
spectra in [0, a] we have
G+H
f (G) + f (H)
(10)
f(
)≥
.
2
2

4

PETER MATHÉ

We mention that operator concave function must be operator monotone, i.e., if G ≤ H then we will have that f (G) ≤ f (H), see [3,
Thm. V.2.5]1. The concept of operator concavity will be crucial for
the interpolation, below. However, the above partial ordering has also
implications for the ranges of the operators, and this is comprised in
Theorem 1 (Douglas’ Range Inclusion Theorem, see [6]). Consider
operators S : Y → X and T : Z → X, acting between Hilbert spaces.
The following assertions are equivalent.
(1) R(S) ⊂ R(T ),
(2) there is a constant C such that SS ∗ ≤ C 2 T T ∗ ,
(3) there is a constant C such that kS ∗ xkY ≤ C kT ∗ xkZ x ∈ X,
(4) there is a ’factor’ R : Y → Z, kRk ≤ C, such that S = T R.
Of course, for self-adjoint operators S, T : X → X the norm estimate from (3) is again for S = S ∗ and T = T ∗ . Also, if the operator T is injective then the composition T −1 S is a bounded operator
and kT −1 Sk = kRk ≤ C.
As it was stressed above, the governing operators C0 and H = B ∗ B
are non-negative self-adjoint. As an immediate application, by considering B : X → Y and its self-adjoint companion B ∗ B : X → X we
plainly have that kBukX = (B ∗ B)1/2 u , u ∈ X, such that R(B ∗ ) =
X

R((B ∗ B)1/2 ) = R(H 1/2 ).

Before formally introducing the link assumption, we first make the
standing assumption that the compound mapping Σ−1/2 K : X → Y
is bounded. The link condition will provide us with a ’tuning index
function’ ψ such that the ranges of ψ(C0 ) and K ∗ Σ−1/2 coincide, and
hence we shall assume that
kψ(C0 )vkX ≍ Σ−1/2 Kv

Y

1/2

,

v ∈ X.

Using this with v := C0 u, u ∈ X we arrive at
1/2

kΘ(C0 )ukX ≍ Σ−1/2 KC0 u
where we introduced the function
(11)

Θ(t) = Θψ (t) :=

Y

= H 1/2 u

√
tψ(t),

Y

,

u ∈ X,

t > 0,
1/2

and the operator H is gives an before from B := Σ−1/2 KC0 as H =
B ∗ B. We observe that its square Θ2 is strictly monotone, and it increases super-linearly. Below, the inverse will play an important role,
and we stress that this will be a sub-linearly increasing index function, as this typically is the case for power type functions g(t) := tq
with 0 < q ≤ 1. So, we formally make the following
1Formally,

operator monotone functions are defined on [0, ∞). The asserted
monotonicity can be seen from the proof of Theorem V.2.5 ibid.

BAYESIAN INVERSE PROBLEMS

5

Assumption 1 (link condition). There are an index function ψ, and
constants 0 < m ≤ 1 ≤ M < ∞ such that
(12)

m kψ(C0 )uk ≤ Σ−1/2 Ku ≤ M kψ(C0 )uk ,

u ∈ X.

Moreover, with the function Θ from (11) the related function

−1 1/2
(13)
f0 (s) := Θ2
(s)
, s > 0,
has an operator concave square f02 .

We first draw the following consequence. For this we let
√
(14)
ϕ0 (t) := t, t > 0,
throughout this study.
1/2

Proposition 1. Under Assumption 1 we have that R(C0 ) = R(f0 (H)).
In particular the operator f0 (H)ϕ0 (C0 )−1 is norm bounded by M.
Proof. Arguing as above, the inequalities in (12) have their counterpart
for the function Θ as
(15)

m kΘ(C0 )uk ≤ H 1/2 u ≤ M kΘ(C0 )uk ,

u ∈ X.

We can rewrite the left hand side as
Θ2 (C0 ) ≤

1
H.
m2

Since f02 is assumed to be operator concave, and hence operator monotone we conclude that


1
1
2
(16)
C0 ≤ f0
H ≤ 2 f02 (H) ,
2
m
m
where we used that 0 < m ≤ 1. Rewriting this in terms of a norm
1/2
≤ m1 kf0 (H)ukX , u ∈ X, and hence
inequality shows that C0 u
1/2
R(C0 )

X

that
⊆ R(f0 (H)). The other inclusion is proven similar by
using the right hand side of (15), and hence it is omitted. The norm
boundedness is a consequence of Theorem 1, as it was stressed after its
formulation.

Basically, the above inequalities in (15) correspond to Assumption 3.1
(2 & 3) from [1], see Section 6, ibid., if the function Θ is a power function.
2.2. Linking commuting operators. In previous studies, dealing
with commuting operators, no functional dependence was assumed,
except the recent survey [2]. We start with the following technical
assertion.

6

PETER MATHÉ

Lemma 1. Suppose that we have two commuting self-adjoint nonnegative compact operators A, B in Hilbert space. If the operator A has
only simple eigenvalues then there is a continuous function ψ : [0, kAk] →
R+ , with limu→0 ψ(u) = 0, such that ψ(A) = B.
Proof. Indeed, the pair A, B is commonly diagonalizable, and we may
consider (infinite) diagonal matrices Ds and Dt , having the eigenvalues
of A and B on the diagonals. Since all eigenvalues of A were assumed to
be simple, we can assume that s1 > s2 > · · · > 0. The corresponding
eigenvalues for B will not be ordered, in general. We consider the
mapping s, t : N → R+ assigning s(j) = sj and t(j) := tj , respectively.
The mapping s is strictly increasing, such that we may consider the
composition ψ̄ := t ◦ s−1 : (sj ) 7→ (tj ). By linear interpolation this
extends to a continuous mapping ψ : (0, kAk] → R+ . We need to show
that ψ(u) → 0 as u → 0. But, since the sequence tj , j = 1, 2, . . . has
zero as only accumulation point, there is N ∈ N with tn ≤ ε for n ≥ N.
Let δ := sN > 0. Then for n ≥ N we find that ψ(sn ) = tn ≤ ε, and by
linear interpolation this extends to the whole interval [0, δ].

Of course, we cannot find that the above function ψ be an index function. For this to hold additional assumptions need to be made. Here
we consider the situation as it was assumed in [8], cf. Assumption 3.1
ibid.
Proposition 2. Suppose that with respect to the common eigenbasis ej , j = 1, 2, . . . the corresponding eigenvalues sj of the prior covariance, and tj of the operator Σ−1/2 K obey some asymptotic behavior, say in the power type case sj = j −(1+2a) and mj −p ≤ tj ≤ Mj −p
for j ∈ N and parameters a, p > 0. For the (index) function ψ(t) =
tp/(1+2a) , t > 0, we have that
m kψ(C0 )uk ≤ Σ−1/2 Ku ≤ M kψ(C0 )uk , u ∈ X.
P
Proof. Let u = ∞
j=1 uj ej ∈ X be any element. Then we find
(17)

2

2

2

m kψ(C0 )uk = m
≤

∞
X

∞
X

ψ

2

(sj )u2j

2

=m

j=1

t2j u2j = Σ−1/2 Ku

∞
X

j −2p u2j

j=1
2

.

j=1

The other inequality is proven similarly, and we omit the proof.



This shows that the present setup of a link condition extends previous studies restricted to commuting operators. The calculus with
using functional dependence instead of asymptotic behavior of singular
numbers seems simpler to handle. Therefore, in order to have a fair
comparison, if C0 and H were commuting we would instead assume

BAYESIAN INVERSE PROBLEMS

Jρ

G : XρG −−−→

 ∗
yS
J

Jϕ

XϕG −−−→

 ∗
yS
Jf

7

X

 ∗
yS

r
H :XrH −−−
→ XfH −−−→ X

Figure 1. The setup of interpolation. The position of
XϕG between XρG and X is given by the function t →
ϕ2 ((ρ2 )−1 (t)), and f is determined in such a way that
XfH has the appropriate position in the scale on bottom.
that Θ2 (C0 ) = H. This should be kept in mind when comparing the
subsequent bounds.
2.3. Variable Hilbert scales and their interpolation. To formulate the result on interpolation we recall the concept of variable Hilbert
scales. Given an injective positive self-adjoint operator G, and some
index function f we equip R(f (G)) with the norm kxkf = kwk, where
the
 element w is (uniquely) obtained from x = f (G)w. This makes
R(f (G)), k·kf a Hilbert space. Since this can be done for any index

function f we agree to denote the resulting spaces by XfG .
For our analysis we shall consider the scales generated by C0 , i.e.,
XϕC0 , and by H, hence the spaces XfH (We shall reserve Greek letters
for index functions related to the scale XϕC0 ).
We shall use interpolation of operators in variable Hilbert scales, and
we recall the fundamental result from [11].

Theorem 2 (Interpolation theorem, [11, Thm. 5]). Let G, H ≥ 0 be
self-adjoint operators with spectra in [0, b] and [0, a], respectively. Furthermore, let ϕ, ρ and r be index functions (ρ strictly increasing) on intervals [0, b] and [0, a], respectively, such that b ≥ kGk and ρ(b) ≥ r(a).
Then the function
f (t) := ϕ((ρ−1 )(r(t))),

0 < t ≤ a,

is well defined. The following assertion holds true: If t → ϕ2 ((ρ2 )−1 (t))
is operator concave on [0, ρ2 (b)] then
(18)
and
(19)
yield
(20)

kSxk ≤ C1 kxk ,

x ∈ X,

kρ(G)Sxk ≤ C2 kr(H)xk ,

x ∈ X,

kϕ(G)Sxk ≤ max {C1 , C2 } kf (H)xk ,

We depict the interpolation setup in Figure 1.

x ∈ X.

8

PETER MATHÉ

Remark 1. We mention the following important fact. The conditions
on the operator S in Theorem 2 correspond to the Figure 1 with S ∗ ,
the adjoint operator. We refer to [11, Cor. 2] for details.
It is important to notice that, in contrast to the commuting case no
link between the scales XϕG and XfH can be established whenever ϕ is
beyond ρ (ρ ≺ ϕ), or f is beyond r (r ≺ f ).
For the above interpolation it is crucial that the space XϕG is intermediate between XρG and X, i.e., we have continuous embeddings as
in Figure 1. The position is described as follows.
Definition 5 (position of an intermediate Hilbert space). The position
of XϕG between XρG and X is given by the function t → ϕ2 ((ρ2 )−1 (t)).
Remark 2. If we imagine that the spaces XρG and XϕG were Sobolev
Hilbert spaces with smoothness 0 < f ≤ r then the position would be
the quotient f /r ≤ 1. In the present context this corresponds to the
power type function t 7→ tf /r which is concave, even operator concave,
and we have seen in Theorem 2 that operator concavity is essential for
establishing results on operator interpolation.
3. Bounding the bias
We shall use the interpolation result with G := C0 and H from
above, and for various index functions and operators S. We recall the
description of the bias in the decomposition (6) given in (7) as
1/2

−1/2 ∗

bx∗ (α) = C0 sα (H)C0

x

.

To proceed we assign smoothness to the data generating element x∗
relative to the covariance operator C0 .
Assumption 2 (source set). There is an index function ϕ such that
x∗ ∈ Sϕ := {x,

x = ϕ(C0 )v, kvk ≤ 1} .

Using Proposition 1 we can bound for x∗ as above, its bias bx∗ (α) as
(21)
(22)

1
f0 (H)sα (H)ϕ0 (C0 )−1 ϕ(C0 )
m
1
=
sα (H)f0(H)ϕ0 (C0 )−1 ϕ(C0 ) .
m

bx∗ (α) ≤

Again, we emphasize that the intermediate operator f0 (H)ϕ0 (C0 )−1 is
norm bounded, such that the right hand side is finite.
In our subsequent analysis we shall distinguish three cases. These are
determined by the position of the given function ϕ within the Hilbert
scale given by C0 , and these cases are highlighted in Figure 2. We turn
to the detailed analysis of these cases.

BAYESIAN INVERSE PROBLEMS

9

regular: This case is obtained when ϕ0 (C0 )−1 ϕ(C0 ) is a bounded
operator, and hence when ϕ0 ≺ ϕ ≺ Θ.
low-order: When 1 ≺ ϕ ≺ ϕ0 we speak of the low-order case.
high-order: If ϕ is beyond the benchmark Θ (Θ ≺ ϕ) then we
call this the high-order case. As mentioned above, we need
additional assumptions to treat the latter.
Figure 2. Cases which are considered for interpolation
−1/2

3.1. Regular case: ϕ0 ≺ ϕ ≺ Θ. In this case the operator C0 ϕ(C0 ) =
ϕ0 (C0 )−1 ϕ(C0 ) is a bounded self-adjoint operator. The position of the
C0
Hilbert space XΘC0 ⊂ Xϕ/ϕ
⊂ X is then given through
0
 2 
−1 
ϕ
2
(23)
g (t) :=
Θ2
(t) , t > 0.
ϕ0
Proposition 3. Suppose that ϕ0 ≺ ϕ ≺ Θ, and that the function g 2 is
operator concave. Under Assumptions 1 & 2 we have that

M
sα (H)ϕ f02 (H) .
bx∗ (α) ≤
m
Proof. Under the assumptions made we apply Theorem 2 with S = I,
the identity operator, to see that
 
ϕ
(C0 )u ≤ M kg(H)uk , u ∈ X.
ϕ0
By Theorem 1 this means that for any u, kuk ≤ 1 we find that ū, kūk ≤
M with ϕϕ0 (C0 )u = g(H)ū. Hence we can bound
 
ϕ
sα (H)f0 (H)
(C0 )u = ksα (H)f0 (H)g(H)ūk
ϕ0
≤ M ksα (H)f0 (H)g(H)k
But, we check that

f0 (t)g(t) = f0 (t)

ϕ(f02 (t))
= ϕ(f02 (t),
f0 (t)

t > 0,

which gives
sα (H)f0 (H)



ϕ
ϕ0



(C0 )u ≤ M sα (H)ϕ f02 (H)



.

Since this holds for arbitrary u ∈ X, kuk ≤ 1 we conclude that
 

1
M
ϕ
bx∗ (α) ≤
sα (H)f0 (H)
(C0 )u ≤
sα (H)ϕ f02 (H) ,
m
ϕ0
m
and this completes the proof in the regular case.



10

PETER MATHÉ

3.2. Low-order case: 1 ≺ ϕ ≺ ϕ0 . For the application of Theorem 2 in this case we recall the definition of the operator S :=
f0 (H)ϕ0 (C0 )−1 : X → X.
Proposition 4. Suppose that the Assumptions 1 & 2 hold, and that
1 ≺ ϕ ≺ ϕ0 . Then the position of XϕC0 between XϕC00 and X is given by
the function ϕ2 . If ϕ2 is operator concave then

M
bx∗ (α) ≤
sα (H)ϕ f02 (H) .
m
Proof. We aim at applying Theorem 2 for the operator S ∗ . From Proposition 1 we know that kS ∗ : X → Xk ≤ M. But also we see that
kϕ0 (C0 )S ∗ uk = ϕ0 (C0 )ϕ0 (C0 )−1 f0 (H)u = kf0 (H)uk , u ∈ X.

The position of ϕ between XϕC00 and X is given by
 

2
2 −1
ϕ
ϕ0
(t) = ϕ2 (t), t > 0,

and this was assumed to be operator concave. Thus Theorem 2 applies
and yields kϕ(C0 )S ∗ uk ≤ M kg(H)uk , u ∈ X for the function g given
from


2
g(t) := ϕ ϕ−1
t > 0.
0 (f0 (t)) = ϕ f0 (t) ,
By virtue of Theorem 1 we find that for every x = Sϕ(C0 )w there is
w̄, kw̄k ≤ M with
Sϕ(C0 )w = g(H)w̄.
We conclude that therefore
1
M
bx∗ (α) ≤
ksα (H)Sϕ(C0 )k ≤
ksα (H)g(H)k
m
m

M
sα (H)ϕ f02 (H)
=
m
The proof is complete.

3.3. High-order case: Θ ≺ ϕ. If we want to extend the results to
smoothness beyond Θ then we need to assume a link condition at a
later position than H 1/2 . Therefore, we shall impose the following
lifting condition.
Assumption 3 (lifting condition). There are some r > 1 and constants
m ≤ 1 ≤ M such that
(24)

mr kΘr (C0 )uk ≤ H r/2 u = M r kΘr (C0 )uk ,

u ∈ X.

This is actually stronger than the original link condition from Assumption 1. Indeed, as the Loewner–Heinz Inequality asserts, for r > 1
the function t 7→ t1/r is operator monotone, see [3, Thm. V.1.9], or [7,
Prop. 8.21] we have
Corollary 1. Assumption 3 yields Assumption 1.

BAYESIAN INVERSE PROBLEMS

11

Remark 3. We stress that in case of commuting operators C0 and H
the assumptions 3 and 1 are equivalent, i.e., the link condition implies
the lifting condition. This can be seen using the Gelfand-Naimark
theorem, and we refer to [5, Prop. 8.1] for a similar assertion with
detailed proof.
Again we shall deal with the smoothness ϕ0 (C0 )−1 ϕ(C0 ), and, as in
the regular case, we ask for the position of the corresponding space between XΘCr0 and X. This gives the following result, extending the cases
ψ
of regular and low smoothness, however, under additional requirement
on the link.
Proposition 5. Suppose that the Assumptions 3 & 2 hold, and that
ϕ/ϕ0 ≺ Θr . Assume that for g from (23) the function t 7→ g 2 (t1/r ) is
operator concave. Then we have

M
bx∗ (α) ≤
sα (H)ϕ f02 (H) .
m
Proof. The proof is similar to the regular case. The function t 7→
C0
g 2 (t1/r ) is exactly the position of Xϕ/ϕ
between XΘCr0 and X is given as
0
 2 
−1 
ϕ
2
gr (t) :=
Θ2r
(t) .
ϕ0
−1

−1

It is readily checked that (Θ2r ) (t) = (Θ2 ) (t1/r ). Therefore, we find
that gr2 (t) = g 2 (t1/r ), and this is assumed to be operator concave, such
that we can use the Interpolation Theorem 2, and we find that
ϕ
(C0 )u ≤ M f (H r/2 )u , u ∈ X,
ϕ0
where the function f is given as


ϕ
ϕ
f (t) :=
(Θr )−1 (t) =
Θ−1 (t1/r ) , t > 0.
ϕ0
ϕ0

ϕ
This yields that f (H r/2 ) = ϕ0 Θ−1 (H 1/2 ) . Now, as in the regular
case, we arrive at
M
bx∗ (α) ≤
ksα (H)f0 (H)g(H)k .
m
We have seen there that f0 (t)g(t) = ϕ(f02 (t)), which completes the
proof.

Remark 4. The following comment seems interesting. In the highorder case, the function g 2 will in general not be operator concave.
However, the assumption which is made above, says that by re-scaling
this will eventually be operator concave if the scaling factor r is large
enough, see the discussion at the end of this section. Of course this
does not mean that the lifting Assumption 3 will hold automatically.
This is still a non-trivial assumption.

12

PETER MATHÉ

3.4. Saturation. In all the above cases, the low-order, regular and the
high-order one, we were able to derive a bias bound as in Proposition 5,
albeit under case specific assumptions. This bound cannot decay arbitrarily fast, and this is known as saturation in the regularization
theory, see again [2]. Indeed, the maximal decay rate, as α → 0 is linear, unless x∗ = 0, which is a result of the structure of the term sα (H).
This maximal decay rate is achieved when ϕ (f02 (t)) ≍ t, which after
rescaling means that ϕ(t) ≍ Θ2 (t). Thus the maximal smoothness for
which optimal decay of the bias can be achieved is given by the index
function Θ2 . This yields the following important remark, specific for
non-commuting operators.
Remark 5. Suppose that smoothness is given as in Assumption 2 with
an index function ϕ, and that we find the function Θ as in (11). Within
the range 0 ≺ ϕ ≺ Θ (low-order and regular cases) the link condition,
Assumption 1, suffices to yield optimal order decay of the bias. However, within the range Θ ≺ ϕ ≺ Θ2 the lifting, as given in Assumption 3
cannot be avoided. This effect cannot be seen for commuting operators C0 and H, because there the lifting is equivalent to the original
link condition, as discussed in Remark 3. We also observe that within
the present context, the lifting to r = 2 would be enough due to the
saturation at the function Θ2 .
We exemplify the above bounds for the bias for power type behavior, both of the smoothness in terms of ϕ(t) = tβ , and the linking
function ψ(t) = tκ . This results in a function Θ2 (t) = t1+2κ , which has
operator concave inverse. Thus, this requirement in Assumption 1 is
fulfilled whatever κ > 0 is found.
Then, the low order case 0 ≺ ϕ ≺ ϕ0 covers the range 0 < β ≤
1/2, and in this range the function ϕ2 (t) = t2β is operator concave,
because 2β ≤ 1.
The regular case ϕ0 ≺ ϕ ≺ Θ coves the exponents 1/2 ≤ β ≤ 1/2+κ.
The operator concavity was assumed to hold for the function
 2 
−1 
2(β−1/2)
ϕ
Θ2
(t) = t 1+2κ .
ϕ0
Thus, the regular case covers the range 1/2 ≤ β ≤ 1/2 + κ.
Finally, it is seen similarly, that the high order case covers the
range 1/2 ≤ β ≤ 1/2 + (r/2)(1 + 2κ), which for r = 2 already is
beyond the saturation point.
4. Bounding the posterior spread
We recall the structure of the posterior spread from (9) as
 


tr Cαδ = δ 2 tr (α + H)−1 C0 .

BAYESIAN INVERSE PROBLEMS

13

As can be seen, the noise level δ enters quadratically, and we aim at
finding the dependence upon the scaling parameter α. To this end the
following result proves useful.
Proposition 6. Under Assumption 1 we have that

 
δ2 
tr Cαδ ≤ 2 tr (α + H)−1 f02 (H) .
m
Proof. We start with the situation as given in (16). This order extends
by multiplying (α + H)−1/2 from both sides, such that we conclude
that
1
(α + H)−1/2 C0 (α + H)−1/2 ≤ 2 (α + H)−1/2 f02 (H) (α + H)−1/2 .
m
Now we apply the Weyl Monotonicity Theorem, see e.g. [3, Cor. III.2.3]
to see that this inequality applies to all singular numbers. But the
operators on both sides are self-adjoint and positive, such that singular
numbers and eigenvalues coincide. Thus we arrive at
h
i
 


tr Cαδ = δ 2 tr (α + H)−1 C0 = δ 2 tr (α + H)−1/2 C0 (α + H)−1/2
i
δ2 h
≤ 2 tr (α + H)−1/2 f02 (H) (α + H)−1/2
m

δ2 
= 2 tr (α + H)−1 f02 (H) ,
m
where we used the cyclic commutativity of the trace. The proof is
complete.

5. Bounding the squared posterior contraction
In the previous sections we derived bounds for both the bias and
the posterior spread. In all the smoothness cases from Section 3 we
arrived at a bound of the following form. If x∗ has smoothness with
index function ϕ, and if the link condition is with operator concave
function f02 from (13) then it was shown in Propositions 3– 5 that

M
sα (H)ϕ f02 (H) , α > 0.
(25)
bx∗ (α) ≤
m
Also, the posterior spread was bounded in Proposition 6 as

 
δ2 
tr Cαδ ≤ 2 tr (α + H)−1 f02 (H) , α > 0.
m
As was discussed in Remark 5 we shall confine to the case when ϕ ≺ Θ2 ,
i.e., before the saturation point. If this is the case then we can bound,
by using that the function sα obeys sα (t)t ≤ α, t, α > 0, the bias by

M
bx∗ (α) ≤
ϕ f02 (α) , α > 0.
m
A similar ’handy’ explicit bound for the posterior spread can hardly be
given. Under additional assumptions on the decay rate of the singular

14

PETER MATHÉ

numbers more explicit bounds can be given. We refer to [10, Sect. 4],
in particular Assumption 5 and Lemma 4.2 ibid. for details.
Overall we obtain the following result.
Theorem 3 (Bound for the SPC). Suppose that assumptions 1 & 2
hold for index functions ϕ and f0 . Under the assumptions of Propositions 3, 4 & 5, respectively, and if ϕ ≺ Θ2 , then
(26)




M2
SPC(α, δ) ≤ 2 inf ϕ2 f02 (α) + 2 tr (α + H)−1 f02 (H) , α, δ > 0.
m α>0
The above analysis is given in abstract terms of index functions, and
it is worthwhile to give an example to compare this with known (and
minimax) bounds for the commuting case.
To this end we treat the case for a moderately ill-posed operator C0 , a
power type link, and Sobolev type smoothness, with parameters a, p >
0 and β > 0 as in the original studies [8, 2].
Example 1 (power type decay).
(1) For some a > 0 we have that sj (C0 ) ≍ j −(1+2a) , j = 1, 2, . . . .
1+2a+2p
(2) There is some p > 0 such that Θ2 (t) ≍ t 1+2a as t → 0, and
2
P
(3) There is some R < ∞ such that j=∞ j 2β x∗j ≤ R2 , where
the sequence x∗j , j = 1, 2, . . . denotes the coefficients of x∗ with
respect to the eigenbasis of C0 .
This gives for the compound operator H that

sj (H) ≍ sj Θ2 (C0 ) = Θ2 (sj (C0 )) ≍ j −(1+2a+2p) ,

Notice furthermore that

sj f02 (H) = f02 (sj (H)) ≍ j −(1+2a) ,

j = 1, 2, . . .

j = 1, 2 . . .

Then we can bound, and we omit the standard calculations, the posterior spread by using Proposition 6 as
tr



Cαδ



∞
X
1+2p
sj (f02 (H)
≍ δ 2 α− 1+2a+2p .
≤δ
α + sj (H)
j=1
2

We turn to the description of the smoothness of x∗ in terms of an index
function ϕ, thus rewriting the condition from Assumption 1(3). This
β
yields that ϕ(t) = t 1+2a , see Section 4 from [2] for details. We see from
Assumption 1 (2) that saturation is at β = 1 + 2a + 2p.
Thus for 0 < β ≤ 1 + 2a + 2p we apply the bias bound from (25)
for obtaining a tight bound for the SPC. We balance the squared bias
1+2p
2β
with the bound for the posterior spread as α 1+2a+2p = δ 2 α− 1+2a+2p .

BAYESIAN INVERSE PROBLEMS

15

β

This gives α∗ = [δ 2 ] 1+2β+2p , and finally this results in rate for the
decay of the SPC as
  2β 
as δ → 0
SPC(α∗ (δ), δ) = O δ 2 1+2β+2p

if β ≤ 1 + 2a + 2p. Such bound is well known for commuting operators,
see [2, § 4.1], and the original study [8, Thm. 4.1].
Next we sketch the way to obtain bounds for the backwards heat
equation, with an exponentially ill-posed operator.
Example 2 (backwards heat equation, cf. [9]).
(1) For some a > 0 we have that sj (C0 ) ≍ j −(1+2a) , j = 1, 2, . . . .
2
1+a

(2) The linking function Θ2 obeys Θ2 (t) = e−2t .
(3) There is some β > 0 such that ϕ(t) = tβ/(1+2a) , t > 0.

First, the smoothness assumption is as in the previous example. We
observe that in this case always ϕ ≺ Θ, such that there is no saturation.
For bounding the bias we see that f02 (t) = 12 log−(1+2a) (1/t), t < 1.
For β/(1 + 2a) ≤ 1/2 the position will thus be
2β/(1+2a)

1
−(1+2a)
log
(1/t)
= 4β log−2β (1/t), as t → 0,
t −→
2
and this is operator concave for 0 < β ≤ 1/2. In the regular case, a
similar calculation reveals that the function
t −→ 4(β−1/2) log−2β+1 (1/t),
must be operator concave, which is true for 1/2 ≤ β ≤ 1.
So, in the range 0 < β ≤ 1 we find that


as α → 0.
bx∗ (α) = O log−β/2 (1/α)
By standard calculations we bound the posterior spread as
 
1
tr Cαδ ≤ Cδ 2 log−a (1/α),
α

for some constant C < ∞. Applying Theorem 3 we let α∗ (δ) :=
δ 2 logβ−a (1/α) and get the rate

SPC(α∗ (δ), δ) = O log−β (1/δ)
as δ → 0.

This corresponds to the contraction rate of the posterior as presented
in [9] with δ ∼ n−1/2 . For details we refer to Section 4 of the survey [2].
However, while these results cover all β > 0, the non-commuting case
will cover only the range 0 < β ≤ 1.

16

PETER MATHÉ

6. Conclusion
We summarize the above findings, and we start with the bias bounds.
In either of the three cases, if there is a valid link condition, if smoothness is given as in Assumption 2, and if the involved functions are
operator concave, then the norm in bx∗ (α) from (7) can be bounded by
M
sα (H)ϕ(f02(H)) .
m
This seems to be the natural extension for the bias bound to the noncommuting context. In the commuting context we would get f02 (H) =
C0 , see § 2.2.
Under these premises the analysis from [2] can be extended to the
non-commuting situation. We stressed in Remark 5 that a lifting of
the original link condition is necessary in order to yield optimal order
bounds for the squared posterior contraction up to the saturation point.
The analysis from [1] covers by different techniques the regular case.
In case of a power type function ψ, and hence of Θ, the requirements
of operator concavity reduce to power type functions with power in the
range between (0, 1), and hence these are automatically fulfilled.
For the posterior spread we derived a similar extension to the noncommuting case in Proposition 6. There is no handy way to derive
the exact increase of the spread as α → 0. Under additional assumption on the regularity of the decay for the singular numbers of H this
problem can be reduced
dimension of the operator H,
 to the effective

given as NH (α) = tr (α + H)−1 H . We did not pursue this line, here.
Instead we refer to the study [10].
Finally, we presented two examples exhibiting the obtained rates for
the SPC, both for moderately and severely ill-posed operators. More
examples, using functional dependence for commuting operators, are
given in the study [2].
bx∗ (α) ≤

References
1. Sergios Agapiou, Stig Larsson, and Andrew M. Stuart, Posterior contraction
rates for the Bayesian approach to linear ill-posed inverse problems, Stochastic
Process. Appl. 123 (2013), no. 10, 3828–3860. MR 3084161
2. Sergios Agapiou and Peter Mathé, Posterior contraction in Bayesian inverse
problems under Gaussian priors, New Trends in Parameter Identification for
Mathematical Models, Springer, 2018.
3. Rajendra Bhatia, Matrix analysis, Graduate Texts in Mathematics, vol. 169,
Springer-Verlag, New York, 1997. MR 1477662 (98i:15003)
4. G. Blanchard and P. Mathé, Discrepancy principle for statistical inverse problems with application to conjugate gradient iteration, Inverse Problems 28
(2012), no. 11, 115011, 23. MR 2992966
5. Albrecht Böttcher, Bernd Hofmann, Ulrich Tautenhahn, and Masahiro Yamamoto, Convergence rates for Tikhonov regularization from different kinds of
smoothness conditions, Appl. Anal. 85 (2006), no. 5, 555–578. MR 2213075
(2006k:65150)

BAYESIAN INVERSE PROBLEMS

17

6. R. G. Douglas, On majorization, factorization, and range inclusion of operators
on Hilbert space, Proc. Amer. Math. Soc. 17 (1966), 413–415. MR 0203464
7. Heinz W. Engl, Martin Hanke, and Andreas Neubauer, Regularization of inverse problems, Mathematics and its Applications, vol. 375, Kluwer Academic
Publishers Group, Dordrecht, 1996. MR 1408680 (97k:65145)
8. B. T. Knapik, A. W. van der Vaart, and J. H. van Zanten, Bayesian inverse
problems with Gaussian priors, Ann. Statist. 39 (2011), no. 5, 2626–2657.
MR 2906881
9.
, Bayesian recovery of the initial condition for the heat equation, Comm.
Statist. Theory Methods 42 (2013), no. 7, 1294–1313. MR 3031282
10. Kui Lin, Shuai Lu, and Peter Mathé, Oracle-type posterior contraction rates
in Bayesian inverse problems, Inverse Probl. Imaging 9 (2015), no. 3, 895–915.
MR 3406424
11. Peter Mathé and Ulrich Tautenhahn, Interpolation in variable Hilbert scales
with application to inverse problems, Inverse Problems 22 (2006), no. 6, 2271–
2297. MR 2277542 (2008c:47028)
12. F. Natterer, Error bounds for Tikhonov regularization in Hilbert scales, Applicable Anal. 18 (1984), no. 1-2, 29–37.
Weierstrass Institute, Mohrenstrasse 39, 10117 Berlin, Germany
E-mail address: peter.mathe@wias-berlin.de

