arXiv:1801.06862v3 [econ.EM] 30 Jan 2018

Testing the Number of Regimes in Markov Regime
Switching Models
Hiroyuki Kasahara∗
Vancouver School of Economics
University of British Columbia
hkasahar@mail.ubc.ca

Katsumi Shimotsu
Faculty of Economics
University of Tokyo
shimotsu@e.u-tokyo.ac.jp

January 28, 2018

Abstract
Markov regime switching models have been used in numerous empirical studies in economics
and finance. However, the asymptotic distribution of the likelihood ratio test statistic for testing
the number of regimes in Markov regime switching models has been an unresolved problem. This
paper derives the asymptotic distribution of the likelihood ratio test statistic for testing the null
hypothesis of M0 regimes against the alternative hypothesis of M0 + 1 regimes for any M0 ≥ 1
both under the null hypothesis and under local alternatives. We show that the contiguous
alternatives converge to the null hypothesis at a rate of n−1/8 in regime switching models with
normal density. The asymptotic validity of the parametric bootstrap is also established.

Key words: Differentiable in quadratic mean expansion; likelihood ratio test; Markov regime
switching model; parametric bootstrap.

1

Introduction

The Markov regime switching model has been a popular framework for empirical work in economics
and finance. Following the seminal contribution by Hamilton (1989), it has been used in numerous
empirical studies to model, for example, the business cycle (Hamilton, 2005; Morley and Piger,
2012), stock market volatility (Hamilton and Susmel, 1994), international equity markets (Ang and
Bekaert, 2002; Okimoto, 2008), monetary policy (Schorfheide, 2005; Sims and Zha, 2006; Bianchi,
2013), and economic growth (Kahn and Rich, 2007). Comprehensive theoretical accounts and
surveys of applications are provided by Hamilton (2008, 2016) and Ang and Timmermann (2012).
The number of regimes is an important parameter in applications of Markov regime switching
models. Despite its importance, however, testing for the number of regimes in Markov regime
∗

This research is supported by the Natural Science and Engineering Research Council of Canada, JSPS Grantin-Aid for Scientific Research (C) No. 17K03653, and the Institute of Statistical Mathematics Cooperative Use
Registration (2017 ISM CUR-171). The authors thank the seminar participants at Indiana University, LSE, and
Vanderbilt University for their helpful comments. The authors also thank Chiyoung Ahn for excellent research
assistance and Marine Carrasco, Liang Hu, and Werner Ploberger for making their code available.

1

switching models has been an unsolved problem because the standard asymptotic analysis of the
likelihood ratio test statistic (LRTS) breaks down because of problems such as unidentifiable parameters, the true parameter being on the boundary of the parameter space, and the degeneracy of
the Fisher information matrix. Testing the number of regimes for Markov regime switching models
with normal density, which are popular in empirical applications, poses a further difficulty because
normal density has the undesirable mathematical property that the second-order derivative with
respect to the mean parameter is linearly dependent on the first derivative with respect to the
variance parameter, leading to further singularity.
This paper proposes the likelihood ratio test of the null hypothesis of M0 regimes against the
alternative hypothesis of M0 + 1 regimes for any M0 ≥ 1 and derives its asymptotic distribution.
To the best of our knowledge, the asymptotic distribution of the LRTS has not been derived for
testing the null hypothesis of M0 regimes with M0 ≥ 2. To test the null hypothesis of no regime
switching, namely M0 = 1, Hansen (1992) derives an upper bound of the asymptotic distribution
of the LRTS, and Garcia (1998) also studies this problem. Carrasco et al. (2014) propose an
information matrix-type test for parameter constancy in general dynamic models including regime
switching models. Cho and White (2007) derive the asymptotic distribution of the quasi-LRTS for
testing the single regime against two regimes by rewriting the model as a two-component mixture
model, thereby ignoring the temporal dependence of the regimes.1 Qu and Zhuo (2017) extend
the analysis of Cho and White (2007) and derive the asymptotic distribution of the LRTS that
properly takes into account the temporal dependence of the regimes under some restrictions on the
transition probabilities of latent regimes. Marmer (2008) and Dufour and Luger (2017) develop
tests for the null hypothesis of no regime switching by using different approaches from the LRTS.
The studies discussed above focus on testing the single regime against two regimes. To the best of
our knowledge, however, the asymptotic distribution of the LRTS for testing the null hypothesis of
M0 regimes with M0 ≥ 2 remains unknown.
Several papers in the literature consider tests when some parameters are not identified under the
null hypothesis. These include Davies (1977, 1987), Andrews and Ploberger (1994, 1995), Hansen
(1996a), Andrews (2001), and Liu and Shao (2003), among others. Estimation and testing with a
degenerate Fisher information matrix are investigated in an iid setting by Chesher (1984), Lee and
Chesher (1986), Rotnitzky et al. (2000), and Gu et al. (2017), among others. Chen et al. (2014)
examine uniform inference on the mixing probability in mixture models.
To facilitate the analysis herein, we develop a version of Le Cam’s differentiable in quadratic
mean (DQM) expansion that expands the likelihood ratio under the loss of identifiability, while
adopting the reparameterization and higher-order expansion of Kasahara and Shimotsu (2015). In
an iid setting, Liu and Shao (2003) develop a DQM expansion under the loss of identifiability in
terms of the generalized score function. We extend Liu and Shao (2003) to accommodate dependent
and heterogeneous data as well as modify them to fit our context of parametric regime switching
1

Carter and Steigerwald (2012) show that ignoring temporal dependence may render the quasi-maximum likelihood
estimator inconsistent.

2

models. Using a DQM-type expansion has an advantage over the “classical” approach based on the
Taylor expansion up to the Hessian term because deriving a higher-order expansion becomes tedious
as the order of expansion increases in a Markov regime switching model. Furthermore, regime
switching models with normal components are not covered by Liu and Shao (2003) because their
Theorem 4.1 assumes that the generalized score function is obtained by expanding the likelihood
ratio twice, whereas our Section 6.2 shows that the score function is a function of the fourth
derivative of the likelihood ratio in the normal case.
Our approach follows Douc et al. (2004) [DMR hereafter], who derive the asymptotic distribution of the maximum likelihood estimator (MLE) of regime switching models. We express the
higher-order derivatives of the period density ratios in terms of the conditional expectation of the
derivatives of the period complete-data log-density, i.e., the log-density when the state variable
is observable, by applying the missing information principle (Woodbury, 1971; Louis, 1982) and
extending the analysis of DMR. We then show that these derivatives of the period density ratios
can be approximated by a stationary, ergodic, and square integrable martingale difference sequence
by conditioning on the infinite past, and this approximation is shown to satisfy the regularity
conditions for our DQM expansion.
We first derive the asymptotic null distribution of the LRTS for testing H0 : M = 1 against
HA : M = 2. When the regime-specific density is not normal, the log-likelihood function is locally
approximated by a quadratic function of the second-order polynomials of the reparameterized parameters. When the density is normal, the degree of deficiency of the Fisher information matrix and
required order of expansion depend on the value of the unidentified parameter; in particular, when
the latent regime variables are serially uncorrelated, the model reduces to a finite mixture normal
model in which the fourth-order DQM expansion is necessary to derive a quadratic approximation
of the log-likelihood function. We expand the log-likelihood with respect to the judiciously chosen
polynomials of the reparameterized parameters—which involves the fourth-order polynomials—to
obtain a uniform approximation of the log-likelihood function in quadratic form and derive the
asymptotic null distribution of the LRTS by maximizing the quadratic form under a set of cone
constraints building on the results of Andrews (1999, 2001).
To derive the asymptotic null distribution of the LRTS for testing H0 : M = M0 against
HA : M = M0 + 1 for M0 ≥ 2, we partition a set of parameters that describes the true null model
in the alternative model into M0 subsets, each of which corresponds to a specific way of generating
the null model. We show that the asymptotic distribution of the LRTS is characterized by the
maximum of the M0 random variables, each of which represents the LRTS for testing each of the
M0 subsets.
We also derive the asymptotic distribution of the LRTS under local alternatives. Carrasco et al.
(2014) show that the contiguous local alternatives of their tests are of order n−1/4 , where n is the
sample size. In a related problem of testing the number of components in finite mixture normal
regression models, Kasahara and Shimotsu (2015) show that the contiguous local alternatives are
of order n−1/8 (see also Chen and Li, 2009; Chen et al., 2012; Ho and Nguyen, 2016). We show
3

that the value of the unidentified parameter affects the convergence rate of the contiguous local
alternatives. When the regime-specific density is normal, some contiguous local alternatives are
of the order n−1/8 , and the LRT is shown to have non-trivial power against them. The tests of
Carrasco et al. (2014) do not have power against such alternatives, whereas the test of Qu and
Zhuo (2017) rules out such alternatives because of their restriction on the parameter space.
The asymptotic validity of the parametric bootstrap is also established both under the null
hypothesis and under local alternatives. The simulations show that our bootstrap LRT has good
finite sample properties. Our results also imply that the bootstrap LRT is valid for testing the
number of hidden states in hidden Markov models because this paper’s model includes the hidden
Markov model as a special case. Although several papers have analyzed the asymptotic property
of the MLE of the hidden Markov model,2 the asymptotic distribution of the LRTS for testing the
number of hidden states has been an open question.3
The remainder of this paper is organized as follows. After introducing the notation and assumptions in Section 2, we discuss the degeneracy of the Fisher information matrix and loss of
identifiability in regime switching models in Section 3. Section 4 establishes the DQM-type expansion. Section 5 presents the uniform convergence of the derivatives of the density ratios. Sections 6
and 7 derive the asymptotic null distribution of the LRTS. Section 8 derives the asymptotic distribution under local alternatives. Section 9 establishes the consistency of the parametric bootstrap.
Section 10 reports the results from the simulations and an empirical application, using U.S. GDP
per capita quarterly growth rate data. Section 11 collects the proofs and auxiliary results.

2

Notation and assumptions

Let := denote “equals by definition.” Let ⇒ denote the weak convergence of a sequence of stochastic
processes indexed by π for some space Π. For a matrix B, let λmin (B) and λmax (B) be the smallest
and largest eigenvalues of B, respectively. For a k-dimensional vector x = (x1 , . . . , xk )0 and a matrix
p
√
B, define |x| := x0 x and |B| := λmax (B 0 B). For a k × 1 vector a = (a1 , . . . , ak )0 and a function
f (a), let ∇a f (a) := (∂f (a)/∂a1 , . . . , ∂f (a)/∂ak )0 , and let ∇ja f (a) denote a collection of derivatives
of the form (∂ j /∂ai1 ∂ai2 . . . ∂aij )f (a). Let I{A} denote an indicator function that takes the value
1 when A is true and 0 otherwise. C denotes a generic non-negative finite constant whose value
may change from one expression to another. Let a ∨ b := max{a, b} and a ∧ b := min{a, b}. Let bxc
denote the largest integer less than or equal to x, and define (x)+ := max{x, 0}. Given a sequence
P
{fk }nk=1 , let νn (fk ) := n−1/2 nk=1 [fk − Eϑ∗ (fk )]. For a sequence Xnε indexed by n = 1, 2, . . .
and ε, we write Xnε = Opε (an ) if, for any δ > 0, there exist ε > 0 and M, n0 < ∞ such that
P(|Xnε /an | ≤ M ) ≥ 1 − δ for all n > n0 , and we write Xnε = opε (an ) if, for any δ1 , δ2 > 0,
2

See, for example, Leroux (1992), Francq and Roussignol (1998), Krishnamurthy and Rydén (1998), Bickel et al.
(1998), Jensen and Petersen (1999), Le Gland and Mevel (2000), and Douc and Matias (2001).
3
Gassiat and Keribin (2000) show that the LRTS for testing H0 : M = 1 against HA : M = 2 diverges when
state-specific densities have known and distinct parameter values. Dannemann and Holtzmann (2008) analyze the
modified quasi-LRTS for testing the null of two states against three.

4

there exist ε > 0 and n0 such that P(|Xnε /an | ≤ δ1 ) ≥ 1 − δ2 for all n > n0 . Loosely speaking,
Xnε = Opε (an ) and Xnε = opε (an ) mean that Xnε = Op (an ) and Xnε = op (an ) when ε is sufficiently
small, respectively. All limits are taken as n → ∞ unless stated otherwise. The proofs of all the
propositions and lemmas are presented in the appendix.
Consider the Markov regime switching process defined by a discrete-time stochastic process
{(Xk , Yk , Wk )}, where (Xk , Yk , Wk ) takes values in a set XM × Y × W with Y ⊂ Rqy and W ⊂ Rqw ,
and let B(XM × Y × W) denote the associated Borel σ-field. For a stochastic process {Zk } and
a < b, define Zba := (Za , Za+1 , . . . , Zb ). Denote Yk−1 := (Yk−1 , . . . , Yk−s ) for a fixed integer s and
b

Ya := (Ya , Ya+1 , . . . , Yb ). Here, Yk is an observable variable, Xk is an unobservable state variable,
Yk−1 is the lagged Yk ’s used as a covariate, and Wk is a weakly exogenous covariate. DMR’s model
does not include Wk .
Assumption 1. (a) {Xk }∞
k=0 is a first-order Markov chain with the state space XM :=
k−1

{1, 2, . . . , M }. (b) For each k ≥ 1, Xk is independent of (X0k−2 , Y0

, W0∞ ) given Xk−1 . (c) For

k−1
k−s−1
∞
, Xk−1
each k ≥ 1, Yk is conditionally independent of (Y−s+1
0 , W0 , Wk+1 ) given (Y k−1 , Xk , Wk ).

(d) W1∞ is conditionally independent of (Y0 , X0 ) given W0 .4 (e) {(Xk , Yk , Wk )}∞
k=0 is a strictly
stationary ergodic process.
When Wk is absent, DMR provide a sufficient condition for the ergodicity of (Xk , Yk ) in their
Assumption (A2). We assume the ergodicity of (Xk , Yk , Wk ) for brevity.
The unobservable Markov chain {Xk } is called the regime. The integer M represents the number
of regimes specified in the model. The parameter ϑM = (ϑ0M,y , ϑ0M,x )0 belongs to ΘM = ΘM,y ×ΘM,x ,
a compact subset of RqM . ϑM,x contains the parameter of the transition probability of Xk , which
we denote by qϑM,x (xk−1 , xk ) := P(Xk = xk |Xk−1 = xk−1 ). Let pij := qϑM,x (i, j) for i = 1, . . . , M
P −1
and j = 1, . . . , M − 1, and qϑM,x (i, M ) is determined by qϑM,x (i, M ) = 1 − M
j=1 pij . ϑM,y =
0 , γ 0 )0 contains the parameter of the conditional density of Y given (Y
(θ10 , . . . , θM
k
k−1 , Xk , Wk ),
P
which is given by gϑM,y (yk |yk−1 , xk , wk ) := j∈XM I{xk = j}f (yk |yk−1 , wk ; γ, θj ). Here, γ is the

structural parameter that does not vary across regimes, θj is the regime-specific parameter that
varies across regimes, and f (yk |yk−1 , wk ; γ, θj ) is the conditional density of yk given (yk−1 , wk )
when xk = j. Let
pϑ (yk , xk |yk−1 , xk−1 , wk ) := qϑx (xk−1 , xk )gϑy (yk |yk−1 , xk , wk )
X
I{xk = j}f (yk |yk−1 , wk ; γ, θj ).
= qϑx (xk−1 , xk )
j∈XM

We assume ΘM,y = Θθ × · · · × Θθ × Θγ , and the true parameter value is denoted by ϑ∗M .
We make the following assumptions that correspond to (A1)–(A3) in DMR.
Assumption 2. (a) 0 < σ− := inf ϑM,x ∈ΘM,x minx,x0 ∈XM qϑM,x (x, x0 ) and
σ+ := supϑM,x ∈ΘM,x maxx,x0 ∈XM qϑM,x (x, x0 ) < 1 for each M . (b) For all y 0 ∈ Y, y ∈ Y s , and
4

k−1

Assumptions 1(a)–(d) imply that Wk is conditionally independent of (Xk−1
, Y0
0

5

) given W0k−1 .

w ∈ W, 0 < inf ϑy ∈ΘM,y

P

x∈XM

gϑM,y (y 0 |y, x, w) and supϑM,y ∈ΘM,y

P

x∈XM

gϑM,y (y 0 |y, x, w) < ∞.
1

(c) b+ := supϑM,y ∈Θy supy0 ,y1 ,w,x gϑM,y (y1 |y0 , x, w) < ∞ and Eϑ∗ (| log b− (Y0 , W1 )|) < ∞, where
P
b− (y10 , w1 ) := inf ϑM,y ∈ΘM,y x∈XM gϑM,y (y1 |y0 , x, w1 ).
As discussed on p. 2260 of DMR, Assumption 2(a) implies that the Markov chain {Xk } has a
unique invariant distribution and is uniformly ergodic for all θM,x ∈ ΘM,x .5 For notational brevity,
we drop the subscript M from XM , ϑM , ΘM , etc., unless it is important to clarify the specific
∞
value of M . Assumptions 1(b) and (c) imply that {Zk }∞
k=0 := {(Xk , Y k )}k=0 is a Markov chain on
k−2
k−1
∞
Z := X × Y s given {Wk }∞
k=0 , and Zk is conditionally independent of (Z0 , W0 , Wk+1 ) given

(Zk−1 , Wk ). Consequently, Lemma 1, Corollary 1, and Lemma 9 of DMR go through even in the
∞
∞
presence of {Wk }∞
k=0 . Because {(Zk , Wk )}k=0 is stationary, we can and will extend {(Zk , Wk )}k=0 to

a stationary process {(Zk , Wk )}∞
k=−∞ with doubly infinite time. We denote the probability measure
6
and associated expectation of {(Zk , Wk )}∞
k=∞ under stationarity by Pϑ and Eϑ , respectively.

Under Assumptions 1(a)–(d), the density of Y1n given X0 = x0 , Y0 and W0n is given by
pϑM (Y1n |Y0 , W0n , x0 ) =

n
X Y

pϑM (Yk , xk |Yk−1 , xk−1 , Wk ).

(1)

n
xn
1 ∈XM k=1

Define the conditional log-likelihood function and stationary log-likelihood function as
`n (ϑ, x0 ) := log pϑ (Y1n |Y0 , W0n , x0 ) =

n
X

k−1

log pϑ (Yk |Y0

, W0k , x0 ),

k=1

`n (ϑ) := log pϑ (Y1n |Y0 , W0n ) =

n
X

k−1

log pϑ (Yk |Y0

, W0k ),

k=1
k−1

where we use the fact that pϑ (Yk |Y0
k−1
pϑ (Yk |Y0 , W0k ),

k−1

, W0n , x0 ) = pϑ (Yk |Y0

k−1

, W0k , x0 ) and pϑ (Yk |Y0

, W0n ) =

which follows from Assumption 1. Note that

k−1

k−1

pϑ (Yk |Y0 , W0k , x0 ) − pϑ (Yk |Y0 , W0k )


X
k−1
k−1
=
pϑ (Yk , xk |Yk−1 , xk−1 , Wk ) × Pϑ (xk−1 |Y0 , W0k−1 , x0 ) − Pϑ (xk−1 |Y0 , W0k−1 ) ,
(xk−1 ,xk )∈X 2
k−1

and Pϑ (xk−1 |Y0

, W0k−1 ) =

P

k−1

x0 ∈X

Pϑ (xk−1 |Y0

k−1

, W0k−1 , x0 )Pϑ (x0 |Y0

, W0k−1 ). Let ρ := 1 −

σ− /σ+ ∈ [0, 1). Lemma 11(a) in the appendix shows that, for all probability measures µ1 and µ2
5

Assumptions 1(c) and 2(a) are also employed in DMR. As discussed in Kasahara and Shimotsu (2017), these
assumptions together rule out models in which the conditional density Yk depends on both current and lagged regimes.
Kasahara and Shimotsu (2017) show the asymptotic normality of the MLE while relaxing Assumption 2(a) to allow
for inf ϑM,x ∈ΘM,x minx,x0 ∈XM qϑM,x (x, x0 ) = 0. It is possible to derive the asymptotic distribution of the LRT under
similar assumptions to Kasahara and Shimotsu (2017), albeit with a tedious derivation.
6
DMR use Pϑ and Eϑ to denote probability and expectation under stationarity on {Zk }∞
k=∞ , because their Section 7
deals with the case when Z0 is drawn from an arbitrary distribution. Because we assume {(Zk , Wk )}∞
k=∞ is stationary,
we use notations such as Pϑ and Eϑ without an overline for simplicity.

6

k−1
on B(X ) and all (yk−1
0 , w0 ),

sup

X

A

x0 ∈X

k−1
Pϑ (Xk−1 ∈ A|yk−1
0 , w0 , x0 )µ1 (x0 ) −

X

k−1
k−1
.
Pϑ (Xk−1 ∈ A|yk−1
0 , w0 , x0 )µ2 (x0 ) ≤ ρ

x0 ∈X

(2)
Consequently,

k−1
k−1
pϑ (Yk |Y0 , W0k−1 , x0 ) − pϑ (Yk |Y0 , W0k−1 )

goes to zero at an exponential rate as

k → ∞. Therefore, as shown in the following proposition, the difference between `n (θ, x0 ) and
`n (θ) is bounded by a deterministic constant, and the maximum of `n (ϑ, x0 ) and the maximum of
`n (ϑ) are asymptotically equivalent.
Proposition 1. Under Assumptions 1 and 2, for all x0 ∈ X ,
sup |`n (ϑ, x0 ) − `n (ϑ)| ≤ 1/(1 − ρ)2

Pθ∗ -a.s.

ϑ∈Θ
k−1

As discussed on p. 2263 of DMR, the stationary density pϑ (Yk |Y0

, W0k ) is not available in

closed form for some models with autoregression. For this reason, we consider the log-likelihood
function when the initial distribution of X0 follows some distribution
P
ξM ∈ ΞM := {ξ(x0 )x0 ∈XM : ξ(x0 ) ≥ 0 and x0 ∈XM ξ(x0 ) = 1}.
Define the MLE, ϑ̂M,ξM , by the maximizer of the log-likelihood:
`n (ϑM , ξM ) := log

M
X

!
pϑM (Y1n |Y0 , W0n , x0 )ξM (x0 )

,

(3)

x0 =1

where pϑM (Y1n |Y0 , W0n , x0 ) is given in (1). We define the number of regimes by the smallest number
M such that the data density admits the representation (3). Our objective is to test H0 : M = M0
against HA : M = M0 + 1.

3

Degeneracy of the Fisher information matrix and nonidentifiability under the null hypothesis

Consider testing H0 : M = 1 against HA : M = 2 in a two-regime model. The null hypothesis can
be written as H0 : θ1∗ = θ2∗ .7 When θ1 = θ2 , the parameter ϑ2,x is not identified because Yk has the
same distribution across regimes. Furthermore, Section 6 shows that when θ1 = θ2 , the scores with
respect to θ1 and θ2 are linearly dependent, so that the Fisher information matrix is degenerate.
The log-likelihood function of Markov regime switching models with normal density has further
degeneracy. In a two-regime model where Yk in the j-th regime follows N (µj , σj2 ), the model
reduces to a heteroscedastic normal mixture model when the Xk ’s are serially uncorrelated, i.e.,
7
The null hypothesis of H0 : M = 1 also holds when (p11 , p22 ) = (1, 0). We impose Assumption 2(a) to bound pjj
away from 0 and 1 because the log-likelihood function may become unbounded as p11 or p22 tends to 1 in view of
Gassiat and Keribin (2000).

7

p11 = 1 − p22 . Kasahara and Shimotsu (2015) show that in a heteroscedastic normal mixture
model, the first and second derivatives of the log-likelihood function are linearly dependent and the
score function is a function of the fourth-order derivative. Consequently, one needs to expand the
log-likelihood function four times to derive the score function.

4

Quadratic expansion under the loss of identifiability

When testing the number of regimes by the LRT, a part of ϑ is not identified under the null
hypothesis. Let π denote the part of ϑ that is not identified under the null, and split ϑ as ϑ =
(ψ 0 , π 0 )0 . For example, in testing H0 : M = 1 against HA : M = 2, we have ψ = ϑ2,y and π = ϑ2,x .
We denote the conditional log-likelihood function as `n (ψ, π, x0 ) := `n (ϑ, x0 ) and use pϑ and pψπ
interchangeably.
Denote the true parameter value of ψ by ψ ∗ , and denote the set of (ψ, π) corresponding to the
null hypothesis by Γ∗ = {(ψ, π) ∈ Θ : ψ = ψ ∗ }. Let tϑ be a continuous function of ϑ such that
tϑ = 0 if and only if ψ = ψ ∗ . For ε > 0, define the neighborhood of Γ∗ by
Nε := {ϑ ∈ Θ : |tϑ | < ε}.
When the MLE is consistent, the asymptotic distribution of the LRTS is determined by the local
properties of the likelihood functions in Nε .
We establish a general quadratic expansion of the log-likelihood function `n (ψ, π, ξ) := `n (ϑ, ξ)
defined in (3) around `n (ψ ∗ , π, ξ) that expresses `n (ψ, π, ξ) − `n (ψ ∗ , π, ξ) as a quadratic function
of tϑ . Once we derive a quadratic expansion, the asymptotic distribution of the LRTS can be
characterized by taking its supremum with respect to tϑ under an appropriate constraint and using
the results of Andrews (1999, 2001).
Denote the conditional density ratio by
k−1

lϑkx0 :=
so that `n (ψ, π, x0 ) − `n (ψ ∗ , π, x0 ) =

, W0k , x0 )
,
k−1
pψ∗ π (Yk |Y0 , W0k , x0 )
pψπ (Yk |Y0

(4)

Pn

k=1 log lϑkx0 .

We assume that lϑkx0 can be expanded around
P
lϑ∗ kx0 = 1 as follows. With a slight abuse of the notation, let Pn (fk ) := n−1 nk=1 fk and recall
P
νn (fk ) := n−1/2 nk=1 [fk − Eϑ∗ (fk )].
Assumption 3. For all k = 1, . . . , n, lϑkx0 − 1 admits an expansion
lϑkx0 − 1 = t0ϑ sπk + rϑk + uϑkx0 ,

(5)

where tϑ satisfies ψ → ψ ∗ if tϑ → 0 and (sπk , rϑk , uϑkx0 ) satisfy, for some C ∈ (0, ∞), δ > 0,
ε > 0, and ρ ∈ (0, 1), (a) Eϑ∗ supπ∈Θπ |sπk |2+δ < C, (b) supπ∈Θπ |Pn (sπk s0πk ) − Iπ | = op (1) with
0 < inf π∈Θπ λmin (Iπ ) ≤ supπ∈Θπ λmax (Iπ ) < ∞, (c) Eϑ∗ [supϑ∈Nε |rϑk /(|tϑ ||ψ − ψ ∗ |)|2 ] < ∞, (d)
8

supϑ∈Nε [νn (rϑk )/(|tϑ ||ψ − ψ ∗ |)] = Op (1), (e) supx0 ∈X supϑ∈Nε Pn (|uϑkx0 |/|ψ − ψ ∗ |)j = Op (n−1 ) for
j = 1, 2, 3, (f ) supx0 ∈X supϑ∈Nε Pn (|sπk ||uϑkx0 |/|ψ−ψ ∗ |) = Op (n−1 ), (g) supϑ∈Nε |νn (sπk )| = Op (1).
We first establish an expansion of `n (ψ, π, x0 ) in the neighborhood of Nc/√n for any c > 0.
Proposition 2. Suppose that Assumptions 3 (a)–(f ) hold. Then, for any c > 0,
sup

sup

x0 ∈X

ϑ∈Nc/√n

`n (ψ, π, x0 ) − `n (ψ ∗ , π, x0 ) −

√

nt0ϑ νn (sπk ) + nt0ϑ Iπ tϑ /2 = op (1).

The following proposition expands `n (ψ, π, x0 ) in Anε (x0 , η) := {ϑ ∈ Nε : `n (ψ, π, x0 ) −
`n (ψ ∗ , π, x0 ) ≥ −η} for some η ∈ [0, ∞). This proposition is useful for deriving the asymptotic distribution of the LRTS because a consistent MLE is in Anε (x0 , η) by definition.
Anεc (x0 , η) := Anε (x0 , η) ∪
Proposition

3. Suppose that Assumption 3 holds.

supx0 ∈X supϑ∈Anε (x0 ,η) |tϑ | = Opε
sup

sup

Let

Nc/√n .
(n−1/2 ),

Then,

for any η

>

0,

(a)

and (b) for any c > 0,

`n (ψ, π, x0 ) − `n (ψ ∗ , π, x0 ) −

x0 ∈X ϑ∈Anεc (x0 ,η)

√ 0
ntϑ νn (sπk ) + nt0ϑ Iπ tϑ /2 = opε (1).

The following corollary of Propositions 2 and 3 shows that `n (ϑ, ξ) defined in (3) admits a
similar expansion to `n (ϑ, x0 ) for all ξ. Consequently, the asymptotic distribution of the LRTS
does not depend on ξ, and `n (ϑ, ξ) may be maximized in ϑ while fixing ξ or jointly in ϑ and ξ. Let
Anε (ξ, η) := {ϑ ∈ Nε : `n (ψ, π, ξ) − `n (ψ ∗ , π, ξ) ≥ −η} and Anεc (ξ, η) := Anε (ξ, η) ∪ Nc/√n , which
includes a consistent MLE with any ξ.
Corollary 1. (a) Under the assumptions of Proposition 2, we have
√
supξ∈Ξ supϑ∈Nc/√n |`n (ψ, π, ξ) − `n (ψ ∗ , π, ξ) − nt0ϑ νn (sπk ) + nt0ϑ Iπ tϑ /2| = op (1) for any c > 0.
(b) Under the assumptions of Proposition 3, for any η > 0 and c > 0, supξ∈Ξ supϑ∈Anε (ξ,η) |tϑ | =
√
Opε (n−1/2 ) and supξ∈Ξ supϑ∈Anεc (ξ,η) |`n (ψ, π, ξ) − `n (ψ ∗ , π, ξ) − nt0ϑ νn (sπk ) + nt0ϑ Iπ tϑ /2| = opε (1).

5

Uniform convergence of the derivatives of the log-density and
density ratios

In this section, we establish approximations that enable us to apply the results in Section 4 to
the log-likelihood function of regime switching models. Because of the presence of singularity,
the expansion (5) of the density ratio lϑkx0 involves the higher-order derivatives of the density
ratios ∇jψ lϑkx0 with j ≥ 2. Note that ∇jψ lϑkx0 can be expressed in terms of the derivatives of logk−1

densities, ∇jψ log pψπ (Yk |Y0

, W0k , x0 ). We show that these derivatives are approximated by their
k−1

k−1

k ) in place of (Y
stationary counterpart that condition on the infinite past (Y−∞ , W−∞
0

, W0k ).

Consequently, the sequence {∇jψ lϑkx0 }∞
k=0 is approximated by a stationary martingale difference
sequence.
9

For 1 ≤ k ≤ n and m ≥ 0, let
k
k
pϑ (Y−m+1
|Y−m , W−m
)

X

:=

k
Y

k
pϑ (Yt , xt |Yt−1 , Wt , xt−1 )Pϑ∗ (x−m |Y−m , W−m
),

xk−m ∈X k+m+1 t=−m+1

(6)
k
Y−m+1

k }, where
denote the stationary density of
associated with ϑ conditional on {Y−m , W−m
k−1
k ).
X−m is drawn from its true conditional stationary distribution Pϑ∗ (x−m |Y−m , W−m
Let
k−1
k−1
k−1
k
k
k
pϑ (Yk |Y−m , W−m ) := pϑ (Y−m+1 |Y−m , W−m )/pϑ (Y−m+1 |Y−m , W−m ) denote the associated
k−1
k ).8
conditional density of Yk given (Y−m , W−m
k−1
k−1
k ,X
k
Define the density ratio as lk,m,x (ϑ) := pϑ (Yk |Y−m , W−m
−m = x)/pϑ∗ (Yk |Y −m , W−m , X−m =

x). For j = 1, 2, . . . , 6, 1 ≤ k ≤ n, m ≥ 0, and x ∈ X , define the derivatives of the log-densities and
density ratios by, with suppressing the subscript ϑ from ∇jϑ for brevity,
k−1

j

∇ `k,m,x (ϑ) := ∇

j

k−1
k
log pϑ (Yk |Y−m , W−m
, X−m

= x),

j

∇ lk,m,x (ϑ) :=

k ,X
∇j pϑ (Yk |Y−m , W−m
−m = x)
k−1

k ,X
pϑ∗ (Yk |Y−m , W−m
−m = x)
k−1

k−1

k
),
∇j `k,m (ϑ) := ∇j log pϑ (Yk |Y−m , W−m

and ∇j lk,m (ϑ) :=

k )
∇j pϑ (Yk |Y−m , W−m
k−1

k )
pϑ∗ (Yk |Y−m , W−m

.

The following assumption corresponds to (A6)–(A8) in DMR and is tailored to our setting where
some elements of ϑ∗x are not identified and X is finite. Note that Assumptions (A6) and (A7) in
DMR pertaining to qϑx (x, x0 ) hold in our case because the pij ’s are bounded away from 0 and 1.
P
Let Gϑk := xk ∈X gϑy (Yk |Yk−1 , xk , Wk ). Gϑk satisfies Assumption 4(b) in general when N ∗ is
sufficiently small.
Assumption 4. There exists a positive real δ such that on N ∗ := {ϑ ∈ Θ : |ϑy − ϑ∗y | < δ} the
following conditions hold: (a) For all (y, y 0 , x, w) ∈ Y s × Y × X × W, gϑy (y 0 |y, x, w) is six times
continuously differentiable on N ∗ . (b) Eϑ∗ [supϑ∈N ∗ supx∈X |∇j log gϑy (Y1 |Y0 , x, W )|2qj ] < ∞ for
j = 1, 2, . . . , 6 and Eϑ∗ supϑ∈N ∗ |Gϑk /Gϑ∗ k |qg < ∞ with q1 = 6q0 , q2 = 5q0 , . . . , q6 = q0 , where
q0 = (1 + ε)qϑ and qg = (1 + ε)qϑ /ε for some ε > 0 and qϑ > max{3, dim(ϑ)}. (c) For almost all
(y, y 0 , w) ∈ Y s ×Y ×W, supϑ∈N ∗ gϑy (y 0 |y, x, w) < ∞ and, for almost all (y, x, w) ∈ Y s ×X ×W, for
j
j
j = 1, 2, . . . , 6, there exist functions fy,w,x
: Y → R+ in L1 such that |∇j gϑy (y 0 |y, x, w)| ≤ fy,x,w
(y 0 )

for all ϑ ∈ N ∗ .
The following proposition shows that {∇j `k,m,x (ϑ)}m≥0 and {∇j `k,m (ϑ)}m≥0 are Lrj (Pϑ∗ )Cauchy sequences that converge to ∇j `k,∞ (ϑ) Pϑ∗ -a.s. and in Lrj (Pϑ∗ ) uniformly in ϑ ∈ N ∗ and
x ∈ X.
Proposition 4. Under Assumptions 1, 2, and 4, for j = 1, . . . , 6, there exist random variables
8

k−1

Note that DMR use the same notation pϑ (·|Y−m ) for a different purpose. On p. 2263 and in some other (but
k−1
not all) places, DMR use pϑ (Yk |Y0 ) to denote an (ordinary) stationary conditional distribution of Yk .

10

,

(Kj , {Mj,k }nk=1 ) ∈ Lrj (Pϑ∗ ) and ρ∗ ∈ (0, 1) such that, for all 1 ≤ k ≤ n and m0 ≥ m ≥ 0,
(a)

sup sup |∇j `k,m,x (ϑ) − ∇j `k,m (ϑ)| ≤ Kj (k + m)7 ρk+m−1
∗

Pϑ∗ -a.s.,

x∈X ϑ∈N ∗

(b)

sup sup |∇j `k,m,x (ϑ) − ∇j `k,m0 ,x (ϑ)| ≤ Kj (k + m)7 ρk+m−1
∗

x∈X ϑ∈N ∗

(c)

sup sup sup |∇j `k,m,x (ϑ)| + sup sup |∇j `k,m (ϑ)| ≤ Mj,k

m≥0 x∈X ϑ∈N ∗

Pϑ∗ -a.s.,
Pϑ∗ -a.s.,

m≥0 ϑ∈N ∗

where r1 = 6q0 , r2 = 3q0 , r3 = 2q0 , r4 = 3q0 /2, r5 = 6q0 /5, and r6 = q0 . (d) Uniformly in ϑ ∈ N ∗
and x ∈ X , ∇j `k,m,x (ϑ) and ∇j `k,m (ϑ) converge Pϑ∗ -a.s. and in Lrj (Pϑ∗ ) to ∇j `k,∞ (ϑ) ∈ Lrj (Pϑ∗ )
as m → ∞.
As shown by the following proposition, we may prove the uniform convergence of the derivatives
of the density ratios by expressing them as polynomials of the derivatives of the log-density and
applying Proposition 4 and Hölder’s inequality.
Proposition 5. Under Assumptions 1, 2, and 4, for j = 1, . . . , 6, there exist random variables
{Kj,k }nk=1 ∈ Lqϑ (Pϑ∗ ) and ρ∗ ∈ (0, 1) such that, for all 1 ≤ k ≤ n and m0 ≥ m ≥ 0,
(a)

sup sup |∇j lk,m,x (ϑ) − ∇j lk,m (ϑ)| ≤ Kj,k (k + m)7 ρk+m−1
∗

Pϑ∗ -a.s.,

x∈X ϑ∈N ∗

(b)

sup sup |∇j lk,m,x (ϑ) − ∇j lk,m0 ,x (ϑ)| ≤ Kj,k (k + m)7 ρk+m−1
∗

x∈X ϑ∈N ∗

(c)

sup sup sup |∇j lk,m,x (ϑ)| + sup sup |∇j lk,m (ϑ)| ≤ Kj,k

m≥0 x∈X ϑ∈N ∗

m≥0 ϑ∈N ∗

Pϑ∗ -a.s.,

Pϑ∗ -a.s.

(d) Uniformly in ϑ ∈ N ∗ and x ∈ X , ∇j lk,m,x (ϑ) and ∇j lk,m (ϑ) converge Pϑ∗ -a.s. and in Lqϑ (Pϑ∗ )
Pϑ∗ -a.s.
to ∇j lk,∞ (ϑ) ∈ Lqϑ (Pϑ∗ ) as m → ∞. (e) supϑ∈N ∗ |∇j lk,0 (ϑ) − ∇j lk,∞ (ϑ)| ≤ Kj,k k 7 ρk−1
∗
When we apply the results in Section 4 to regime switching models, lk,0,x (ϑ) corresponds to lϑkx0
on the left-hand side of (5), and sπk in (5) is a function of the ∇j lk,0 (ϑ)’s. Proposition 5 and the
dominated convergence theorem for conditional expectations (Durrett, 2010, Theorem 5.5.9) imply
k−1

that Eϑ∗ [∇j lk,∞ (ϑ)|Y−∞ ] = 0 for all ϑ ∈ N ∗ . Therefore, {∇j lk,∞ (ϑ)}∞
k=−∞ is a stationary, ergodic,
and square integrable martingale difference sequence, and {∇j lk,∞ (ϑ)}5j=1 satisfies Assumption
3(a)(b)(g).

6

Testing homogeneity

Before developing the LRT of M0 components, we analyze a simpler case of testing the null hypothesis H0 : M = 1 against HA : M = 2 when the data are from H0 . We assume that the
parameter space for ϑ2,x = (p11 , p22 )0 takes the form [, 1 − ]2 for a small  ∈ (0, 1/2). Denote the
true parameter in the one-regime model by ϑ∗1 := ((θ∗ )0 , (γ ∗ )0 )0 . The two-regime model gives rise
to the true density pϑ∗1 (Y1n |Y0 , x0 ) if the parameter ϑ2 = (θ1 , θ2 , γ, p11 , p22 )0 lies in a subset of the
11

parameter space
Γ∗ := {(θ1 , θ2 , γ, p11 , p22 ) ∈ Θ2 : θ1 = θ2 = θ∗ and γ = γ ∗ } .
Note that (p11 , p22 ) is not
identified under H0 .
P

2
n |Y , Wn , x )ξ (x ) denote the two-regime log-likelihood
p
(Y
Let `n (ϑ2 , ξ2 ) := log
0
0 2 0
1
0
x0 =1 ϑ2
for a given initial distribution ξ2 (x0 ) ∈ Ξ2 , and let ϑ̂2 := arg maxϑ2 ∈Θ2 `n (ϑ2 , ξ2 ) denote the MLE
of ϑ2 given ξ2 , where ξ2 is suppressed from ϑ̂2 because ξ2 does not matter asymptotically. Let
ϑ̂1 denote the one-regime MLE that maximizes the one-regime log-likelihood function `0,n (ϑ1 ) :=
Pn
0 0 0
k=1 log f (Yk |Y k−1 , Wk ; γ, θ) under the constraint ϑ1 = (θ , γ ) ∈ Θ1 .
We introduce the following assumption for the consistency of ϑ̂1 and ϑ̂2 . Assumption 5(b)
corresponds to Assumption (A4) of DMR. Assumption 5(c) is a standard identification condition
for the one-regime model. Assumption 5(d) implies that the Kullback–Leibler divergence between
0

0

0 ) and p (Y |Y
0
∗
pϑ∗1 (Y1 |Y−m , W−m
ϑ2 1
−m , W−m ) is 0 if and only if ϑ2 ∈ Γ .

Assumption 5. (a) Θ1 and Θ2 are compact, and ϑ∗1 is in the interior of Θ1 .
(x, x0 )
ϑ∗1 ,

∈ X and all

(y, y 0 , w)

∈

Ys

× Y × W,

f (y 0 |y

then Pϑ∗1 f (Y1 |Y0 , W1 ; γ, θ) 6= f (Y1 |Y0 , W1

0 , w; γ, θ)

∗
; γ , θ∗ ) > 0.

0

(b) For all

is continuous in (γ, θ). (c) If ϑ1 6=
0

1 )] =
(d) Eϑ∗1 [log pϑ2 (Y1 |Y−m , W−m

1 )] for all m ≥ 0 if and only if ϑ ∈ Γ∗ .
Eϑ∗1 [log pϑ∗1 (Y1 |Y−m , W−m
2

The following proposition shows the consistency of the MLEs of ϑ∗1 and ϑ∗2,y .
Proposition 6. Suppose that Assumptions 1, 2, and 5 hold. Then, under the null hypothesis of
p

p

M = 1, ϑ̂1 → ϑ∗1 and inf ϑ2 ∈Γ∗ |ϑ̂2 − ϑ2 | → 0.
Let LRn := 2[`n (ϑ̂2 , ξ2 ) − `0,n (ϑ̂1 )] denote the LRTS for testing H0 : M0 = 1 against HA : M0 =
2. Following the notation of Section 4, we split ϑ2 into ϑ2 = (ψ, π), where π is the part of ϑ not
identified under the null hypothesis; the elements of ψ are delineated later. In the current setting, π
corresponds to ϑ2,x = (p11 , p22 )0 . Define % := corrϑ2,x (Xk , Xk+1 ) = p11 +p22 −1 and α := Pϑ2,x (Xk =
1) = (1 − p22 )/(2 − p11 − p22 ). The parameter spaces for % and α under restriction p11 , p22 ∈ [, 1 − ]
are given by Θ% := [−1 + 2, 1 − 2] and Θα := [, 1 − ], respectively. Because the mapping from
(p11 , p22 ) to (%, α) is one-to-one, we reparameterize π as π := (%, α)0 ∈ Θπ := Θ% × Θα , and let
pψπ (·|·) := pϑ2 (·|·). Henceforth, we suppress W0n for notational brevity and write, for example,
pψπ (Y1n |Y0 , W0n , x0 ) as pψπ (Y1n |Y0 , x0 ) and pψπ (yk , xk |yk−1 , xk−1 , wk ) as pψπ (yk , xk |yk−1 , xk−1 )
when doing so does not cause confusion.
We derive the asymptotic distribution of the LRTS by applying Corollary 1 to `n (ψ, π, ξ2 ) and
representing sπk and tϑ in (5) in terms of ϑ, f (Yk |Yk−1 ; γ, θ), and its derivatives; sπk involves
higher-order derivatives, and tϑ consists of the functions of the polynomials of (reparameterized)
ϑ. Section 6.1 analyzes the case when the regime-specific distribution of yk is not normal with
unknown variance. Section 6.2 analyzes the case when the regime-specific distribution yk is normal
with regime-specific and unknown variance. Section 6.3 handles the normal distribution where
12

∞

the variance is unknown and common across regimes. Note that because Y−∞ and X∞
−∞ are
independent when ψ = ψ ∗ , we have
∞

∞
Pψ∗ π (X∞
−∞ |Y −∞ ) = Pψ ∗ π (X−∞ ).

(7)

Define qk := I{Xk = 1}, so that α = Eψ∗ π [qk ].

6.1

Non-normal distribution

When we apply Corollary 1 to regime switching models, sπk is a function of
k−1

∇j pψ∗ π (Yk |Y0
∇j p

k−1

)/pψ∗ π (Yk |Y0

)’s with pψπ (Y1k |Y0 ) defined in (6).

k−1
k−1
ψ ∗ π (Yk |Y 0 )/pψ ∗ π (Yk |Y 0 )

in terms of

∇j f (y|x; γ, θ)

In order to express

via the Louis information princi-

ple (Lemma 1 in the appendix), we first derive the derivatives of the complete data conP2
ditional density pϑ2 (yk , xk |yk−1 , xk−1 ) = gϑ2,y (yk |yk−1 , xk )qϑ2,x (xk−1 , xk ) =
j=1 I{xk =
j}f (yk |yk−1 ; γ, θj )qϑ2,x (xk−1 , xk ).
Consider the following reparameterization. Let
λ
ν

!
:=

θ1 − θ2

!

αθ1 + (1 − α)θ2

,

so that

θ1
θ2

!
=

ν + (1 − α)λ
ν − αλ

!
.

(8)

Let η := (γ 0 , ν 0 )0 and ψα := (η 0 , λ0 )0 ∈ Θη × Θλ . Under the null hypothesis of one regime, the true
value of ψα is given by ψα∗ := (γ ∗ , θ∗ , 0)0 . Henceforth, we suppress the subscript α from ψα . Using
this definition of ψ, let ϑ2 := (ψ 0 , π 0 )0 ∈ Θψ × Θπ . By using reparameterization (8) and noting that
qk = I{xk = 1}, we have pψπ (yk , xk |yk−1 , xk−1 ) = gψ (yk |yk−1 , xk )qπ (xk−1 , xk ) and
gψ (yk |yk−1 , xk ) = f (yk |yk−1 ; γ, ν + (qk − α)λ).
Henceforth,

let fk∗ ,

∇fk∗ ,

gk∗ ,

and ∇gk∗ denote f (Yk |Yk−1 ; γ ∗ , θ∗ ),

(9)
∇f (Yk |Yk−1 ; γ ∗ , θ∗ ),

gψ∗ (Yk |Yk−1 , Xk ), and ∇gψ∗ (Yk |Yk−1 , Xk ), respectively, and similarly for log fk∗ ,∇ log fk∗ , log gk∗ ,
and ∇ log gk∗ . Expanding gψ (Yk |Yk−1 , Xk ) twice with respect to ψ = (γ 0 , ν 0 , λ0 )0 and evaluating at
ψ ∗ gives
∇η gk∗ = ∇(γ 0 ,θ0 )0 fk∗ ,

∇λ gk∗ = (qk − α)∇θ fk∗ ,

∇λη0 gk∗ = (qk − α)∇θ(γ 0 ,θ0 ) fk∗ ,

∇λλ0 gk∗ = (qk − α)2 ∇θθ0 fk∗ .

(10)

Recall % := corrϑ∗2 (qk , qk+1 ). Observe that qk satisfies
Eϑ∗2 (qk − α)2 = α(1 − α),

Eϑ∗2 (qk − α)3 = α(1 − α)(1 − 2α),

Eϑ∗2 (qk − α)4 = α(1 − α)(3α2 − 3α + 1),

corrϑ∗2 (qk , qk+` ) = %|`| ,

(11)

where the first three results follow from the property of a Bernoulli random variable, and the last

13

result follows from Hamilton (1994, p. 684). Then, it follows from (7) and (11) that
n

Eϑ∗ [qk − α|Y−∞ ] = 0,

n

Eϑ∗ [(qt1 − α)(qt2 − α)|Y−∞ ] = α(1 − α)%t2 −t1 ,

t2 ≥ t1 .

(12)

From Lemma 1, log pψπ (yk , xk |yk−1 , xk−1 ) = log gψ (yk |yk−1 , xk ) + log qπ (xk−1 , xk ), and the definition of pψπ (Y1k |Y0 ) in (6), we obtain
k−1

∇ψ pψ∗ π (Yk |Y0

k−1

pψ∗ π (Yk |Y0

)

)

k−1

= ∇ψ log pψ∗ π (Yk |Y0

)=

k
X

k−1
i X
i
h
h
k
k−1
Eϑ∗ ∇ψ log gt∗ Y0 −
Eϑ∗ ∇ψ log gt∗ Y0
.

t=1

t=1

Applying (10), (12), and gk∗ = fk∗ to the right-hand side gives
k−1

∇η pψ∗ π (Yk |Y0

)

k−1
pψ∗ π (Yk |Y0 )

k−1

=

∇(γ 0 ,θ0 )0 log fk∗ ,

∇λ pψ∗ π (Yk |Y0

)

k−1
pψ∗ π (Yk |Y0 )

= 0.

(13)

Similarly, it follows from Lemma 1, (10), (12), (13), and gk∗ = fk∗ that
k−1

∇λη0 pψ∗ π (Yk |Y0

k−1

)/pψ∗ π (Yk |Y0

) = 0,

(14)

k−1
k−1
∇λλ0 pψ∗ π (Yk |Y0 )/pψ∗ π (Yk |Y0 )
k−1
= ∇λλ0 log pψ∗ π (Yk |Y0 )

=

k
X

E

ϑ∗

k−1
i X
h
i
h
k
k−1
∗
0
Eϑ∗ ∇λλ0 log gt∗ Y0
∇λλ log gt Y0 −
t=1

t=1
k
X

k
X

∇λ gt∗1
gt∗1

∇λ0 gt∗2
gt∗2

k−1 X
k−1
X

∇λ gt∗1 ∇λ0 gt∗2 k−1
+
Eϑ∗
−
Eϑ∗
Y0
gt∗1
gt∗2
t1 =1 t2 =1
t1 =1 t2 =1
"

#
k−1
∇θθ0 fk∗ X k−t ∇θ ft∗ ∇θ0 fk∗ ∇θ fk∗ ∇θ0 ft∗
+
%
+
.
= α(1 − α)
fk∗
ft∗
fk∗
fk∗
ft∗


k
Y0







(15)

t=1

Because the first-order derivative with respect to λ is identically equal to zero in (13), the unique
k−1

elements of ∇η pψ∗ π (Yk |Y0

k−1

)/pψ∗ π (Yk |Y0

k−1

) and ∇λλ0 pψ∗ π (Yk |Y0

k−1

)/pψ∗ π (Yk |Y0

) constitute

the generalized score sπk in Corollary 1. This score is approximated by a stationary martingale
difference sequence, where the approximation error satisfies Assumption 3.
We collect some notations. Recall ψ = (η 0 , λ0 )0 and η = (γ 0 , ν 0 )0 . For a q × 1 vector λ and a
q × q matrix s, define the qλ × 1 vectors v(λ) and V (s) as
v(λ) := (λ21 , . . . , λ2q , λ1 λ2 , . . . , λ1 λq , λ2 λ3 , . . . , λ2 λq , . . . , λq−1 λq )0 ,
V (s) := (s11 /2, . . . , sqq /2, s12 , . . . , s1q , s23 , . . . , s2q , . . . , sq,q−1 )0 .

14

(16)

Noting that α(1 − α) > 0 for α ∈ Θα , define, with tλ (λ, π) := α(1 − α)v(λ),
t(ψ, π) :=

η − η∗

!

tλ (λ, π)

, s%k :=

k−1

!

sηk
sλ%k

, where sηk :=

∇η pψ∗ π (Yk |Y0

k−1

pψ∗ π (Yk |Y0

)

)

=

∇γ fk∗ /fk∗

!

∇θ fk∗ /fk∗

,

(17)

.

(18)

and sλ%k := V (sλλ%k ) with
k−1

sλλ%k

k−1

∇θθ0 fk∗ X k−t
:=
=
+
%
k−1
fk∗
α(1 − α)pψ∗ π (Yk |Y0 )
t=1
∇λλ0 pψ∗ π (Yk |Y0

)



∇θ ft∗ ∇θ0 fk∗ ∇θ fk∗ ∇θ0 ft∗
+
ft∗
fk∗
fk∗
ft∗



Here, s%k in (17) depends on % but not on α and corresponds to sπk in Corollary 1. The following proposition shows that the log-likelihood function is approximated by a quadratic function of
√
nt(ψ, π). Let Nε := {ϑ2 ∈ Θ2 : |t(ψ, π)| < ε}. Let Anε (ξ) := {ϑ ∈ Nε : `n (ψ, π, ξ) − `n (ψ ∗ , π, ξ) ≥
0} and Anεc (ξ) := Anε (ξ)∪Nc/√n , where we suppress the subscript 2 from ξ2 . We use this definition
of Anεc (ξ) through Sections 6.1–6.3. As shown in Sections 6.2 and 6.3, Assumption 6 does not hold
for regime switching models with a normal distribution.
Assumption 6. 0 < inf %∈Θ% λmin (I% ) ≤ sup%∈Θ% λmax (I% ) < ∞ for I% = limk→∞ Eϑ∗ (s%k s0%k ),
where s%k is given in (17).
Proposition 7. Suppose Assumptions 1, 2, 4, 5, and 6 hold. Then, under the null hypothesis of
M = 1, (a) supξ supϑ∈Anε (ξ) |t(ψ, π)| = Opε (n−1/2 ); and (b) for any c > 0,
sup

sup

`n (ψ, π, ξ) − `n (ψ ∗ , π, ξ) −

√

nt(ψ, π)0 νn (s%k ) + nt(ψ, π)0 I% t(ψ, π)/2 = opε (1). (19)

ξ∈Ξ ϑ∈Anεc (ξ)

We proceed to derive the asymptotic distribution of the LRTS. With s%k defined in (17), define
Iη := Eϑ∗ (sηk s0ηk ),
Iηλ% :=

0
Iλη%
,

Iλ%1 %2 := lim Eϑ∗ (sλ%1 k s0λ%2 k ),
k→∞

Iλ.η%1 %2 := Iλ%1 %2 −

Iλη%1 Iη−1 Iηλ%2 ,

Iλη% := lim Eϑ∗ (sλ%k s0ηk ),
k→∞

Iλ.η% := Iλ.η%% ,

Zλ% := (Iλ.η% )−1 Gλ.η% ,
(20)

where Gλ.η% is a qλ -vector mean zero Gaussian process indexed by % with cov(Gλ.η%1 , Gλ.η%2 ) =
√
Iλ.η%1 %2 . Define the set of admissible values of nα(1 − α)v(λ) when n → ∞ by v(Rq ) := {x ∈
Rqλ : x = v(λ) for some λ ∈ Rq }. Define t̃λ% by
rλ% (t̃λ% ) =

inf

tλ ∈v(Rq )

rλ% (tλ ),

rλ% (tλ ) := (tλ − Zλ% )0 Iλ.η% (tλ − Zλ% ).

(21)

The following proposition establishes the asymptotic null distribution of the LRTS.
Proposition 8. Suppose Assumptions
 1, 2, 4, 5, and 6 hold. Then, under the null hypothesis of
d
0
M = 1, LRn → sup%∈Θ% t̃λ% Iλ.η% t̃λ% .
In Proposition 8, the LRTS and its asymptotic distribution depend on the choice of  because
Θ% = [−1 + 2, 1 − 2]. It is possible to develop a version of the EM test (Chen and Li, 2009;
15

Chen et al., 2012; Kasahara and Shimotsu, 2015) in this context that does not impose an explicit
restriction on the parameter space for p11 and p22 ; however, we leave such an extension to future
research.
Remark 1. When applied to the Markov regime switching model, the tests of Carrasco et al. (2014)
P
k−t (∇ f /f )(∇ 0 f /f ) on ∇ f /f , where
use the residuals from projecting ∇θθ0 fk /fk + 2 k−1
θ t t
θ k k
θ k k
t=1 %
both are evaluated at the one-regime MLE. Therefore, in the non-normal case, the LRT and tests
of Carrasco et al. (2014) are based on the same score function.

6.2

Heteroscedastic normal distribution

Suppose that Yk ∈ R in the j-th regime follows a normal distribution with regime-specific intercept
µj and variance σj2 . We split θj into θj = (ζj , σj2 )0 = (µj , βj0 , σj2 )0 , and write the density of the j-th
regime as
f (yk |yk−1 ; γ, θj ) = f (yk |yk−1 ; γ, ζj , σj2 ) =
for some function $.

1
φ
σj



yk − µj − $(yk−1 ; γ, βj )
σj


,

(22)

In many applications, $ is a linear function of γ and βj , e.g.,

$(yk−1 , wk ; γ, βj ) = (yk−1 )0 βj + wk0 γ. Consider the following reparameterization introduced in
Kasahara and Shimotsu (2015) (θ in Kasahara and Shimotsu corresponds to ζ here):


ζ1





νζ + (1 − α)λζ


 
 ζ2  
νζ − αλζ

 
 σ 2  =  ν + (1 − α)(2λ + C λ2 )
σ
1 µ
 1   σ
σ22
νσ − α(2λσ + C2 λ2µ )




,



(23)

where νζ = (νµ , νβ0 )0 , λζ = (λµ , λ0β )0 , C1 := −(1/3)(1 + α), and C2 := (1/3)(2 − α), so that
C1 = C2 − 1. Collect the reparameterized parameters, except for α, into one vector ψα . As in
Section 6.1, we suppress the subscript α from ψα . Let the reparameterized density be

gψ (yk |yk−1 , xk ) = f yk |yk−1 ; γ, νζ + (qk − α)λζ , νσ + (qk − α)(2λσ + (C2 − qk )λ2µ ) .

(24)

Let ψ := (η 0 , λ0 )0 ∈ Θψ = Θη ×Θλ , where η := (γ 0 , νζ0 , νσ )0 and λ := (λ0ζ , λσ )0 . Because the likelihood
function of a normal mixture model is unbounded when σj → 0 (Hartigan, 1985), we impose σj ≥ σ
for a small σ > 0 in Θψ . We proceed to derive the derivatives of gψ (Yk |Yk−1 , Xk ) evaluated at ψ ∗ .
∇ψ gk∗ , ∇λη0 gk∗ , and ∇λλ0 gk∗ are the same as those given in (10) except for ∇λ2µ gk∗ and that those with
respect to λjσ are multiplied by 2j . The higher-order derivatives of gψ (Yk |Yk−1 , Xk ) with respect
to λµ are derived by following Kasahara and Shimotsu (2015). From Lemma 6 and the fact that

16

normal density f (µ, σ 2 ) satisfies
∇µ2 f (µ, σ 2 ) = 2∇σ2 f (µ, σ 2 ),

∇µ3 f (µ, σ 2 ) = 2∇µσ2 f (µ, σ 2 ), and

∇µ4 f (µ, σ 2 ) = 2∇µ2 σ2 f (µ, σ 2 ) = 4∇σ2 σ2 f (µ, σ 2 ),

(25)

we have
∇λiµ gk∗ = dik ∇µi fk∗ ,

i = 1, . . . , 4,

(26)

where
d1k := qk − α,

d0k := 1,

d3k := 2(qk − α)2 (1 − α − qk ),

d2k := (qk − α)(C2 − α),

d4k := −2(qk − α)4 + 3(qk − α)2 (α − C2 )2 .
n

It follows from Eϑ∗ [qk |Y−∞ ] = α, (11), and elementary calculation that
n

Eϑ∗ [dik |Y−∞ ] = 0,

k

Eϑ∗ [∇λiµ gk∗ |Y−∞ ] = 0,

i = 1, 2, 3,

n

(27)

Eϑ∗ [d4k |Y−∞ ] = α(1 − α)b(α),
k
Eϑ∗ [∇λ4µ gk∗ |Y−∞ ]

= α(1 − α)b(α)∇µ4 fk∗ = α(1 − α)b(α)4∇σ2 σ2 fk∗ =

k
b(α)Eϑ∗ [∇λ2σ gk∗ |Y−∞ ],

k

k

with b(α) := −(2/3)(α2 − α + 1) < 0. Hence, Eϑ∗ [∇λ2σ gk∗ |Y−∞ ] and Eϑ∗ [∇λ4µ gk∗ |Y−∞ ] are linearly
dependent.
k−1

We proceed to derive a representation of ∇j pψ∗ π (Yk |Y0

k−1

)/pψ∗ π (Yk |Y0

) in terms of ∇j fk∗ .

Repeating the calculation leading to (13)–(15) and using (27) gives the following. First, (13) and
k−1

(14) still hold; second, the elements of ∇λλ0 pψ∗ π (Yk |Y0

k−1

)/pψ∗ π (Yk |Y0

) except for the (1, 1)-th

element are given by (15) after adjusting that the derivative with respect to λσ must be multiplied
n

n

by 2 (e.g., Eϑ∗ [∇λσ gk∗ |Y−∞ ] = 2∇σ2 fk∗ and Eϑ∗ [∇λσ λµ gk∗ |Y−∞ ] = 2∇σ2 µ fk∗ ); third,
k−1

∇λ2µ pψ∗ π (Yk |Y0

k−1

pψ∗ π (Yk |Y0
k−1

When % 6= 0, ∇λ2µ pψ∗ π (Yk |Y0

)

= α(1 − α)

)

k−1
X

k−t

%

t=1
k−1

)/pψ∗ π (Yk |Y0



∇µ ft∗ ∇µ fk∗
.
2 ∗
ft
fk∗

(28)

) is a non-degenerate random variable as in the
k−1

non-normal case. When % = 0, however, ∇λ2µ pψ∗ π (Yk |Y0

k−1

)/pψ∗ π (Yk |Y0

) becomes identically

equal to 0, and indeed the first non-zero derivative with respect to λµ is the fourth derivative.
Because of this degeneracy, we derive the asymptotic distribution of the LRTS by expanding
`n (ψ, π, ξ) − `n (ψ ∗ , π, ξ) four times. It is not correct, however, to simply approximate `n (ψ, π, ξ) −
`n (ψ ∗ , π, ξ) by a quadratic function of λ2µ (and other terms) when % 6= 0 and a quadratic function
of λ4µ when % = 0. This results in discontinuity at % = 0 and fails to provide a valid uniform
approximation. We establish a uniform approximation by expanding `n (ψ, π, ξ) four times but
expressing `n (ψ, π, ξ) in terms of %λ2µ , λ4µ , and other terms.

17

For m ≥ 0, define ζk,m (%) :=

Pk−1

k−t−1 2∇ f ∗ ∇ f ∗ /f ∗ f ∗ .
µ t µ k
t k
t=−m+1 %

k−1

∇λ2µ pψ∗ π (Yk |Y0

)

k−1

α(1 − α)pψ∗ π (Yk |Y0

=
)

k−1
X
t=1

Then, we can write (28) as



∇µ f ∗ ∇µ fk∗
= %ζk,0 (%).
%k−t 2 ∗ t
ft
fk∗

(29)

k−1

Note that ζk,m (%) satisfies Eϑ∗ [ζk,m (%)|Y−m ] = 0 and is non-degenerate even when % = 0. Define
v(λβ ) as v(λ) in (16) but replacing λ with λβ . Collect the relevant parameters as
t(ψ, π) :=
where

!

η − η∗

,

tλ (λ, π)


%λ2µ

(30)







λµ λσ


λ2 + b(α)λ4 /12


µ
tλ (λ, π) := α(1 − α)  σ
,


λ
λ
µ
β




λβ λσ


v(λβ )

(31)

with b(α) = −(2/3)(α2 − α + 1) < 0. Recall θj = (ζj0 , σj2 )0 = (µj , βj0 , σj2 )0 . Similarly to (18), define
the elements of the generalized score by


∗

sλµβ %k sλµσ %k



k−1

∗

 ∇θθ0 fk X k−t
+
%
sλβµ %k sλββ %k sλβσ %k  =
∗
fk
t=1
sλσµ %k sλβσ %k sλσσ %k



∇θ ft∗ ∇θ0 fk∗ ∇θ fk∗ ∇θ0 ft∗
+
ft∗
fk∗
fk∗
ft∗


.

(32)

Define the generalized score as


s%k :=

sηk
sλ%k

!
,

where

sηk :=

∇γ fk∗ /fk∗
∇θ fk∗ /fk∗

!
and sλ%k






:= 





ζk,0 (%)/2




2sλµσ %k 

2sλσσ %k 

.
sλβµ %k 

2sλβσ %k 

V (sλββ %k )

(33)

The following proposition establishes a uniform approximation of the log-likelihood ratio.
Assumption 7. (a) 0 < inf %∈Θ% λmin (I% ) ≤ sup%∈Θ% λmax (I% ) < ∞ for I% = limk→∞ Eϑ∗ (s%k s0%k ),
where s%k is given in (33). (b) σ1∗ , σ2∗ > σ .
Proposition 9. Suppose Assumptions 1, 2, 4, 5, and 7 hold and the density of the j-th regime is
given by (22). Then, under the null hypothesis of M = 1, (a) supϑ∈Anε (ξ) |t(ψ, π)| = Opε (n−1/2 );

18

and (b) for any c > 0,
sup

sup

`n (ψ, π, ξ) − `n (ψ ∗ , π, ξ) −

√

nt(ψ, π)0 νn (s%k ) + nt(ψ, π)0 I% t(ψ, π)/2 = opε (1). (34)

ξ∈Ξ ϑ∈Anεc (ξ)

The asymptotic null distribution of the LRTS is characterized by the supremum of 2t0λ Gλ.η% −
t0λ Iλ.η% tλ , where Gλ.η% and Iλ.η% are defined analogously to those in (20) but with s%k defined in
(33), and the supremum is taken with respect to tλ and % ∈ Θ% under the constraint implied by the
√
limit of the set of possible values of ntλ (λ, π) as n → ∞. This constraint is given by the union of
Λ1λ and Λ2λ% , where qβ := dim(β), qλ := 3 + 2qβ + qβ (qβ + 1)/2, and
Λ1λ := {tλ = (t%µ2 , tµσ , tσ2 , t0βµ , t0βσ , t0v(β) )0 ∈ Rqλ :
(t%µ2 , tµσ , tσ2 , t0βµ )0 ∈ R × R × R− × Rqβ , tβσ = 0, tv(β) = 0},
Λ2λ% := {tλ = (t%µ2 , tµσ , tσ2 , t0βµ , t0βσ , t0v(β) )0 ∈ Rqλ : t%µ2 = %λ2µ , tµσ = λµ λσ ,

(35)

tσ2 = λ2σ , tβµ = λβ λµ , tβσ = λβ λσ , tv(β) = vβ (λβ ) for some λ ∈ R2+qβ }.
Note that Λ2λ% depends on %, whereas Λ1λ does not depend on %. Heuristically, Λ1λ and Λ2λ% correspond
√
to the limits of the set of possible values of ntλ (λ, π) when lim inf n→∞ n1/8 |λµ | > 0 and λµ =
o(n−1/8 ), respectively. When lim inf n→∞ n1/8 |λµ | > 0, we have (λ̂σ , λ̂β ) = Op (n−3/8 ) because
√
tλ (λ̂, π) = Op (n−1/2 ). Further, the set of possible values of n%λ2µ converges to R because % can
√
be arbitrarily small. Consequently, the limit of ntλ (λ, π) is characterized by Λ1λ .
Define Zλ% and Iλ.η% as in (20) but with sπk defined in (33). Let Zλ0 and Iλ.η0 denote Zλ% and
Iλ.η% evaluated at % = 0. Define t̃1λ and t̃2λ% by
rλ (t̃1λ ) = inf rλ (tλ ),
tλ ∈Λ1λ

rλ% (t̃2λ% ) = inf rλ% (tλ ),
tλ ∈Λ2λ%

rλ (tλ ) := (tλ − Zλ0 )0 Iλ.η0 (tλ − Zλ0 )
rλ% (tλ ) := (tλ − Zλ% )0 Iλ.η% (tλ − Zλ% ).

(36)

The following proposition establishes the asymptotic null distribution of the LRTS.
Proposition 10. Suppose that the assumptions in Proposition 9 hold. Then, under the null hyd

pothesis of M = 1, LRn → max{I{% = 0}(t̃1λ )0 Iλ.η0 t̃1λ , sup%∈Θ% (t̃2λ% )0 Iλ.η% t̃2λ% }.
Remark 2. Qu and Zhuo (2017) derive the asymptotic distribution of the LRTS under the restriction that % ≥  > 0.
Remark 3. It is possible to extend our analysis to the exponential-LR type tests studied by Andrews
and Ploberger (1994) and Carrasco et al. (2014).

6.3

Homoscedastic normal distribution

Suppose that Yk ∈ R in the j-th regime follows a normal distribution with the regime-specific
intercept µj but with common variance σ 2 . We split γ and θj into γ = (γ̃ 0 , σ 2 )0 and θj = (µj , βj0 )0 ,
19

and write the density of the j-th regime as
f (yk |yk−1 ; γ, θj ) = f (yk |yk−1 ; γ̃, θj , σ 2 ) =

1
φ
σ



yk − µj − $(yk−1 ; γ̃, βj )
σ


,

(37)

for some function $. Consider the following reparameterization:


θ1





νθ + (1 − α)λ





 
νθ − αλ
,
 θ2  = 
σ2
νσ − α(1 − α)λ2µ

(38)

where νθ = (νµ , νβ0 )0 and λ = (λµ , λ0β )0 . Collect the reparameterized parameters, except for α, into
one vector ψα . Suppressing α from ψα , let the reparameterized density be

gψ (yk |yk−1 , xk ) = f yk |yk−1 ; γ̃, νθ + (qk − α)λ, νσ − α(1 − α)λ2µ .

(39)

Let η = (γ̃ 0 , νθ0 , νσ )0 ; then, the first and second derivatives of gψ (yk |yk−1 , xk ) with respect to η and
λ are the same as those given in (10) except for ∇λ2µ gψ (yk |yk−1 , xk ). We derive the higher-order
derivatives of gψ (yk |yk−1 , xk ) with respect to λµ . From Lemmas 6 and (25), we obtain
∇ληi gk∗ = d1k ∇θηi fk∗

for i = 0, 1, . . . ,

∇λiµ gk∗ = dik ∇µi fk∗

for i = 0, 1, . . . , 4,

(40)

where d0k := 1, d1k := qk − α, d2k := (qk − α)2 − α(1 − α), d3k := (qk − α)3 − 3(qk − α)α(1 − α),
n

and d4k := (qk − α)4 − 6(qk − α)2 α(1 − α) + 3α2 (1 − α)2 . It follows from Eϑ∗ [qk |Y−∞ ] = α, (11),
and elementary calculation that
k

Eϑ∗ [∇λiµ gk∗ |Y0 ] = 0,
k
Eϑ∗ [d3k |Y0 ]

k

Eϑ∗ [dik |Y0 ] = 0,

= α(1 − α)(1 − 2α),

i = 1, 2,
k

Eϑ∗ [d4k |Y0 ] = α(1 − α)(1 − 6α + 6α2 ).

(41)

Repeating the calculation leading to (13)–(15) and using (41) gives the following. First, (13)
k−1

and (14) still hold; second, the elements of ∇λλ0 pψ∗ π (Yk |Y0

k−1

)/pψ∗ π (Yk |Y0

) are given by (15)

k−1
k−1
except for the (1, 1)-th element; third, ∇λ2µ pψ∗ π (Yk |Y0 )/pψ∗ π (Yk |Y0 ) is given by (28). Further,
k−1
k−1
Lemma 8 in the appendix shows that when % = 0, ∇λ3µ pψ∗ π (Yk |Y0 )/pψ∗ π (Yk |Y0 ) = α(1 −
k−1
k−1
α)(1 − 2α)∇µ3 fk∗ /fk∗ and ∇λ4µ pψ∗ π (Yk |Y0 )/pψ∗ π (Yk |Y0 ) = α(1 − α)(1 − 6α + 6α2 )∇µ4 fk∗ /fk∗ .
k−1
k−1
Because ∇λ3µ pψ∗ π (Yk |Y0 )/pψ∗ π (Yk |Y0 ) = 0 when α = 1/2 and % = 0, we expand `n (ψ, π, ξ)
four times and express it in terms of %λ2µ , (1 − 2α)λ3µ , λ4µ , and other terms to establish a uniform

approximation.

20

Collect the relevant parameters as


t(ψ, π) :=

η−

η∗

!

tλ (λ, π)

%λ2µ





 (1 − 2α)λ3µ 


2 4
and tλ (λ, π) := α(1 − α) 
(1 − 6α + 6α )λµ  .


λβ λµ


v(λβ )

(42)

Define the generalized score as


s%k :=

sηk
sλ%k

!
,

where

sηk :=

∇γ fk∗ /fk∗
∇θ fk∗ /fk∗

!
and sλ%k

ζk,0 (%)/2


 sλ3µ k /3!

:= 
 sλ4µ k /4!

 sλβµ %k





,




(43)

V (sλββ %k )
where ζk,m (%) is defined as in (29), sλiµ k := ∇µi fk∗ /fk∗ for i = 3, 4, and sλβµ %k and sλββ %k are
defined as in (32) but using the density (37) in place of (22). Define, with qβ := dim(β) and
qλ := 3 + qβ + qβ (qβ + 1)/2,
Λ1λ := {tλ = (t%µ2 , tµ3 , tµ4 , t0βµ , t0v(β) )0 ∈ Rqλ : (t%µ2 , tµ3 , tµ4 , t0βµ )0 ∈ R × R × R− × Rqβ , tv(β) = 0},
Λ2λ% := {tλ = (t%µ2 , tµ3 , tµ4 , t0βµ , t0v(β) )0 ∈ Rqλ : t%µ2 = %λ2µ , tµ3 = tµ4 = 0, tβµ = λβ λµ ,
tv(β) = vβ (λβ ) for some λ ∈ R1+qβ }.
(44)
The following two propositions correspond to Propositions 9 and 10, establishing a uniform
approximation of the log-likelihood ratio and asymptotic distribution of the LRTS.
Assumption 8. 0 < inf %∈Θ% λmin (I% ) ≤ sup%∈Θ% λmax (I% ) < ∞ for I% = limk→∞ Eϑ∗ (s%k s0%k ),
where s%k is given in (43).
Proposition 11. Suppose Assumptions 1, 2, 4, 5, and 8 hold and the density of the j-th regime is
given by (37). Then, statements (a) and (b) of Proposition 9 hold.
Proposition 12. Suppose that the assumptions in Proposition 11 hold. Then, under the null
d

hypothesis of M = 1, LRn → max{I{% = 0}(t̃1λ )0 Iλ.η0 t̃1λ , sup%∈Θ% (t̃2λ% )0 Iλ.η% t̃2λ% }, where t̃1λ and t̃2λ%
are defined as in (36) but in terms of (Zλ% , Iλ.η% , Zλ0 , Iλ.η0 ) constructed with s%k defined in (43)
and Λ1λ and Λ2λ% defined in (44).

7

Testing H0 : M = M0 against HA : M = M0 + 1 for M0 ≥ 2

In this section, we derive the asymptotic distribution of the LRTS for testing the null hypothesis
of M0 regimes against the alternative of M0 + 1 regimes for general M0 ≥ 2. We suppress the
21

covariate Wab unless confusion might arise.
Let ϑ∗M0 = ((ϑ∗M0 ,x )0 , (ϑ∗M0 ,y )0 )0 denote the parameter of the M0 -regime model, where ϑ∗M0 ,x
contains p∗ij = qϑ∗M

(i, j) > 0 for i = 1, . . . , M0 and j = 1, . . . , M0 − 1, and ϑ∗M0 ,y =
PM0 −1 ∗
∗ )0 , (γ ∗ )0 )0 . We assume max
∗
∗
((θ1∗ )0 , . . . , (θM
i
j=1 pij < 1 and θ1 < . . . < θM0 for identification.
0
0 ,x

The true M0 -regime conditional density of Y1n given Y0 and x0 is
pϑ∗M (Y1n |Y0 , x0 ) =
0

n
Y

X

pϑ∗M (Yk , xk |Yk−1 , xk−1 ),

(45)

0

n
xn
1 ∈XM k=1
0

where pϑ∗M (yk , xk |yk−1 , xk−1 ) = gϑ∗M ,y (yk |yk−1 , xk )qϑ∗M ,x (xk−1 , xk ) with gϑ∗M ,y (yk |yk−1 , xk ) =
0
0
0
0
P
∗
j=1,...,M0 I{xk = j}f (yk |yk−1 ; γ, θj ).
Let the conditional density of Y1n of an (M0 + 1)-regime model be
pϑM0 +1 (Y1n |Y0 , x0 ) :=

n
Y

X
n
xn
1 ∈XM0 +1

pϑM0 +1 (Yk , xk |Yk−1 , xk−1 ),

(46)

k=1

where pϑM0 +1 (yk , xk |yk−1 , xk−1 ) is defined similarly to pϑ∗M (yk , xk |yk−1 , xk−1 ) with ϑM0 +1,x :=
0

0
{pij }i=1,...,M0 +1,j=1,...,M0 and ϑM0 +1,y := (θ10 , . . . , θM
, γ 0 )0 . We assume that mini,j pij ≥  for
0 +1

some  ∈ (0, 1/2).
0
Write the null hypothesis as H0 = ∪M
m=1 H0m with

H0m : θ1 < · · · < θm = θm+1 < · · · < θM0 +1 .
Define the set of values of ϑM0 +1 that yields the true density (45) under Pϑ∗M as Υ∗ := {ϑM0 +1 ∈
0

ΘM0 +1, : pϑM0 +1 (Y1n |Y0 , x0 ) = pϑ∗M (Y1n |Y0 , x0 ) Pϑ∗M -a.s.}. Under H0m , the (M0 + 1)-regime
0

0

∗ and the transition matrix
model (46) generates the true M0 -regime density (45) if θm = θm+1 = θm

of Xk reduces to that of the true M0 -regime model.
0 )0 ,
We reparameterize the transition probability of Xk by writing ϑM0 +1,x as ϑM0 +1,x = (ϑ0xm , πxm

where ϑxm is point identified under H0m , while πxm is not point identified under H0m . The
transition probability of Xk under ϑM0 +1,x equals the transition probability of Xk under ϑ∗M0 ,x if
and only if ϑxm = ϑ∗xm . The detailed derivation including the definition of ϑ∗xm is provided in
Section 11.2.5 in the appendix. Define the subset of Υ∗ that corresponds to H0m as

∗
Υ∗m := ϑM0 +1 ∈ ΘM0 +1 : θj = θj∗ for 1 ≤ j < m; θm = θm+1 = θm
;
∗
θj = θj−1
for h + 1 < j ≤ M0 + 1; γ = γ ∗ ; ϑxm = ϑ∗xm ;

then, Υ∗ = Υ∗1 ∪ · · · ∪ Υ∗M0 holds.
For M = M0 , M0 + 1, let `n (ϑM , ξM ) := log

P

M
n
x0 =1 pϑM (Y1 |Y 0 , x0 )ξM (x0 )



denote the M -

regime log-likelihood for a given initial distribution ξM (x0 ) ∈ ΞM . We treat ξM (x0 ) as fixed. Let
ϑ̂M0 := arg maxϑM0 ∈ΘM0 `n (ϑM0 , ξM0 ) and ϑ̂M0 +1 := arg maxϑM0 +1 ∈ΘM0 +1 `n (ϑM0 +1 , ξM0 +1 ). The
22

following proposition shows that the MLE is consistent in the sense that the distance between
ϑ̂M0 +1 and Υ∗ tends to 0 in probability. The proof of Proposition 13 is essentially the same as the
proof of Proposition 6 and hence is omitted.
Assumption 9. (a) ΘM0 and ΘM0 +1 are compact, and ϑ∗M0 is in the interior of ΘM0 . (b) For
all (x, x0 ) ∈ X and all (y, y 0 , w) ∈ Y s × Y × W, f (y 0 |y0 , w; γ, θ) is continuous in (γ, θ). (c)
0

0

1 )] = E ∗ [log p ∗ (Y |Y
1
Eϑ∗M [log(pϑM0 (Y1 |Y−m , W−m
1
ϑM
ϑM
−m , W−m )] for all m ≥ 0 if and only if
0

0

0

0

0

0 )] = E ∗ [log p ∗ (Y |Y
1
ϑM0 = ϑ∗M0 . (d) Eϑ∗M [log(pϑM0 +1 (Y1 |Y−m , W−m
1
ϑM
ϑM
−m , W−m )] for all m ≥ 0
0

0

if and only if ϑM0 +1 ∈ Υ∗ .

0

Proposition 13. Suppose Assumptions 1, 2, and 9 hold. Then, under the null hypothesis of
p

p

M = M0 , ϑ̂M0 → ϑ∗M0 and inf ϑM0 +1 ∈Υ∗ |ϑ̂M0 +1 − ϑM0 +1 | → 0.
Let LRM0 ,n := 2[`n (ϑ̂M0 +1 , ξM0 +1 ) − `n (ϑ̂M0 , ξM0 )] denote the LRTS for testing H0 : M = M0
against HA : M = M0 +1. We proceed to derive the asymptotic distribution of the LRTS by analyzing the behavior of the LRTS when ϑM0 +1 ∈ Υ∗m for each m. Define Jm := {m, m+1}. Observe that
k , then Xk follows a two-state Markov chain on J whose transition probability is charif Xk1 ∈ Jm
m
1
2 ).
acterized by αm := PϑM0 +1 (Xk = m|Xk ∈ Jm ) and %m := corrϑM0 +1 (Xk−1 , Xk |(Xk−1 , Xk ) ∈ Jm

Collect reparameterized πxm into πxm := (%m , αm , φ0m )0 , where φm does not affect the transition
k . See Section 11.2.5 in the appendix for the detailed derivation.
probability of Xk1 when Xk1 ∈ Jm

Define qkj := I{Xk = j}; then, we can write αm and %m as αm = EϑM0 +1 (qkm |Xk ∈ Jm )
∞

2 ). Because Y
and %m = corrϑM0 +1 (qk−1,m , qkm |(Xk−1 , Xk ) ∈ Jm
−∞ provides no information for

distinguishing between Xk = m and Xk = m + 1 if θm = θm+1 , we can write αm and %m as
∞

αm = EϑM0 +1 (qkm |Xk ∈ Jm , Y−∞ )

∞

2
, Y−∞ ).
and %m = corrϑM0 +1 (qk−1,m , qkm |(Xk−1 , Xk ) ∈ Jm

(47)

7.1

Non-normal distribution

For non-normal component distributions, consider the following reparameterization similar to (8):
θm
θm+1

!
=

νm + (1 − αm )λm
νm − αm λm

!
.

0 , λ0 )0 , where η
Collect the reparameterized identified parameters into one vector ψm := (ηm
m =
m
m−1 0
0
0
0 +1
, νm , {θj0 }M
(γ 0 , {θj0 }j=1
j=m+2 , ϑxm ) , so that the reparameterized (M0 +1)-regime log-likelihood func∗ = (η ∗ , λ∗ ) = ((ϑ∗ )0 , 00 )0 denote the value of ψ under H
tion is `n (ψm , πxm , ξM0 +1 ). Let ψm
m
0m .
m m
M0

Define the reparameterized conditional density of yk as
gψm (yk |yk−1 , xk ) := I{xk ∈ Jm }f (yk |yk−1 ; γ, νm + (qkm − αm )λm ) +

X
j∈J m

23

qkj f (yk |yk−1 ; γ, θj ),

∗ denote f (Y |Y
∗ ∗
where J m := {1, . . . , M0 + 1} \ Jm . Let fmk
k
k−1 ; γ , θm ). It follows from (47) and

the law of iterated expectations that
"

#
I{Xk ∈ Jm }(qkm − αm ) n
Eϑ∗M
Y−∞
0
∗ (Yk |Y k−1 , Xk )
gψm




qkm − αm
n
n
∗
∗
= EϑM EϑM
Xk ∈ Jm , Y−∞ I{Xk ∈ Jm } Y−∞ = 0,
∗
0
0
fmk
"
#
I{Xt1 ∈ Jm }I{Xt2 ∈ Jm }(qt1 h − αm )(qt2 h − αm ) n
(48)
Eϑ∗M
Y−∞
0
∗ (Yt |Y t −1 , Xt )gψ ∗ (Yt |Y t −1 , Xt )
gψm
2
1
1
1
2
2
m




(qt1 h − αm )(qt2 h − αm ) t2
n
n
2
t2 −t1 +1
∗
∗
Xt1 ∈ Jm
, Y−∞ I{(Xt1 , Xt2 ) ∈ Jm } Y−∞
= EϑM EϑM
∗ f∗
0
0
fmt
1 mt2
=

t2 −t1
αm (1 − αm )%m
n
2
|Y−∞ ),
Pϑ∗M ((Xt1 , Xt2 ) ∈ Jm
∗
∗
0
fmt1 fmt2

t2 ≥ t1 ,

∗
∗ (Yk |Y k−1 , Xk ) = f
where the second equality holds because gψm
mk if Xk ∈ Jm , and the last equality
n

t2
t2 −t1 +1 , Y
holds because, conditional on {Xtt21 ∈ Jm
−∞ }, Xt1 is a two-state stationary Markov process

with parameter (αm , %m ).
∗ , q ∗ , and p∗ denote g ∗
Let g0k
ϑM
0k
0k

0 ,y

∗
Let ∇g0k
and ∇p∗0k

(Yk , Xk |Yk−1 , Xk−1 ), qϑ∗M

k−1

0 ,x

(Xk−1 , Xk ), and pϑ∗M (Yk |Y0

denote the derivative of gϑM0 ,y (Yk , Xk |Yk−1 , Xk−1 ) evaluated at

ϑ∗M0 ,y ,

0

and define

).

∗
∇q0k

similarly. Repeating a derivation similar to (10)–(15) but using (48) in place of (12), we

obtain

k−1

∇ηm pψm
∗ π (Yk |Y 0
=

k
X

k−1

)/pψm
∗ π (Yk |Y 0

k−1
h
h
i X
i
k
k−1
∗ ∗
∗ ∗
q0t ) Y0 −
q0t ) Y0
Eϑ∗ ∇ϑM0 log(g0t
Eϑ∗ ∇ϑM0 log(g0t

t=1
k−1

= ∇ϑM0 pϑ∗M (Yk |Y0
0

k−1

∇λm pψm
∗ π (Yk |Y 0

k−1
X
t=1

)/pϑ∗M

k−1

)/pψm
∗ π (Yk |Y 0

k−1
∇λm λm 0 pψm
)
∗ π (Yk |Y 0
k−1
pψm
)
∗ π (Yk |Y 0

+ αm (1 − αm )

)

) = 0,

= αm (1 − αm )

%k−t
m



0

(49)

t=1
k−1
(Yk |Y0 ),

k−1

0 p ∗ (Yk |Y
∇ λm η m
ψm π
0

k−1

)/pψm
∗ π (Yk |Y 0

) = 0,

(50)

∗
∇θθ0 fmk
k
Pϑ∗M (Xk ∈ Jm |Y0 )
∗
0
fmk

∗
∗ ∇ 0f∗
∗
∇θ fmk
∇θ fmt
∇θ0 fmt
θ mk
+
∗
∗
∗
∗
fmt
fmk
fmk
fmt



k

2
Pϑ∗M ((Xt , Xk ) ∈ Jm
|Y0 ).

(51)

0

Define %̃ := (%1 , . . . , %M0 )0 , define tλ (λm , πm ) as tλ (λ, π) in (17) by replacing (λ, π) with (λm , πm ),

24

and let

s1λ%1 k
 . 
. 
:= 
 . ,
0
sM
λ%M0 k
(52)


t(ψm , πm ) :=

!

ηm − η ∗

tλ (λm , πm )

s̃ηk

, s̃%̃k :=

k−1

!

s̃λ%̃k

, where s̃ηk :=

∇ηm pψm
∗ π (Yk |Y 0

k−1

pψm
∗ π (Yk |Y 0

)

)

, s̃λ%̃k

m
m
and sm
λ%m k := V (sλλ%m k ), where sλλ%m k is defined similarly to (18) as
∗
∇θθ0 fmk
k
Pϑ∗M (Xk ∈ Jm |Y0 )
∗
0
fmk


k−1
∗
∗ ∇ 0f∗
∗
X
∇θ fmk
∇θ fmt
∇θ0 fmt
k
θ mk
2
+
%k−t
+
Pϑ∗M ((Xt , Xk ) ∈ Jm
|Y0 ).
m
∗
∗
∗
∗
0
fmt
fmk
fmk
fmt

sm
λλ%m k :=

(53)

t=1

Similarly to (20), define
Ĩη := Eϑ∗M (s̃ηk s̃0ηk ),
0

0
Ĩηλ%̃ := Ĩλη
%̃ ,

Ĩλ%̃1 %̃2 := lim Eϑ∗M (s̃λ%̃1 k s̃0λ%̃2 k ),
k→∞

0

Ĩλη%̃ := lim Eϑ∗M (s̃λ%̃k s̃0ηk ),
k→∞

0

0
m
m
:= Eϑ∗M [Gm
Ĩλ.η%
λ.η%m (Gλ.η%m ) ],
m

Ĩλ.η%̃1 %̃2 := Ĩλ%̃1 %̃2 − Ĩλη%̃1 Ĩη−1 Ĩηλ%̃2 ,

(54)

0

m
m
Zλ%
:= (Ĩλ.η%
)−1 Gm
λ.η%m ,
m
m
0 0
0
where Gλ.η%̃ = ((G1λ.η%1 )0 , . . . , (GM
λ.η%M ) ) is an M0 qλ -vector mean zero Gaussian process with
0

cov(Gλ.η%̃1 , Gλ.η%̃2 ) = Ĩλ.η%̃1 %̃2 . Note that Gλ.η%̃ corresponds to the residuals from projecting s̃λ%̃k
on s̃ηk . Define t̃m
λ%m by
m
gλ%
(t̃m
λ%m ) =
m

inf

tλ ∈v(Rq )

m
gλ%
(tλ ),
m

m
m
m
m
gλ%
(tλ ) := (tλ − Zλ%
)0 Ĩλ.η%
(tλ − Zλ%
).
m
m
m
m

The following proposition gives the asymptotic null distribution of the LRTS. Under the stated
assumptions, the log-likelihood function permits a quadratic approximation in the neighborhood
of Υ∗m similar to the one in Proposition 7. Define Am
nεc (ξ) := {ϑM0 +1 ∈ ΘM0 +1 : {`n (ψm , πm , ξ) −
∗ , π , ξ) ≥ 0} ∧ |t(ψ , π )| < ε} ∪ N √ . Under H : M = M , for any c > 0, for m =
`n (ψm
m
m m
0
0
c/ n

1, . . . , M0 , and uniformly in ξ ∈ Ξ and ϑM0 +1 ∈ Am
nεc (ξ),
∗
`n (ψm , πm , ξ) − `n (ψm
, πm , ξ) −

√

nt(ψm , πm )0 νn (s%m k ) + nt(ψm , πm )0 I%m t(ψm , πm )/2 = opε (1),

0 0
0
∗
where s%m k := (s̃0ηk , (sm
λ%m k ) ) and I%m = limk→∞ EϑM (s%m k s%m k ). Consequently, the LRTS is
0

asymptotically distributed as the maximum of the M0 random variables, each of which represents
the asymptotic distribution of the LRTS that tests H0m . Denote the parameter space for %m by
Θ%m , and let Θ̃% := Θ%1 × . . . × Θ%M0 .
Assumption 10. 0 < inf %̃∈Θ̃% λmin (Ĩ%̃ ) ≤ sup%̃∈Θ̃% λmax (Ĩ%̃ ) < ∞ for Ĩ%̃ := limk→∞ Eϑ∗M (s̃%̃k s̃0%̃k ),
0

where s̃%̃k is given in (52).

25

Proposition 14. Suppose
1, 2, 4, 9, and
n Assumptions

o 10 hold. Then, under H0 : M = M0 ,
d
m
0
m
m
(t̃λ%m ) Ĩλ.η%m t̃λ%m .
LRM0 ,n → maxm=1,...,M0 sup%m ∈Θm
%

7.2

Heteroscedastic normal distribution

As in Section 6.2, we assume that Yk ∈ R in the j-th regime follows a normal distribution with
the regime-specific intercept and variance of which density is given by (22). Consider the following
reparameterization similar to (23):


ζm


 ζm+1

 σ2

m
2
σm+1







νζm + (1 − αm )λζm

 
 
νζm − αm λζm
=
  ν + (1 − α )(2λ + C λ2 )
m
σm
1 µm
  σm
2
νσm − αm (2λσm + C2 λµm )



,



where νζm = (νµ , νβ0 )0 , λζm = (λµm , λ0βm )0 , C1 := −(1/3)(1 + αm ), and C2 := (1/3)(2 − αm ). As
0 , λ0 )0 , where
in Section 7.1, we collect the reparameterized identified parameters into ψm := (ηm
m
0
0 M0 +1
0
0
0
0
ηm = (γ 0 , {θj0 }m−1
j=1 , νζm , νσm , {θj }j=m+2 , ϑxm ) and λm := (λζm , λσm ) . Similar to (24), define the

reparameterized conditional density of yk as
gψm (yk |yk−1 , xk ) =

X

qkj f (yk |yk−1 ; γ, θj )

j∈J m


+ I{xk ∈ Jm }f yk |yk−1 ; γ, νζm + (qkm − αm )λζm , νσm + (qkm − αm )(2λσm + (C2 − qkm )λ2µm ) .
∗ , f ∗ , ∇g ∗ , and ∇f ∗ denote g ∗ (Y |Y
∗ ∗
∗ (Yk |Y k−1 , Xk ),
Let gmk
ψm k
k−1 , Xk ), f (Yk |Y k−1 ; γ , θm ), ∇gψm
mk
mk
mk
∗ ). From (26) and a derivation similar to (48), we obtain the following result
and ∇f (Yk |Yk−1 ; γ ∗ , θm

that corresponds to (27) in testing homogeneity:
h

Eϑ∗M

0

Eϑ∗M

0

i
k
∗
∗
∇λiµm gmk
/gmk
Y−∞ = 0, i = 1, 2, 3,
h
i
k
k
∗
∗
∗
∗
/gmk
/fmk
)Pϑ∗M (Xk ∈ Jm |Y−∞ )
∇λ4µm gmk
Y−∞ = αm (1 − αm )b(αm )(∇µ4 fmk
0
h
i
k
∗
∗
= b(αm )Eϑ∗M ∇λ2σm gmk
/gmk
Y−∞ .

(55)

0

Repeating the calculation leading to (49)–(51) and using (55) gives the following. First, (49) and
k−1

(50) still hold; second, the elements of ∇λm λ0m pψ∗ π (Yk |Y0

k−1

)/pψ∗ π (Yk |Y0

) except for the (1, 1)-th

element are given by (51) while adjusting the derivative with respect to λσm by multiplying by 2;
third,
k−1

∇λ2µm pψm
∗ π (Yk |Y 0
k−1

pψm
∗ π (Yk |Y 0

)

= αm (1 − αm )

)

m (% ) :=
For m ≥ 0, define ζk,m
m

k−1
X
t=1


∗ ∇ f∗ 
∇µ fmt
k
µ mk
2
%k−t
2
Pϑ∗M ((Xt , Xk ) ∈ Jm
|Y0 ).
m
∗
∗
0
fmt
fmk

Pk−1

k−t−1 2(∇ f ∗ ∇ f ∗ /f ∗ f ∗ )P ∗ ((X , X )
µ mt µ mk mt mk ϑM
t
k
t=−m+1 %m
0

26

∈

k

2 |Y ). Similarly to (32), define the elements of the generalized score as
Jm
0



m
sm
λµβ %m k sλµσ %m k

∗



∗
 m

m
m
 := ∇θθ0 fmk Pϑ∗ (Xk ∈ Jm |Yk0 )
s
s
s
∗
 λβµ %m k λββ %m k λβσ %m k 
M0
fmk
m
m
sm
s
s
λσµ %m k
λβσ %m k
λσσ %m k


k−1
∗
∗
∗
∗
X
∇θ fmk
∇θ0 fmt
k
2
k−t ∇θ fmt ∇θ0 fmk
+
+
Pϑ∗M ((Xt , Xk ) ∈ Jm
|Y0 ).
%m
∗
∗
∗
∗
0
fmt
fmk
fmk
fmt

(56)

t=1

Similarly to (33), define s̃%̃k as in (52) by redefining sm
λ%m k in (52) as

0
m (% )/2 2sm
m
m
0 2(sm
0 V (sm
0 .
sm
:=
ζ
2s
(s
)
)
)
λ%m k
k,0 m
λµσ %m k
λσσ %m k
λβµ %m k
λβσ %m k
λββ %h k

(57)

m
m
m
m
m
as in (54) with sm
and Zλ%
Define Iλ.η%
λ%m k defined in (57). Let Zλ0 and Iλ.η0 denote Zλ%m
m
m
m
and Iλ.η%
evaluated at %m = 0.
m

Define Λ1λ as in (35), and define Λ2λ%m as in (35) by re-

m
m1
and t̃m2
Similar to (36), define t̃m1
λ%m by rλ (t̃λ ) = inf tλ ∈Λ1 rλ (tλ ) and
λ

placing % with %m .

λ

rλ%m (t̃m2
λ%m ) = inf tλ ∈Λ2

λ%m

m (t ), where r m (t ) := (t − Z m )0 I m (t − Z m ) and r m (t ) :=
rλ%
λ
λ
λ%m λ
λ0
λ.η0 λ
λ0
λ λ
m

m )0 I m
m
(tλ − Zλ%
λ.η%m (tλ − Zλ%m ).
m

The following proposition establishes the asymptotic null distribution of the LRTS. As in the
non-normal case, the LRTS is asymptotically distributed as the maximum of the M0 random
variables.
Assumption 11. Assumption 10 holds when s̃%̃m k is given in (57).
Proposition 15. Suppose Assumptions 1, 2, 4, 9, and 11 hold and the component density of the
d

j-th regime is given by (22). Then, under H0 : m = M0 , LRM0 ,n → maxm=1,...,M0 {max{I{%m =
m2
0 m
m1
0 m
(t̃m2
0}(t̃m1
λ%m ) Iλ.η%m t̃λ%m }}.
λ ) Iλ.η0 t̃λ , sup%m ∈Θm
%

7.3

Homoscedastic normal distribution

As in Section 6.3, we assume that Yk ∈ R in the j-th regime follows a normal distribution with the
regime-specific intercept and common variance whose density is given by (37).
The asymptotic distribution of the LRTS is derived by using a reparameterization


θm





νθm + (1 − αm )λm




 

νθm − αm λm
 θm+1  = 
,
2
2
σ
νσm − αm (1 − αm )λµm
similar to (38) and following the derivation in Sections 6.3 and 7.2. For brevity, we omit the details
m
of the derivation. Define sm
λλ%m k as in (53), and denote each element of sλλ%m k as

sm
λλ%m k =

∗

sm
λµβ %m k

m
sm
λβµ %m k sλββ %m k

27

!
.

Similarly to (43), define s̃%̃k as in (52) by redefining sm
λ%m k in (52) as

0
m
m
m
m
0 V (sm
0
sm
,
λ%m k := ζk,0 (%m )/2 sλ3µ k /3! sλ4µ k /4! (sλβµ %k )
λββ %k )
where sm
:= Pϑ∗
λi k

k
M0 (Xk ∈Jm |Y 0 )

µ

(58)

∗ )/f (Y |Y
∗ ∗
∇µi f (Yk |Yk−1 ; γ ∗ , θm
k
k−1 ; γ , θm ) for i = 3, 4.

The following proposition establishes the asymptotic null distribution of the LRTS.
Assumption 12. Assumption 10 holds when s̃%̃m k is given in (58).
Proposition 16. Suppose Assumptions 1, 2, 4, 9, and 12 hold and the component density of the
d

j-th regime is given by (37). Then, under H0 : m = M0 , LRM0 ,n → maxm=1,...,M0 {max{I{%m =
0 m
m1
m2 0 m
m2
m1
m2
0}(t̃m1
λ ) Iλ.η0 t̃λ , sup%m ∈Θ%m  (t̃λ%m ) Iλ.η%m t̃λ%m }}, where t̃λ and t̃λ%m are defined as in Proposition
m , Im
m
m
m
1
2
15 but in terms of (Zλ%
λ.η%m , Zλ0 , Iλ.η0 ) constructed with sλ%m k given in (58) and Λλ and Λλ%m
m

defined as in (44) but replacing % with %m .

8

Asymptotic distribution under local alternatives

In this section, we derive the asymptotic distribution of our LRTS under local alternatives. While
we focus on the case of testing H0 : M = 1 against HA : M = 2, it is straightforward to extend the
analysis to the case of testing H0 : M = M0 against HA : M = M0 + 1 for M0 ≥ 2.
√
Given π ∈ Θπ , we define a local parameter h := nt(ψ, π), so that
h=

hη

!

hλ

=

!
√
n(η − η ∗ )
,
√
ntλ (λ, π)

where tλ (λ, π) differs across the different models and is given by (18), (31), and (42). Given
h = (h0η , h0λ )0 and π ∈ Θπ , we consider the sequence of contiguous local alternatives ϑn = (ψn0 , πn0 )0 =
(ηn0 , λ0n , πn0 )0 ∈ Θη × Θλ × Θπ such that
hη =

√

n(ηn − η ∗ ),

hλ =

√

and πn − π = o(1).

ntλ (λn , πn ) + o(1),

(59)

Let Pnϑ,x0 be the probability measure on {Yk }nk=1 under ϑ conditional on the value of Y0 , X0 ,
and W1n . Then, the log-likelihood ratio is given by
log

dPnϑn ,x0
dPnϑ∗ ,x0

P
= `n (ψn , πn , x0 ) − `n (ψ ∗ , π, x0 ) = log

xn
1

Qn

k=1 fk (ηn , λn )qπn (xk−1 , xk )
Qn
∗
k=1 fk (η , 0)

!
,

where fk (η, λ) is defined by the right-hand side of (9), (24), and (39) for the models of the nonnormal distribution, heteroscedastic normal distribution, and homoscedastic normal distribution,
respectively. The following result follows from Le Cam’s first and third lemmas and facilitates the
derivation of the asymptotic distribution of the LRTS under Pnϑn ,x0 .
28

Proposition 17. Suppose that the assumptions of Propositions 7, 9, and 11 hold for the models
of the non-normal, heteroscedastic normal, and homoscedastic normal distributions, respectively.
Then, uniformly in x0 ∈ X , (a) Pnϑn ,x0 is mutually contiguous with respect to Pnϑ∗ ,x0 , and (b) under
d

Pnϑn ,x0 , we have log(dPnϑn ,x0 /dPnϑ∗ ,x0 ) = h0 νn (s%n k ) − 21 h0 I% h + op (1) with νn (s%n k ) → N (I% h, I% ).

8.1

Non-normal distribution

For the non-normal distribution, the sequence of contiguous local alternatives is given by λn =
√
λ̄/n1/4 because then hλ = nα(1−α)v(λn ) = α(1−α)v(λ̄) holds. The following proposition derives
the asymptotic distribution of the LRTS for the non-normal distribution under H1n : (πn , ηn , λn ) =
(π̄, η ∗ , λ̄/n1/4 ).
Proposition 18. Suppose that the assumptions of Proposition 8 hold.
λ̄ 6= 0, define hλ := ᾱ(1 − ᾱ)v(λ̄). Then, under H1n : (πn , ηn , λn ) =
d

LRn →
(Iλ.η%

8.2

sup%∈Θ% (t̃λ%h )0 Iλ.η% t̃λ%h ,

)−1 G

λ.η%

For π̄ ∈ Θπ and

(π̄, η ∗ , λ̄/n1/4 ),

we have

where t̃λ%h is defined as in (21) but replacing Zλ% in (21) with

+ hλ .

Heteroscedastic normal distribution

For the model with the heteroscedastic normal distribution, the sequences of contiguous local
alternatives characterized by (59) include the local alternatives of order n−1/8 .
Proposition 19. Suppose that the assumptions of Proposition 10 hold for model (22). For %̄ ∈
(−1, 1), ᾱ ∈ (0, 1), and λ̄ := (λ̄µ , λ̄σ , λ̄0β )0 6= (0, 0, 0)0 , let
a
H1n
: (%n , αn , ηn , λµn , λσn , λβn ) = (%̄/n1/4 , ᾱ, η ∗ , λ̄µ /n1/8 , λ̄σ /n3/8 , λ̄β /n3/8 ),
b
H1n
: (%n , αn , ηn , λµn , λσn , λβn ) = (%̄, ᾱ, η ∗ , λ̄µ /n1/4 , λ̄σ /n1/4 , λ̄β /n1/4 ),

and define
haλ : = ᾱ(1 − ᾱ) × (%̄λ̄2µ , λ̄µ λ̄σ , b(ᾱ)λ̄4µ /12, λ̄0β λ̄µ , 0, 0)0 ,
hbλ : = ᾱ(1 − ᾱ) × (%̄λ̄2µ , λ̄µ λ̄σ , λ̄2σ , λ̄0β λ̄µ , λ̄0β λ̄σ , v(λ̄β )0 )0 .
d

j
1j
2j 0
2j
0
Then, for j ∈ {a, b}, under H1n
, we have LRn → max{I{% = 0}(t̃1j
λh ) Iλ.η0 t̃λh , sup%∈Θ% (t̃λ%h ) Iλ.η% t̃λ%h },
2j
j
−1
where t̃1j
λh and t̃λ%h are defined as in (36) but replacing Zλ% with (Iλ.η% ) Gλ.η% + hλ .
a , % converges to 0, and λ
In the local alternative H1n
n
µn converges to 0 at a slower rate than

n−1/4 . Our test has non-trivial power against these local alternatives in the neighborhood of % = 0.
By contrast, the test of Carrasco et al. (2014) does not have power against the local alternatives in
the neighborhood of % = 0, as discussed in Section 5 of Carrasco et al. (2014). The test proposed
by Qu and Zhuo (2017) assumes that % is bounded away from zero and hence their test rules out
a .
H1n

29

8.3

Homoscedastic normal distribution

The local alternatives for the model with the homoscedastic distribution also include those of order
n−1/8 in the neighborhood of % = 0.
Proposition 20. Suppose that the assumptions of Proposition 11 hold for model (37). For %̄ ∈
(−1, 1), ᾱ ∈ (0, 1), ∆α 6= 0, and λ̄ := (λ̄µ , λ̄0β )0 6= (0, 0)0 , let
a
H1n
: (%n , αn , ηn , λµn , λβn ) = (%̄/n1/4 , 1/2 + ∆α /n1/8 , η ∗ , λ̄µ /n1/8 , λ̄β /n3/8 ),
b
H1n
: (%n , αn , ηn , λµn , λβn ) = (%̄, ᾱ, η ∗ , λ̄µ /n1/4 , λ̄β /n1/4 ),

and define haλ := (1/4)×(%̄λ̄2µ , ∆α λ̄3µ , −λ̄4µ /2, λ̄0β λ̄µ , 0)0 and hbλ := ᾱ(1− ᾱ)×(%̄λ̄2µ , 0, 0, λ̄0β λ̄µ , v(λ̄β )0 )0 .
2j
j
−1
For j = {a, b}, define t̃1j
λh and t̃λ%h as in (36) but replacing Zλ% with (Iλ.η% ) Gλ.η% + hλ , where

Iλ.η% and Gλ.η% are constructed with s%k defined in (43), and Λ1λ and Λ2λ% are defined in (44). Then,
d

2j
2j 0
1j
j
0
, we have LRn → max{I{% = 0}(t̃1j
under H1n
λh ) Iλ.η0 t̃λh , sup%∈Θ% (t̃λ%h ) Iλ.η% t̃λ%h }.

9

Parametric bootstrap

We consider the following parametric bootstrap to obtain the bootstrap critical value cα,B and
bootstrap p-value of our LRTS for testing H0 : M = M0 against HA : M = M0 + 1.
1. Using the observed data, compute ϑ̂M0 , ϑ̂M0 +1 , and LRM0 ,n .
2. Given ϑ̂M0 and ξM0 , generate B independent samples {Y1b , . . . , Ynb }B
b=1 under H0 with ϑM0 =
ϑ̂M0 conditional on the observed value of Y0 and W1n .
3. For each simulated sample {Ykb }nk=1 with (Y0 , W1n ), estimate ϑ̂bM0 and ϑ̂bM0 +1 as in Step 1,
b
and let LRM
:= 2[`n (ϑ̂bM0 +1 , ξM0 +1 ) − `n (ϑ̂bM0 , ξM0 )] for b = 1, . . . , B.
0 ,n
b
4. Let cα,B be the (1 − α) quantile of {LRM
}B , and define the bootstrap p-value as
0 ,n b=1
P
B
b
B −1 b=1 I{LRM
> LRM0 ,n }.
0 ,n

The following proposition shows the consistency of the bootstrap critical values cα,B for testing
H0 : M0 = 1. We omit the result for testing H0 : M0 ≥ 2; it is straightforward to extend the
analysis to the case for M0 ≥ 2 with more tedious notations.
Proposition 21. Suppose that the assumptions of Propositions 7, 9, and 11 hold for the models
of the non-normal, heteroscedastic normal, and homoscedastic normal distributions, respectively.
Then, the bootstrap critical values cα,B converge to the asymptotic critical values in probability as
n and B go to infinity under H0 and under the local alternatives described in Propositions 18, 19,
and 20.

30

10

Simulations and empirical application

10.1

Simulations

We consider the following two models:
Model 1 : Yk = µXk + βYk−1 + εk ,
Model 2 : Yk = µXk + βYk−1 + εk ,

εk ∼ N (0, σ 2 ),
εk ∼

2
N (0, σX
),
k

(60)
(61)

where Xk ∈ {1, . . . , M } with pij = p(Xk = i|Xk−1 = j). Model 1 in (60) is similar to the model
used in Cho and White (2007). This model has a switching intercept but the variance parameter σ 2
does not switch across regimes. In Model 2 in (61), both the intercept and the variance parameters
switch across regimes.
We compare the size and power property of our bootstrap LRT and that of the QLR test
of Cho and White (2007) with Θµ = [−2, 2] and the supTS test of Carrasco et al. (2014) with
ρ ∈ [−0.9.0.9]. The critical values are computed by bootstrap with B = 199 bootstrap samples.
Note that this comparison favors the LRT over the supTS test because the supTS test is designed
to detect general parameter variation including Markov chain.
In Model 2, we set the lower bound of σ as σ = 0.01σ̂, where σ̂ is the estimate of σ from
one-regime model. We also find that adding a penalty term to the log-likelihood function improves
the finite sample property of the LRT. The penalty term prevents σj from taking an extremely
P
2
2
2
2
−1/2 and
small value and takes the form −an M
j=1 {σ̂ /σj + log(σj /σ̂ ) − 1}. We set an = 20n
compute the test statistic using the log-likelihood function without the penalty term. Because the
penalty term and its derivatives are op (1) from the compactness of ΘM , adding this penalty term
does not affect the consistency of the MLE or the asymptotic distribution of the LRTS.
We first examine the rejection frequency of H0 : M = 1 against HA : M = 2 when the data are
generated by H0 : M = 1 with (β, µ, σ) = (0.5, 0, 1). The first panel in Table 1 reports the rejection
frequency of the bootstrap tests at the nominal 10%, 5%, and 1% levels over 3000 replications with
n = 200 and 500. Overall, all the tests have good sizes.
Table 2 reports the power of the three tests for testing the null hypothesis of M = 1 at the
nominal level of 5%. We generate 3000 data sets for n = 500 under the alternative hypothesis of
M = 2 by setting µ1 = 0.2, 0.6, and 1.0 and µ2 = −µ1 , while (p11 , p22 ) = (0.25, 0.25), (0.50, 0.50),
(0.70, 0.70), and (0.90, 0.90). We set σ = 1 for Model 1 and (σ1 , σ2 ) = (1.1, 0.9) for Model 2.
For Model 1, the power of all the tests increases as µ1 increases except for the supTS test with
(p11 , p22 ) = (0.9, 0.9). As (p11 , p22 ) moves away from (0.5, 0.5), the power of the LRT increases,
whereas the QLRT has decreasing power. The LRT performs better than the supTS and QLR tests
except for the case with (p11 , p22 ) = (0.25, 0.25), where the supTS test performs very well, and the
case with (p11 , p22 ) = (0.5, 0.5), where the QLRT outperforms the LRT, because the true model is
a finite mixture in this case. The last three columns of Table 2 report the power of the LRT and
supTS test to detect alternative models with switching variances (i.e., Model 2 with M = 2). We
31

did not examine the power of the QLRT because this test assumes non-switching variance. The
LRT has stronger power than the supTS test in most cases.
The second panel in Table 1 reports the rejection frequency of the LRT for testing H0 : M = 2
against HA : M = 3 when the data are generated under the null hypothesis, showing its good size
property, while neither the QLRT nor the supTS test is applicable for testing H0 : M = 2 against
HA : M = 3.
Table 3 reports the power of our LRT for testing the null hypothesis of M = 2 at the nominal
level of 5%. We generate 3000 data sets for n = 500 under the alternative hypothesis of M = 3
across different values of (µ1 , µ2 , µ3 ) and (p11 , p22 , p33 ) with pij = (1 − pii )/2 for j 6= i, where we
set (β, σ) = (0.5, 1.0) for Model 1 and (β, σ1 , σ2 ) = (0.5, 0.9, 1.2) for Model 2. Similar to the case
of H0 : M = 1, the power of the LRT for testing H0 : M = 2 against HA : M = 3 increases when
the alternative is further away from H0 or when latent regimes become more persistent.

10.2

Empirical example

Using U.S. GDP per capita quarterly growth rate data from 1960Q1 to 2014Q4, we estimate the
regime switching models with common variance (i.e., Model 1 in (60)) and with switching variances
(i.e., Model 2 in (61)) for M = 1, 2, 3, and 4 and sequentially test the null hypothesis of M = M0
against the alternative hypothesis M = M0 + 1 for M0 = 1, 2, 3, and 4.9 We also report the Akaike
information criteria (AIC) and Bayesian information criteria (BIC) for reference, although, to our
best knowledge, the consistency of the AIC and BIC for selecting the number of regimes has not
been established in the literature.
Table 5 reports the selected number of regimes by the AIC, BIC, and LRT. For Model (60)
with common variance, our LRT selects M = 4, while the AIC and BIC select M = 3 and M = 1,
respectively. For Model (61) with switching variance, both the LRT and the AIC select M = 3,
while the BIC selects M = 2.
Panel A of Table 4 and Figure 1 report the parameter estimates and posterior probabilities of
being in each regime for the model with common variance for M = 2, 3, and 4. Across the different
specifications in M , the estimated values of µ1 , µ2 , . . . , µM are well separated in the common
variance model, indicating that each regime represents a booming or recession period with different
degrees. In Figure 1, when the number of regimes is specified as M = 2, the posterior probability of
the “recession” regime (Regime 1) against that of the “booming” regime (Regime 2) sharply rises
during the collapse of Lehman Brothers in 2008 and then declines after 2009. When the number of
regimes is specified as M = 3, in addition to the “recession” and “booming” regimes corresponding
to Regimes 1 and 2, respectively, the regime with a rapid change in the growth rates from low to
high is captured by Regime 3; for the model with M = 3 in Figure 1, the posterior probability of
Regime 3 rises in late 2009 when the U.S. economy started to recover from the Lehman collapse.
9
For both models, we restrict the parameter values for the transition probabilities by setting  = 0.05 to prevent
the issue of unbounded likelihood.

32

When the number of regimes is specified as M = 4, Regime 1 now captures a rapid change in the
growth rates from high to low, where the posterior probability of Regime 1 becomes high when the
growth rate of the U.S. economy rapidly declined in the middle of the Lehman collapse. The LRT
selects the model with four regimes, which capture the rapid changes in the growth rates of U.S.
GDP per capita during the Lehman collapse period.
Panel B of Table 4 and Figure 2 report the parameter estimates and posterior probabilities of
being in each regime for the model with switching variance, respectively. When the number of
regimes is specified as M = 2, the variance parameter estimates are very different between the two
regimes, while the intercept estimates are similar, indicating that Regime 1 is the “low volatility”
regime, while Regime 2 is the “high volatility” regime. When the number of regimes is specified
as M = 3, different regimes capture different states of the U.S. economy in terms of both growth
rates and volatilities.10 Regime 1 is characterized by the negative value of the intercept with high
volatility, capturing a recession period. Regime 2 is characterized by the positive value of the
intercept with low volatility, capturing a booming/stable economy. Regime 3 is characterized by a
high value of the intercept and high variance, capturing both a rapid recovery in the growth rates
and high volatility in the aftermath of the Lehman collapse in 2009. The LRT selects the model
with three regimes when the model is specified with switching variance.

11

Appendix

Henceforth, for notational brevity, we suppress Wab from the conditioning variables and conditional
densities when doing so does not cause confusion.

11.1

Proof of propositions and corollaries

Proof of Proposition 1. The proof is essentially identical to the proof of Lemma 2 in DMR. Therefore, the details are omitted. The only difference from DMR is (i) we do not impose Assumption
(A2) of DMR, but this does not affect the proof because Assumption (A2) is not used in the proof
of Lemma 2 in DMR, and (ii) we have W1n , but our Lemma 11(a) extends Corollary 1 of DMR to
accommodate the Wk ’s. Consequently, the argument of the proof of DMR goes through.
Proof of Proposition 2. Define hϑkx0 :=

p
lϑkx0 −1. By using the Taylor expansion of 2 log(1+x) =

2x − x2 (1 + o(1)) for small x, we have, uniformly in x0 ∈ X and ϑ ∈ Nc/√n ,
∗

`n (ψ, π, x0 ) − `n (ψ , π, x0 ) = 2

n
X

log(1 + hϑkx0 ) = nPn (2hϑkx0 − [1 + op (1)]h2ϑkx0 ).

(62)

k=1
10
We may test the null hypothesis of σ1 = σ2 = σ3 in the model with switching variance given M = 3 by the
standard LRT with the critical value obtained from the chi-square distribution with two degrees of freedom. With
LRT S = 2 × (−297.01 + 307.39) = 20.76, the null hypothesis of σ1 = σ2 = σ3 is rejected at the 1% significance level,
suggesting that the model with switching variance is more appropriate than the model with common variance.

33

The stated result holds if we show that
sup

nPn (h2ϑkx0 ) − nt0ϑ Iπ tϑ /4 = op (1)

sup

x0 ∈X ϑ∈Nc/√n

sup

sup

x0 ∈X

ϑ∈Nc/√n

|nPn (hϑkx0 ) −

√

and

(63)

nt0ϑ νn (sπk )/2 + ntϑ Iπ t0ϑ /8| = op (1),
√

because then the right-hand side of (62) is equal to

(64)

nt0ϑ νn (sπk ) − tϑ Iπ t0ϑ /2 + op (1) uniformly in

x0 ∈ X and ϑ ∈ Nc/√n .
We first show (63). Let mϑk := t0ϑ sπk + rϑk , so that lϑkx0 − 1 = mϑk + uϑkx0 . Observe that
max

sup

1≤k≤n ϑ∈N

√
c/ n

|mϑk | = max

sup

1≤k≤n ϑ∈N

√
c/ n

|t0ϑ sπk + rϑk | = op (1),

(65)

from Assumptions 3(a) and (c) and Lemma 10. Write 4Pn (h2ϑkx0 ) as
4Pn (h2ϑkx0 ) = Pn

4(lϑkx0 − 1)2
p
( lϑkx0 + 1)2

!
= Pn (lϑkx0 − 1)2 − Pn

(lϑkx0

!
p
(
l
+
3)
ϑkx0
− 1)3 p
.
( lϑkx0 + 1)3

(66)

It follows from Assumptions 3(a)(b)(c)(e)(f) and (E|XY |)2 ≤ E|X|2 E|Y |2 that, uniformly in ϑ ∈
Nε ,
Pn (lϑkx0 −1)2 = t0ϑ Pn (sπk s0πk )tϑ +2t0ϑ Pn [sπk (rϑk +uϑkx0 )]+Pn (rϑk +uϑkx0 )2 = t0ϑ Pn (sπk s0πk )tϑ +ζϑnx0 ,
(67)
where ζϑnx0 satisfies supx0 ∈X |ζϑnx0 | = Op (|tϑ

|2 |ψ − ψ ∗ |) + O

(n−1 |t

∗
−1
∗ 2
ϑ ||ψ − ψ |) + Op (n |ψ − ψ | ).

p
0
Then, (63) holds because supπ∈Θπ |Pn (sπk sπk ) − Iπ | = op (1) and the second term on the right-hand
side of (66) is bounded by, from (65), Pn (m2ϑk ) = t0ϑ Iπ tϑ + op (|tϑ |2 ), and Assumption 3(e),

C sup

sup

x0 ∈X ϑ∈Nc/√n

≤ op (1) sup



Pn |mϑk |3 + 3m2ϑk |uϑkx0 | + 3|mϑk |u2ϑkx0 + C sup

sup

x0 ∈X ϑ∈Nc/√n

sup

x0 ∈X ϑ∈Nc/√n



Pn m2ϑk + u2ϑkx0 + C sup

sup

x0 ∈X ϑ∈Nc/√n

Pn (|uϑkx0 |3 )

Pn (|uϑkx0 |3 ) = op (n−1 ).

We proceed to show (64). Consider the following expansion of hϑkx0 :
hϑkx0 = (lϑkx0 − 1)/2 − h2ϑkx0 /2 = (t0ϑ sπk + rϑk + uϑkx0 )/2 − h2ϑkx0 /2.

(68)

Then, (64) follows from (63), (68), and Assumptions 3(d) and (e), and the stated result follows.
Proof of Proposition 3. For part (a), it follows from log(1 + x) ≤ x and hϑkx0 = (lϑkx0 − 1)/2 −

34

h2ϑkx0 /2 (see (68)) that
∗

`n (ψ, π, x0 ) − `n (ψ , π, x0 ) = 2

n
X

log(1 + hϑkx0 ) ≤ 2nPn (hϑkx0 ) =

√

nνn (lϑkx0 − 1) − nPn (h2ϑkx0 ).

k=1

Observe that h2ϑkx0

(69)
p
√
= (lϑkx0 − 1)2 /( lϑkx0 + 1)2 ≥ I{lϑkx0 ≤ κ}(lϑkx0 − 1)2 /( κ + 1)2 for any κ > 0.

Therefore,


√
Pn (h2ϑkx0 ) ≥ ( κ + 1)−2 Pn I{lϑkx0 ≤ κ}(lϑkx0 − 1)2 .

(70)

Substituting (67) into the right-hand side of (70) gives


√
Pn (h2ϑkx0 ) ≥ ( κ + 1)−2 t0ϑ Pn (sπk s0πk ) − Pn (I{lϑkx0 > κ}sπk s0πk ) tϑ + ζϑnx0 .

(71)

From Hölder’s inequality, we have Pn (I{lϑkx0 > κ}|sπk |2 ) ≤ [Pn (I{lϑkx0 > κ})]δ/(2+δ) [Pn (|sπk |2+δ )]2/(2+δ) .
The right-hand side is no larger than κ−δ/(2+δ) Op (1) uniformly in x0 ∈ X and ϑ ∈ Nε because (i) it follows from κI{lϑkx0 > κ} ≤ lϑkx0 that Pn (I{lϑkx0 > κ}) ≤ κ−1 Pn (lϑkx0 ) and
supx0 ∈X supϑ∈Nε |Pn (lϑkx0 )−1| = op (1) from Assumptions 3(d)–(g), and (ii) Pn (supπ∈Θπ |sπk |2+δ ) =
Op (1) from Assumption 3(a).

Consequently, P(supx0 ∈X supϑ∈Nε Pn (I{lϑkx0 > κ}|sπk |2 ) ≥

λmin /4) → 0 as κ → ∞, and hence we can write (71) as Pn (h2ϑkx0 ) ≥ τ (1 + op (1))t0ϑ Iπ tϑ +
√
Op (|tϑ |2 |ψ − ψ ∗ |) + Op (n−1 ) for τ := ( κ + 1)−2 /2 > 0 by taking κ sufficiently large. Because
√
√
nνn (lϑkx0 − 1) = nt0ϑ νn (sπk ) + Op (1) from Assumptions 3(d) and (e), it follows from (69) that,
uniformly in x0 ∈ X and ϑ ∈ Nε ,
− η ≤ `n (ψ, π, x0 ) − `n (ψ ∗ , π, x0 ) ≤
Let Tn :=

1/2 √
Iπ
ntϑ .

√

nt0ϑ νn (sπk ) − τ (1 + op (1))nt0ϑ Iπ tϑ + Op (n|tϑ |2 |ψ − ψ ∗ |) + Op (1).
(72)

From (72), Assumptions 3(b) and (g), and the fact ψ

− ψ∗

→ 0 if tϑ → 0, we

obtain the following result: for any δ > 0, there exist ε > 0 and M, n0 < ∞ such that

P

inf

inf

x0 ∈X ϑ∈Nε


|Tn |M − (τ /2)|Tn |2 + M ≥ 0


≥ 1 − δ, for all n > n0 .

(73)

Rearranging the terms inside P(·) gives supx0 ∈X supϑ∈Nε (|Tn |−(M/τ ))2 ≤ 2M/τ +(M/τ )2 . Taking
its square root gives P(supx0 ∈X supϑ∈Nε |Tn | ≤ M1 ) ≥ 1 − δ for a constant M1 , and part (a) follows.
Part (b) follows from part (a) and Proposition 2.
Proof of Corollary 1. Because the logarithm is monotone, we have inf x0 ∈X `n (ψ, π, x0 )

≤

`n (ψ, π, ξ) ≤ supx0 ∈X `n (ψ, π, x0 ). Part (a) then follows from Proposition 2. For part (b), note
that we have ϑ ∈ Anε (ξ, η) only if ϑ ∈ Anε (x0 , η) for some x0 . Consequently, part (b) follows from
Proposition 3.
Proof of Proposition 4. First, observe that parts (a) and (b) hold when the right-hand side is
replaced with Kj (k + m)7 ρb(k+m−1)/24c and Kj (k + m)7 ρb(k+m−1)/1340c by using Lemmas 2 and
35

4 and noting that q1 = 6q0 , q2 = 5q0 , q3 = 4q0 , . . . , q6 = q0 . For example, when j = 2, we can
bound supx∈X supϑ∈N ∗ |∇2 `k,m,x (ϑ) − ∇j 2`k,m (ϑ)| from ∇2 `k,m,x (ϑ) = ∆21,k,m,x (ϑ) + ∆1,1
2,k,m,x (ϑ),
I(j)

I(j)

supx∈X supϑ∈N ∗ |∆j,k,m,x (ϑ) − ∆j,k,m (ϑ)| ≤ KI(j) (k + m)7 ρb(k+m−1)/24c , KI(j) ∈ LrI(j) (Pϑ∗ ), r(2) =
q2 = 5q0 , and r(1,1) = q1 /2 = 3q0 . Second, letting ρ∗ = ρ1/1340 I{ρ > 0} and redefining Kj gives
parts (a) and (b). Parts (c) and (d) follow from Lemmas 2 and 4.
Proof of Proposition 5. First, we prove part (a). The proof of part (b) is essentially the same as
that of part (a) and hence omitted. Observe that
k−1

j
(ϑ) −
∇lk,m,x

j
∇lk,m (ϑ)

where

= Ψjk,m,x (ϑ)

:=

!

−
k−1
k−1
pϑ∗ (Yk |Y−m , X−m = x) pϑ∗ (Yk |Y−m )
k−1

pϑ (Yk |Y−m )  j
j
+
Ψ
(ϑ)
−
Ψ
(ϑ)
,
k,m
k,m,x
k−1
pϑ∗ (Yk |Y−m )
k−1

Ψjk,m,x (ϑ)

k−1

pϑ (Yk |Y−m )

pϑ (Yk |Y−m , X−m = x)

∇j pϑ (Yk |Y−m , X−m = x)
k−1

k−1

,

pϑ (Yk |Y−m , X−m = x)

j
Ψk,m (ϑ)

:=

∇j pϑ (Yk |Y−m )
k−1

.

pϑ (Yk |Y−m )

In view of Lemma 5 and Hölder’s inequality, part (a) holds if, for j = 1, . . . , 6, there exist random
variables ({Aj,k }nk=1 , Bj ) ∈ Lq0 (Pϑ∗ ) and ρ∗ ∈ (0, 1) such that, for all 1 ≤ k ≤ n and m ≥ 0,
(A)

sup sup sup |Ψjk,m,x (ϑ)| ≤ Aj,k ,

m≥0 x∈X

ϑ∈N ∗

(B)

j

sup sup |Ψjk,m,x (ϑ)−Ψk,m (ϑ)| ≤ Bj (k+m)7 ρk+m−1
.
∗

x∈X

ϑ∈N ∗

(74)
We show (A) and (B). From (91) we have, suppressing (ϑ) and superscript 1 from ∇1 `k,m,x ,
Ψ1k,m,x = ∇`k,m,x ,

Ψ2k,m,x = ∇2 `k,m,x + (∇`k,m,x )2 ,

Ψ3k,m,x = ∇3 `k,m,x + 3∇2 `k,m,x ∇`k,m,x + (∇`k,m,x )3 ,
Ψ4k,m,x = ∇4 `k,m,x + 4∇3 `k,m,x ∇`k,m,x + 3(∇2 `k,m,x )2 + 6∇2 `k,m,x (∇`k,m,x )2 + (∇`k,m,x )4 ,
Ψ5k,m,x = ∇5 `k,m,x + 5∇4 `k,m,x ∇`k,m,x + 10∇3 `k,m,x ∇2 `k,m,x + 10∇3 `k,m,x (∇`k,m,x )2
+ 15(∇2 `k,m,x )2 ∇`k,m,x + 10∇2 `k,m,x (∇`k,m,x )3 + (∇`k,m,x )5 ,
Ψ6k,m,x = ∇6 `k,m,x + 6∇5 `k,m,x ∇`k,m,x + 15∇4 `k,m,x ∇2 `k,m,x + 15∇4 `k,m,x (∇`k,m,x )2
+ 10(∇3 `k,m,x )2 + 60∇3 `k,m,x ∇2 `k,m,x ∇`k,m,x + 20∇3 `k,m,x (∇`k,m,x )3
+ 15(∇2 `k,m,x )3 + 45(∇2 `k,m,x )2 (∇`k,m,x )2 + 15∇2 `k,m,x (∇`k,m,x )4 + (∇`k,m,x )6 ,
j

and Ψk,m is written analogously with ∇j `k,m replacing ∇j `k,m,x . Therefore, (A) of (74) follows
from Proposition 4(c) and Hölder’s inequality. (B) of (74) follows from Proposition 4(a)(c), the
P
n−1−i bi ), and Hölder’s inequality.
relation ab − cd = a(b − c) − c(a − d), an − bn = (a − b) n−1
i=0 (a
k−1

j
j
For part (c), the bound of ∇lk,m,x
(ϑ) follows from writing ∇lk,m,x
(ϑ) = [pϑ (Yk |Y−m , X−m =
k−1

j

x)/pϑ∗ (Yk |Y−m , X−m = x)]Ψjk,m,x (ϑ) and using (74) and Lemma 5. ∇ lk,m (ϑ) is bounded by
36

a similar argument. Part (d) follows from parts (a)–(c), the completeness of Lq (Pϑ∗ ), Markov’s
inequality, and the Borel–Cantelli Lemma. Part (e) follows from combining parts (a) and (b) and
letting m0 → ∞ in part (b).
Proof of Proposition 6. The consistency of ϑ̂1 follows from Theorem 2.1 of Newey and McFadden (1994) because (i) ϑ∗1 uniquely maximizes Eϑ∗1 log f (Y1 |Y0 , W1 ; γ, θ) from Assumption 5(c),
p

and (ii) supϑ1 ∈Θ1 |n−1 `0,n (ϑ1 ) − Eϑ∗1 log f (Y1 |Y0 , W1 ; γ, θ)| → 0 and Eϑ∗1 log f (Y1 |Y0 , W1 ; γ, θ)
is continuous because (Yk , Wk ) is strictly stationary and ergodic from Assumption 1(e) and
Eϑ∗1 supϑ1 ∈Θ1 | log f (Y1 |Y0 , W1 ; γ, θ)| < ∞ from Assumption 2(c).
We proceed to prove the consistency of ϑ̂2 .
∆k,m,x (ϑ2 )

:=

k−1
k ,X
log pϑ2 (Yk |Y−m , W−m
−m

Define, similarly to pp. 2265–6 in DMR,

=

x), ∆k,m (ϑ2 )

:=

k−1

k ),
log pϑ2 (Yk |Y−m , W−m

∆k,∞ (ϑ2 ) := limm→∞ ∆k,m (ϑ2 ), and `(ϑ2 ) := Eϑ∗1 [∆0,∞ (ϑ)]. Observe that Lemmas 3 and 4 as
well as Proposition 2 of DMR hold for our {∆k,m,x (ϑ2 ), ∆k,m (ϑ2 ), ∆k,∞ (ϑ2 ), `n (ϑ2 , x0 ), `(ϑ2 )} under our assumptions because (i) our Assumption 1(e) can replace their Assumption (A2) in the
proof of their Lemmas 3 and 4 and Proposition 2, and (ii) our Lemma 11(a) extends Corollary 1
of DMR to accommodate the Wk ’s. It follows that (i) `(ϑ2 ) is maximized if and only if ϑ2 ∈ Γ∗
0

1 )] converges to `(ϑ ) uniformly in ϑ as
from Assumption 5(d) because Eϑ∗1 [log pϑ2 (Y1 |Y−m , W−m
2
2

m → ∞ from Lemma 3 of DMR and the dominated convergence theorem, (ii) `(ϑ2 ) is continuous
p

from Lemma 4 of DMR, and (iii) supξ2 supϑ2 ∈Θ2 |n−1 `n (ϑ2 , ξ2 ) − `(ϑ2 )| → 0 holds from Proposition
p

2 of DMR and `n (ϑ2 , ξ2 ) ∈ [minx0 `n (ϑ2 , x0 ), maxx0 `n (ϑ2 , x0 )]. Consequently, inf ϑ2 ∈Γ∗ |ϑ̂2 −ϑ2 | → 0
follows from Theorem 2.1 of Newey and McFadden (1994) with an adjustment for the fact that the
maximizer of `(ϑ2 ) is a set, not a singleton.
Proof of Proposition 7. We prove the stated result by applying Corollary 1 to lϑkx0 − 1 with lϑkx0
defined in (4). Because the first and second derivatives of lϑkx0 − 1 play the role of the score, we
expand lϑkx0 − 1 with respect to ψ up to the third order. Let q = dim(ψ). For a k × 1 vector a,
define a⊗p := a ⊗ a ⊗ · · · ⊗ a (p times) and ∇a⊗p := ∇a ⊗ ∇a ⊗ · · · ⊗ ∇a (p times). Recall that the
(p + 1)-th order Taylor expansion of f (x) with x ∈ Rq around x = x∗ is given by
p
X
1
1
∇ ⊗j 0 f (x∗ )(x − x∗ )⊗j +
∇ ⊗(p+1) )0 f (x)(x − x∗ )⊗(p+1) ,
f (x) = f (x ) +
j! (x )
(p + 1)! (x
∗

j=1

where x lies between x and x∗ , and x may differ from element to element of ∇x⊗(p+1) f (x).
Choose  > 0 sufficiently small so that N is a subset of N ∗ in Assumption 4. For m ≥ 0 and
j = 1, 2, . . ., let
k−1

Λjk,m,x−m (ψ, π) :=

∇ψ⊗j pψπ (Yk |Y−m , x−m )
k−1

j!pψ∗ π (Yk |Y−m , x−m )

k−1

,

Λjk,m (ψ, π) :=

∇ψ⊗j pψπ (Yk |Y−m )
k−1

,

j!pψ∗ π (Yk |Y−m )

and ∆ψ := ψ − ψ ∗ . With this notation, expanding lϑkx0 − 1 three times around ψ ∗ while fixing π

37

gives, with ψ ∈ [ψ, ψ ∗ ],
lϑkx0 − 1 = Λ1k,0,x0 (ψ ∗ , π)0 ∆ψ + Λ2k,0,x0 (ψ ∗ , π)0 (∆ψ)⊗2 + Λ3k,0,x0 (ψ, π)0 (∆ψ)⊗3
= Λ1k,0 (ψ ∗ , π)0 ∆ψ + Λ2k,0 (ψ ∗ , π)0 (∆ψ)⊗2 + Λ3k,0 (ψ, π)0 (∆ψ)⊗3 + ukx0 (ψ, π),
where ψ may differ from element to element of Λ3k,0,x0 (ψ, π), and ukx0 (ψ, π) :=
k−1

k−1

) = 0 and ∇λη0 pψ∗ π (Yk |Y0

j
∗
j=1 [Λk,0,x0 (ψ , π)−

P2

Λjk,0 (ψ ∗ , π)]0 (∆ψ)⊗j + [Λ3k,0,x0 (ψ, π) − Λ3k,0 (ψ, π)]0 (∆ψ)⊗3 .
Noting that ∇λ pψ∗ π (Yk |Y0

(75)

) = 0 from (13), we may rewrite (75)

as
lkϑx0 − 1 = t(ψ, π)0 s%k + rk,0 (ψ, π) + ukx0 (ψ, π),

(76)

e k,0 (π)0 (∆η)⊗2 +Λ3 (ψ, π)0 (∆ψ)⊗3 , where Λ
e k,0 (π) denotes
where s%k is defined in (17), rk,0 (ψ, π) := Λ
k,0
the part of Λ2k,0 (ψ ∗ , π) corresponding to (∆η)⊗2 .
For m ≥ 0, define vk,m (ϑ) := (Λ1k,m (ψ, π)0 , Λ2k,m (ψ, π)0 , Λ3k,m (ψ, π)0 )0 , and define vk,∞ (ϑ) :=
limm→∞ vk,m (ϑ). In order to apply Corollary 1 to lϑkx0 − 1, we first show
sup Pn [vk,0 (ϑ)vk,0 (ϑ)0 ] − Eϑ∗ [vk,∞ (ϑ)vk,∞ (ϑ)0 ] = op (1),

(77)

ϑ∈N

νn (vk,0 (ϑ)) ⇒ W (ϑ),

(78)

where W (ϑ) is a mean-zero continuous Gaussian process with Eϑ∗ [W (ϑ1 )W (ϑ2 )0 ]
Eϑ∗ [vk,∞ (ϑ1 )vk,∞ (ϑ2

)0 ].

(77) holds because supϑ∈N Pn [vk,0 (ϑ)vk,0

from Proposition 5, and vk,∞ (ϑ)vk,∞

(ϑ)0

(ϑ)0

− vk,∞ (ϑ)vk,∞

(ϑ)0 ]

=

= op (1)

satisfies a uniform law of large numbers (Lemma 2.4 and

footnote 18 of Newey and McFadden (1994)) because vk,∞ (ϑ) is continuous in ϑ from the continuity of ∇j lk,m,x (ϑ) and Proposition 5, and Eϑ∗ supϑ∈N |vk,∞ (ϑ)|2 < ∞ from Proposition 5. (78)
holds because supϑ∈N νn (vk,0 (ϑ) − vk,∞ (ϑ)) = op (1) from Proposition 5 and νn (vk,∞ (ϑ)) ⇒ W (ϑ)
from Theorem 10.2 of Pollard (1990) because (i) the space of ϑ is totally bounded, (ii) the finite
dimensional distributions of νn (vk,∞ (·)) converge to those of W (·) from a martingale CLT because
vk,∞ (ϑ) is a stationary L2 (Pϑ∗ ) martingale difference sequence for all ϑ ∈ N from Proposition 5,
and (iii) {νn (vk,∞ (·)) : n ≥ 1} is stochastically equicontinuous from Theorem 2 of Hansen (1996b)
because vk,∞ (ϑ) is Lipschitz continuous in ϑ and both vk,∞ (ϑ) and the Lipschitz coefficient are in
Lq (Pϑ∗ ) with q > dim(ϑ) from Proposition 5.
We proceed to show that the terms on the right-hand side of (76) satisfy Assumptions 3(a)–(g).
Observe that t(ψ, π) = 0 if and only if ψ = ψ ∗ . First, s%k satisfies Assumptions 3(a), (b), and
(g) by Proposition 5, (77), (78), and Assumption 6. Second, rk,0 (ψ, π) satisfies Assumptions 3(c)
and (d) from Proposition 5 and (78). Third, ukx0 (ψ, π) satisfies Assumptions 3(e) and (f) from
Proposition 5(c). Therefore, the stated result follows from Corollary 1(b).
Proof of Proposition 8. The proof is similar to that of Proposition 3 of Kasahara and Shimotsu
(2015).

Let tη := η − η ∗ and tλ := α(1 − α)v(λ), so that t(ψ, π) = (t0η , t0λ )0 .

arg maxψ∈Θψ `n (ψ, π, ξ) denote the MLE of ψ, and split t(ψ̂π , π) as t(ψ̂π , π) =
38

Let ψ̂π :=

(t̂0η , t̂0λ )0 ,

where

we suppress the dependence of t̂η and t̂λ on π. Define G%n := νn (s%k ). Let
"
G%n =

Gηn

#
,

Gλ%n

Gλ.η%n := Gλ%n − Iλη% Iη−1 Gηn ,

−1
Zλ.η%n := Iλ.η%
Gλ.η%n ,

tη.λ% := tη + Iη−1 Iηλ% tλ .

Then, we can write (19) as
sup

sup

√
√
2 [`n (ψ, π, ξ) − `n (ψ ∗ , π, ξ)] − An ( ntη.λ% ) − B%n ( ntλ ) = op (1),

(79)

ξ∈Ξ ϑ∈Anεc (ξ)

where
An (tη.λ% ) = 2t0η.λ% Gηn − t0η.λ% Iη tη.λ% ,
0
B%n (tλ ) = 2t0λ Gλ.η%n − t0λ Iλ.η% tλ = Zλ%n
Iλ.η% Zλ%n − (tλ − Zλ%n )0 Iλ.η% (tλ − Zλ%n ).

(80)

√
√
Observe that 2[`0n (ϑ̂0 ) − `0n (ϑ∗0 )] = maxtη [2 nt0η Gηn − nt0η Iη tη ] + op (1) = maxtη.λ% An ( ntη.λ% ) +
√
op (1) from applying Corollary 1 to `0n (ϑ0 ) and noting that the set of possible values of both ntη
√
and ntη.λ% approaches Rdim(η) . In conjunction with (79), we obtain, uniformly in π ∈ Θπ ,
√
2[`n (ψ̂π , π, ξ) − `0n (ϑ̂0 )] = B%n ( nt̂λ ) + op (1).

(81)

√
√
Define t̃λ by B%n ( nt̃λ ) = maxtλ ∈α(1−α)v(Θλ ) B%n ( ntλ ). Then, we have
√
2[`n (ψ̂π , π, ξ) − `0n (ϑ̂0 )] = B%n ( nt̃λ ) + op (1),
√
uniformly in π ∈ Θπ because (i) B%n ( nt̃λ ) ≥ 2[`n (ψ̂π , π, ξ) − `0n (ϑ̂0 )] + op (1) from the definition
√
of t̃λ and (81), and (ii) 2[`n (ψ̂π , π, ξ) − `0n (ϑ̂0 )] ≥ B%n ( nt̃λ ) + op (1) from the definition of ψ̂, (79),
and t̃λ = Op (n−1/2 ).

√
Finally, the asymptotic distribution of sup% B%n ( nt̃λ ) follows from applying Theorem 1(c)
√
of Andrews (2001) to B%n ( nt̃λ ). First, Assumption 2 of Andrews (2001) holds trivially for
√
B%n ( nt̃λ ). Second, Assumption 3 of Andrews (2001) is satisfied by (78) and Assumption 6. Assumption 4 of Andrews (2001) is satisfied by Proposition 7. Assumption 5∗ of Andrews (2001) holds
with BT = n1/2 because α(1 − α)v(Θλ ) is locally equal to the cone v(Rq ) given that α(1 − α) > 0
√
d
for all α ∈ Θα . Therefore, sup%∈Θ% B%n ( nt̃λ ) → sup%∈Θ% (t̃0λ% Iλ.η% t̃λ% ) follows from Theorem 1(c)
of Andrews (2001).
Proof of Proposition 9. The proof is similar to that of Proposition 7. Define Λjk,m,x−m (ψ, π) and
Λjk,m (ψ, π) as in the proof of Proposition 7. Expanding lkϑx0 − 1 five times around ψ ∗ similarly to
(75) while fixing π gives, with ψ ∈ [ψ, ψ ∗ ],
lkϑx0 − 1 =

4
X

Λjk,0 (ψ ∗ , π)0 (∆ψ)⊗j + Λ5k,0 (ψ, π)0 (∆ψ)⊗5 + ukx0 (ψ, π),

j=1

39

(82)

where ukx0 (ψ, π) :=
Define pψπk,0 :=

j
j
∗
∗
0
⊗j + [Λ5
5
0
⊗5
j=1 [Λk,0,x0 (ψ , π) − Λk,0 (ψ , π)] (∆ψ)
k,0,x0 (ψ, π) − Λk,0 (ψ, π)] (∆ψ) .
k−1
pψπ (Yk |Y0 ). Observe that s%k defined in (33) satisfies

P4



s%k

k−1

Noting that ∇λ pψ∗ π (Yk |Y0

∇η pψ∗ πk,0 /pψ∗ πk,0





ζk,0 (%)/2




 ∇λ λ p ∗

µ σ ψ πk,0 /α(1 − α)pψ ∗ πk,0




:=  ∇λ2σ pψ∗ πk,0 /2α(1 − α)pψ∗ πk,0  .


 ∇λ λ p ∗

/α(1
−
α)p
∗
ψ
πk,0
ψ
πk,0
β µ




 ∇λβ λσ pψ∗ πk,0 /α(1 − α)pψ∗ πk,0 
V (∇λβ λβ pψ∗ πk,0 )/α(1 − α)pψ∗ πk,0
k−1

) = 0 and ∇λη0 pψ∗ π (Yk |Y0

) = 0 from (13) and (14), we may rewrite

(82) as, with t(ψ, π) and s%k defined in (30) and (33),
lϑkx0 − 1 = t(ψ, π)0 s%k + rk,0 (π) + ukx0 (ψ, π),

(83)

e k,0 (π)0 τ (ψ) + Λ5 (ψ, π)0 (∆ψ)⊗5 + λ4µ [∇λ4 pψ∗ πk,0 − b(α)∇λ2 pψ∗ πk,0 ]/4!pψ∗ πk,0 ,
where rk,0 (π) := Λ
k,0
µ
σ
e k,0 (π)
τ (ψ) is the vector that collects the elements of {(∆ψ)⊗j }4 that are not in t(ψ, π), and Λ
j=2

denotes the vector of the corresponding elements of {Λjk,0 (ψ ∗ , π)}4j=2 .
The stated result follows from Corollary 1 if the terms on the right-hand side of
(83) satisfy Assumption 3.

Similarly to the proof of Proposition 8, define vk,m (ϑ) :=

(ζk,m (%), Λ1k,m (ψ, π)0 , . . . , Λ5k,m (ψ, π)0 )0 .

Note that ζk,m (%) satisfies Proposition 5 because the
k−1

k−1

mean value theorem and ∇λ2µ pψ∗ 0α (Yk |Y−m ) = 0 gives ζk,m (%) = [∇λ2µ pψ∗ %α (Yk |Y0
k−1
k−1
∇λ2µ pψ∗ 0α (Yk |Y0 )]/[%α(1−α)pψ∗ %α (Yk |Y0 )]

=

) −

k−1
k−1
∇% ∇λ2µ pψ∗ α%̄ (Yk |Y−m )/[α(1−α)pψ∗ %̄α (Yk |Y−m )]

for %̄ ∈ [0, %]. Therefore, vk,∞ (ϑ) := limm→∞ vk,m (ϑ) is well-defined, and vk,0 (ϑ) and vk,∞ (ϑ) satisfy
(77)–(78) from repeating the argument in the proof of Proposition 8.
We proceed to show that the terms on the right-hand side of (83) satisfy Assumption 3. Observe that t(ψ, π) = 0 if and only if ψ = ψ ∗ . s%k and ukx0 (ψ, π) satisfy Assumption 3 by noting
that s%k is a linear function of vk,0 (ϑ) and using the argument in the proof of Proposition 7
by replacing Assumption 6 with Assumption 7. We show that each component of rk,0 (π) satisfies Assumptions 3(c) and (d). First, Λ5k,0 (ψ, π)0 (∆ψ)⊗5 satisfies Assumptions 3(c) and (d) from
Proposition 5, (78) and λ5µ = (12λµ /b(α))[λ2σ + b(α)λ4µ /12] − 12(λσ /b(α))λµ λσ = O(|ψ||t(ψ, π)|).
Second, λ4µ [∇λ4µ pψ∗ πk,0 − b(α)∇λ2σ pψ∗ πk,0 ]/pψ∗ πk,0 satisfies Assumptions 3(c) and (d) from Lemma
e k,0 (π)0 τ (ψ), observe that ∇ληj pψ∗ πk,0 = 0 for any j ≥ 1 in view of (24)–(27).
7(b). Third, for ∆
e k,0 (π)0 τ (ψ) is written as, with ∆η := η − η ∗ ,
Therefore, ∆
e k,0 (π)0 τ (ψ) = ∇(η⊗2 )0 pψ∗ πk,0 (∆η)⊗2 /2!pψ∗ πk,0 + R3kϑ + R4kϑ ,
∆

40

(84)

where R3kϑ := ∇(ψ⊗3 )0 pψ∗ πk,0 (∆ψ)⊗3 /3!pψ∗ πk,0 and
R4kϑ := [∇(ψ⊗4 )0 pψ∗ πk,0 (∆ψ)⊗4 − ∇λ4µ pψ∗ πk,0 λ4µ ]/4!pψ∗ πk,0 .

(85)

The first term in (84) clearly satisfies Assumptions 3(c) and (d). The terms in R3kϑ belong to
one of the following three groups: (i) the term associated with λ3σ , (ii) the term associated with
λ3µ , or (iii) the other terms. These terms satisfy Assumptions 3(c) and (d) because the term (i)
is bounded by |ψ||t(ψ, π)| because λ3σ = λσ [λ2σ + b(α)λ4µ /12] − (λ3µ b(α))λµ λσ /12, the term (ii) is
bounded by %λ3µ from Lemma 7(a), and the terms in (iii) are bounded by |ψ||t(ψ, π)| because they
either contain ∆η or a term of the form λiµ λjσ λkβ with i + j + k = 3 and i, j 6= 3. Similarly, the
terms in R4kϑ satisfy Assumptions 3(c) and (d) because they either contain ∆η or a term of the
form λiµ λjσ λkβ with i + j + k = 4 and i 6= 4. This proves that rk,0 (π) satisfies Assumptions 3(c) and
(d), and the stated result is proven.
Proof of Proposition 10. The proof is similar to the proof of Proposition 3(c) of Kasahara and
Shimotsu (2015). Let (ψ̂α , %̂α ) := arg max(ψ,%)∈Θψ ×Θ% `n (ψ, %, α, ξ) denote the MLE of (ψ, %) for
a given α.

Consider the sets Θ1λ := {λ ∈ Θλ : |λµ | ≥ n−1/8 (log n)−1 } and Θ2λ := {λ ∈

Θλ : |λµ | ≤ n−1/8 (log n)−1 }, so that Θλ = Θ1λ ∪ Θ2λ .
arg max(ψ,%)∈Θ

j
ψ ×Θ% ,λ∈Θλ

For j = 1, 2, define (ψ̂αj , %̂jα ) :=

`n (ψ, %, α, ξ). Then, uniformly in α,

n
o
`n (ψ̂α , %̂α , α, ξ) = max `n (ψ̂α1 , %̂1α , α, ξ), `n (ψ̂α2 , %̂2α , α, ξ) .
Henceforth, we suppress the dependence of ψ̂α , %̂α , etc. on α.
Define B%n (tλ (λ, %, α)) as in (80) in the proof of Proposition 8 but using t(ψ, π) and s%k defined in
(30) and (33) and replacing tλ in (80) with tλ (λ, %, α). Observe that the proof of Proposition 8 goes
through up to (81) with the current notation and that G%n and I% are continuous in %. Further, %̂1 =
Op (n−1/4 (log n)2 ) because %̂1 (λ̂1µ )2 = Op (n−1/2 ) from Proposition 9(a) and |λ̂1µ | ≥ n−1/8 (log n)−1 .
√
√
Consequently, B%̂1 n ( ntλ (λ̂1 , %̂1 , α)) = B0n ( ntλ (λ̂1 , %̂1 , α)) + op (1), and, uniformly in α,
√
√
2[`n (ψ̂, %̂, α, ξ) − `0n (ϑ̂0 )] = max{B0n ( ntλ (λ̂1 , %̂1 , α)), B%̂2 n ( ntλ (λ̂2 , %̂2 , α))} + op (1).

(86)

We proceed to construct parameter spaces Λ̃1λα and Λ̃2λα% that are locally equal to the cones
Λ1λ and Λ2λ% defined in (35). Define c(α) := α(1 − α), and denote the elements of tλ (λ̂j , %̂j , α)

41

corresponding to (31) by


t̂j%µ2





%̂j (λ̂jµ )2



 j 


 t̂µσ 


λ̂µ λ̂σ




 t̂j 
(λ̂j )2 + b(α)(λ̂j )4 /12
σ
µ
2




tλ (λ̂j , %̂j , α) =  jσ  := c(α) 
.
j
j
 t̂βµ 


λ̂β λ̂µ




j j
 t̂j 


λ̂
λ̂
 βσ 


β σ
t̂jv(β)
v(λ̂jβ )
Note that λ̂1σ = Op (n−3/8 log n) and λ̂1β = Op (n−3/8 log n) because (t̂1µσ , t̂1βµ ) = Op (n−1/2 ) from
Proposition 9(a) and |λ̂1µ | ≥ n−1/8 (log n)−1 . Furthermore, t̂2σ2 = c(α)(λ̂2σ )2 + op (n−1/2 ) because
|λ̂2µ | ≤ n−1/8 (log n)−1 . Consequently,
t̂1βσ = op (n−1/2 ),

t̂1v(β) = op (n−1/2 ),

t̂1σ2 = c(α)b(α)(λ̂1µ )4 /12 + op (n−1/2 ),

t̂2σ2 = c(α)(λ̂2σ )2 + op (n−1/2 ).

(87)

In view of this, let tλ (λ, %, α) := (t%µ2 , tµσ , tσ2 , t0βµ , t0βσ , t0v(β) )0 ∈ Rqλ , and consider the following sets:
Λ̃1λα := {tλ (λ, %, α) : t%µ2 = c(α)%λ2µ , tµσ = c(α)λµ λσ , tσ2 = c(α)b(α)λ4µ /12,
tβµ = c(α)λβ λµ , tβσ = 0, tv(β) = 0 for some (λ, %) ∈ Θλ × Θ% },
Λ̃2λα%

:= {tλ (λ, %, α) : t%µ2 = c(α)%λ2µ , tµσ = c(α)λµ λσ , tσ2 = c(α)λ2σ ,
tβµ = c(α)λβ λµ , tβσ = c(α)λβ λσ , tv(β) = c(α)v(λβ ) for some λ ∈ Θλ }.

Λ̃1λα is indexed by α but does not depend on % because B0n (·) in (86) does not depend on %,
whereas Λ̃2λα% is indexed by both α and % because B%̂2 n (·) in (86) depends on %̂2 . Define (λ̃1α , %̃1α )
√
√
√
and λ̃2α% by B0n ( ntλ (λ̃1α , %̃1α , α)) = maxtλ (λ,%,α)∈Λ̃1 B0n ( ntλ (λ, %, α)) and B%n ( ntλ (λ̃2α% , %, α)) =
λα
√
maxtλ (λ,%,α)∈Λ̃2 B%n ( ntλ (λ, %, α)).
λα%
√
√
Define Wn (α) := max{B0n ( ntλ (λ̃1α , %̃1α , α)), sup%∈Θ% B%n ( ntλ (λ̃2α% , %, α))}, then we have
2[`n (ψ̂, %̂, α, ξ) − `0n (ϑ̂0 )] = Wn (α) + op (1),

(88)

uniformly in α ∈ Θα because (i) Wn (α) ≥ 2[`n (ψ̂, %̂, α, ξ) − `0n (ϑ̂0 )] + op (1) in view of the definition
of (λ̃1α , %̃1α , λ̃2α% ), (86), and (87), and (ii) 2[`n (ψ̂, %̂, α, ξ) − `0n (ϑ̂0 )] ≥
max{2[maxη `n (η, λ̃1α , %̃1α , α, ξ), sup%∈Θ% maxη `n (η, λ̃2α% , %, α, ξ)} − 2`0n (ϑ̂0 ) + op (1) = Wn (α) + op (1)
from the definition of (ψ̂, %̂).
The asymptotic distribution of the LRTS follows from applying Theorem 1(c) of Andrews
√
√
(2001) to (B0n ( ntλ (λ̃1α , %̃1α , α)), B%n ( ntλ (λ̃2α% , %, α))). First, Assumption 2 of Andrews (2001)
√
holds trivially for B%n ( nt(λ, %, α)). Second, Assumption 3 of Andrews (2001) is satisfied by (78)
and Assumption 7. Assumption 4 of Andrews (2001) is satisfied by Proposition 9. Assumption 5∗

42

of Andrews (2001) holds with BT = n1/2 because Λ̃1λα is locally (in a neighborhood of % = 0, λ = 0)
equal to the cone Λ1λ and Λ̃2λα% is locally equal to the cone Λ2λ% uniformly in % ∈ Θ% . Consequently,
d

Wn (α) → max{I{% = 0}(t̃1λ )0 Iλ.η0 t̃1λ , sup%∈Θ% (t̃2λ% )0 Iλ.η% t̃2λ% } uniformly in α from Theorem 1(c) of
Andrews (2001), and the stated result follows from (88).
Proof of Proposition 11. The proof is similar to that of Proposition 9. Expanding lkϑx0 − 1 five
times around ψ ∗ and proceeding as in the proof of Proposition 9 gives
lϑkx0 − 1 = t(ψ, π)0 s%k + rk,0 (π) + ukx0 (ψ, π),

(89)

where t(ψ, π) is defined in (42), s%k is defined in (43) and satisfies


s%k

∇η pψ∗ πk,0 /pψ∗ πk,0







ζk,0 (%)/2




∗
∗
∇µ3 fk /3!fk


:= 
,
∗
∗


∇µ4 fk /4!fk


∇

p
/α(1
−
α)p
∗
∗
 λβ λµ ψ πk,0
ψ πk,0 
e v(λ ) pψ∗ πk,0 /α(1 − α)pψ∗ πk,0
∇
β

and
e k,0 (π)0 τ (ψ) + Λ5 (ψ, π)0 (∆ψ)⊗5
rk,0 (π) : = Λ
k,0
+ λ3µ [∇λ3µ pψ∗ πk,0 /pψ∗ πk,0 − α(1 − α)(1 − 2α)∇µ3 fk∗ /fk∗ ]/3!
+ λ4µ [∇λ4µ pψ∗ πk,0 /pψ∗ πk,0 − α(1 − α)(1 − 6α + 6α2 )∇µ4 fk∗ /fk∗ ]/4!,
where ukx0 (ψ, π), pψπk,m , and the terms in the definition of rk,0 (π) are defined similarly to those
in the proof of Proposition 9.
The stated result is proven if the terms on the right-hand side of (89) satisfy Assumption
3. t(ψ, π) = 0 if and only if ψ = ψ ∗ . s%k and ukx0 (ψ, π) satisfy Assumption 3 by the same
argument as the proof of Proposition 9. For rk,0 (π), first, Λ5k,0 (ψ, π)0 (∆ψ)⊗5 satisfies Assumptions
3(c) and (d) from a similar argument to the proof of Proposition 9; λ5µ is dominated by λ3µ or
λ4µ because inf 0≤α≤1 max{|1 − 2α|, |1 − 6α + 6α2 |} > 0. Second, similar to (84) in the proof of
e k,0 (π)0 τ (ψ) = ∇(η⊗2 )0 pψ∗ πk,0 (∆η)⊗2 /2!pψ∗ πk,0 + R̃3kϑ + R4kϑ , where R̃3kϑ :=
Proposition 9, write Λ
[∇(ψ⊗3 )0 pψ∗ πk,0 (∆ψ)⊗3 − ∇λ3µ pψ∗ πk,0 λ3µ ]/3!pψ∗ πk,0 , and R4kϑ is defined as R4kϑ in (85). The term
∇(η⊗2 )0 pψ∗ πk,0 (∆η)⊗2 /2!pψ∗ πk,0 clearly satisfies Assumptions 3(c) and (d). The terms in R̃3kϑ satisfy
Assumptions 3(c) and (d) because they contain either ∆η or λ2µ λβ or λµ λ2β or λ3β . The terms in
R4kϑ satisfy Assumptions 3(c) and (d) because they either contain ∆η or a term of the form λiµ λ4−i
β
with 1 ≤ i ≤ 3. The last two terms in rk,0 (π) satisfy Assumptions 3(c) and (d) from Lemma 8.
Therefore, rk,0 (π) satisfies Assumptions 3(c) and (d), and the stated result is proven.

43

Proof of Proposition 12. The proof is similar to the proof of Proposition 10. Let (ψ̂, %̂, α̂) :=
arg max(ψ,%,α)∈Θψ ×Θ% ×Θα `n (ψ, %, α, ξ) denote the MLE of (ψ, %, α).

Consider the sets Θ1λ :=

{λ ∈ Θλ : |λµ | ≥ n−1/6 (log n)−1 } and Θ2λ := {λ ∈ Θλ : |λµ | ≤ n−1/6 (log n)−1 }, so that
Θλ = Θ1λ ∪ Θ2λ . For j = 1, 2, define (ψ̂ j , %̂j , α̂j ) := arg max(ψ,%,α)∈Θ
so that `n (ψ̂, %̂, α̂, ξ) = maxj∈{1,2} `n

j
ψ ×Θ% ×Θα ,λ∈Θλ

`n (ψ, %, α, ξ),

(ψ̂ j , %̂j , α̂j , ξ).

Define B%n (tλ (λ, %, α)) as in (80) in the proof of Proposition 8 but using t(ψ, π) and s%k defined in
(42) and (43) and replacing tλ in (80) with tλ (λ, %, α). Observe that %̂1 = Op (n−1/6 (log n)2 ) because
%̂1 (λ̂1µ )2 = Op (n−1/2 ) from Proposition 11(a) and |λ̂1µ | ≥ n−1/6 (log n)−1 . Using the argument of the
proof of Proposition 10 leading to (86), we obtain
√
√
2[`n (ψ̂, %̂, α̂, ξ) − `0n (ϑ̂0 )] = max{B0n ( ntλ (λ̂1 , %̂1 , α̂1 )), B%̂2 n ( ntλ (λ̂2 , %̂2 , α̂2 ))} + op (1).
We proceed to construct parameter spaces that are locally equal to the cones Λ1λ and Λ2λ% defined
in (44). Define c(α) := α(1 − α), and denote the elements of tλ (λ̂j , %̂j , α̂j ) corresponding to (42) by


t̂j%µ2





%̂j (λ̂jµ )2



 j 


 t̂ 3 
(1 − 2α̂j )(λ̂jµ )3


 µ 


 j 
j 4
j 
j j
j
j
j
2
tλ (λ̂ , %̂ , α̂ ) =  t̂µ4  := c(α̂ ) (1 − 6α̂ + 6(α̂ ) )(λ̂µ )  .
 j 


 t̂ 
λ̂jβ λ̂jµ


 βµ 
j
j
v(λ̂β )
t̂v(β)
Note that λ̂1β = Op (n−1/3 log n) because t̂1βµ = Op (n−1/2 ) from Proposition 11(a) and |λ̂1µ | ≥
n−1/6 (log n)−1 . Furthermore, |λ̂2µ | ≤ n−1/6 (log n)−1 . Therefore,
t̂1v(β) = op (n−1/2 ),

t̂2µ3 = op (n−1/2 ),

t̂2µ4 = op (n−1/2 ).

In view of this, let tλ (λ, %, α) := (t%µ2 , tµ3 , tµ4 , t0βµ , t0v(β) )0 ∈ Rqλ , and consider the following sets:
Λ̃1λ := {tλ (λ, %, α) : t%µ2 = c(α)%λ2µ , tµ3 = c(α)(1 − 2α)λ3µ , tµ4 = c(α)(1 − 6α + 6α2 )λ4µ ,
tβµ = c(α)λβ λµ , tv(β) = 0 for some (λ, %, α) ∈ Θλ × Θ% × Θα },
Λ̃2λα% := {tλ (λ, %, α) : t%µ2 = c(α)%λ2µ , tµ3 = tµ4 = 0,
tβµ = c(α)λβ λµ , tv(β) = c(α)v(λβ ) for some λ ∈ Θλ }.
√
√
Define (λ̃1 , %̃1 , α̃1 ) and λ̃2α% by B0n ( ntλ (λ̃1 , %̃1 , α̃1 )) = maxtλ (λ,%,α)∈Λ̃1 B0n ( ntλ (λ, %, α)) and
λ
√
√
B%n ( ntλ (λ̃2α% , %, α)) = maxtλ (λ,%,α)∈Λ̃2 B%n ( ntλ (λ, %, α)). Λ̃1λ is locally (in the neighborhood of
λα%

% = 0, λ = 0) equal to the cone Λ1λ because, when |1 − 2α| ≥  > 0 for some positive constant , we
have tµ4 /tµ3 → 0 as λµ → 0, and when α is in the neighborhood of 1/2, we have 1 − 6α + 6α2 < 0.
Λ̃2λα% is locally equal to the cone Λ2λ% uniformly in % ∈ Θ% .
√
√
Define Wn := max{B0n ( ntλ (λ̃1 , %̃1 , α̃1 )), sup(α,%)∈Θα ×Θ% B%n ( ntλ (λ̃2α% , %, α))}.

44

Proceed-

ing as in the proof of Proposition 10 gives 2[`n (ψ̂, %̂, α̂, ξ) − `0n (ϑ̂0 )] = Wn + op (1), and the
asymptotic distribution of the LRTS follows from applying Theorem 1(c) of Andrews (2001) to
√
√
(B0n ( ntλ (λ̃1 , %̃1 , α̃1 )), B%n ( ntλ (λ̃2α% , %, α)).
∗ denote an arbitrarily small neighborhood of Υ∗ , and
Proof of Propositions 14, 15, and 16. Let Nm
m
∗ . Proposition
let ψ̂m denote a local MLE that maximizes `n (ψm , πm , ξM0 +1 ) subject to ψm ∈ Nm
∗
0
13 and Υ∗ = ∪M
m=1 Υm imply that `n (ϑ̂M0 +1 , ξM0 +1 ) = maxm=1,...,M0 `n (ψ̂m , πm , ξM0 +1 ) with prob∗ for any ` 6= m, it follows from Proposition 13 that
ability approaching one. Because ψ`∗ ∈
/ Nm
∗ = o (1).
ψ̂m − ψm
p
∗ ,π ,ξ
Next, `n (ψm , πm , ξM0 +1 ) − `n (ψm
m M0 +1 ) admits the same expansion as `n (ψ, π, ξ) −

`n (ψ ∗ , π, ξ) in (19) or (34). Therefore, the stated result follows from applying the proof of Propositions 8, 10, and 12 to `n (ψ̂m , πm , ξM0 +1 ) − `n (ϑ̂M0 , ξM0 ) for each m and combining the results to
0
derive the joint asymptotic distribution of {`n (ψ̂m , πm , ξM0 +1 ) − `n (ϑ̂M0 , ξM0 )}M
m=1 .

Proof of Proposition 17. Observe that Proposition 2 holds under Pnϑ∗ ,x0 under the assumptions of
Propositions 7, 9, and 11. Because ϑn = (ηn0 , λ0n , πn0 )0 ∈ Nc/√n by choosing c > |h|, it follows from
Proposition 2 that
sup log
x0 ∈X

dPnϑn ,x0
dPnϑ∗ ,x0

1
− h0 νn (s%n k ) + h0 I%n h = oPnϑ∗ ,x (1),
0
2

(90)

where s%k is given by (17), (33), and (43) for the models of the non-normal distribution, heteroscedastic normal distribution, and homoscedastic normal distribution, respectively. Furthermore, νn (s%n k ) ⇒ G% under Pnϑ∗ ,x0 , where G% is a mean zero Gaussian process with cov(G%1 , G%2 ) =
I%1 %2 := limk→∞ Eϑ∗ (s%1 k s0%2 k ). Therefore, dPnϑn ,x0 /dPnϑ∗ ,x0 converges in distribution under Pnϑ∗ ,x0


to exp N (µ, σ 2 ) with µ = −(1/2)h0 I% h and σ 2 = h0 I% h, so that E(exp N (µ, σ 2 ) ) = 1. Consequently, part (a) follows from Le Cam’s first lemma (see, e.g., Corollary 12.3.1 of Lehmann
and Romano (2005)). Part (b) follows from Le Cam’s third lemma (see, e.g., Corollary 12.3.2 of
Lehmann and Romano (2005)) because part (a) and (90) imply that


νn (s%n k )


d


→N
dPn
log dPϑnn∗ ,x0
ϑ ,x0

0

!

− 12 h0 I% h

,

I%

I% h

!!

h0 I% h0 I% h

under Pnϑ∗ ,x0 .

Proof of Proposition 18. The proof follows the argument in the proof of Proposition 8. Observe
√
that hη = 0 and hλ = ntλ (λn , πn ) hold under H1n . Therefore, Proposition 17 holds under Pnϑn ,x0
implied by H1n , and, in conjunction with Theorem 12.3.2(a) of Lehmann and Romano (2005),
Propositions 5 and 7 hold under Pnϑn ,x0 . Consequently, the proof of Proposition 8 goes through if
we replace Gλ.η%n ⇒ Gλ.η% with Gλ.η%n ⇒ Gλ.η% + (Iλ%% − Iλη% Iη−1 Iηλ% )hλ = Gλ.η% + Iλ.η% hλ , and
the stated result follows.
45

Proof of Propositions 19 and 20. The proof is similar to the proof of Proposition 18. Observe that,
√
j
for j ∈ {a, b}, hjη = 0 and hjλ = ntλ (λn , πn ) + o(1) hold under H1n
. Therefore, Proposition 17
j
holds under Pnϑn ,x0 implied by H1n
, and the stated result follows from repeating the argument of

proof of Proposition 18.
Proof of Proposition 21. We only provide the proof for the models of the non-normal distribution
with M0 = 1 because the proof for the other models is similar. The proof follows the argument in
the proof of Theorem 15.4.2 in Lehmann and Romano (2005). Define Cη as the set of sequences
√
{ηn } satisfying n(ηn − η ∗ ) → hη for some finite hη . Denote the MLE of the one-regime model
√
parameter by η̂n . For the MLE under H0 , n(η̂n − η ∗ ) converges in distribution to a Pϑ∗ -a.s. finite
random variable by the standard argument. Then, by the Almost Sure Representation Theorem
(e.g., Theorem 11.2.19 of Lehmann and Romano (2005)), there exist random variables η̃n and
h̃η defined on a common probability space such that η̂n and η̃n have the same distribution and
√
n(η̃n − η ∗ ) → h̃η almost surely. Therefore, {η̃n } ∈ Cη with probability one, and the stated result
under H0 follows from Lemma 9 because η̂n and η̃n have the same distribution.
For the MLE under H1n , note that the proof of Proposition 18 goes through when hη is finite
√
even if hη 6= 0. Therefore, n(η̂n −η ∗ ) converges in distribution to a Pϑn -a.s. finite random variable
under H1n . Hence, the stated result follows from Lemma 9 and repeating the argument in the case
of H0 .

11.2
11.2.1

Auxiliary results
Missing information principle

The following lemma extends equations (3.1) and (3.2) in Louis (1982), expressing the higher-order
derivatives of the log-likelihood function in terms of the conditional expectation of the derivatives of
the complete data log-likelihood function. For notational brevity, assume ϑ is scalar. Adaptations to
vector-valued ϑ are straightforward but need more tedious notation. Let ∇j `(Y ) := ∇jϑ log P (Y ; ϑ)
and ∇j `(Y, X) := ∇jϑ log P (Y, X; ϑ). For random variables V1 , . . . , Vq and Y , define the central conr

r

ditional moment of (V1r1 · · · Vq q ) as Ec [V1r1 · · · Vq q |Y ] := E[(V1 − E[V1 |Y ])r1 · · · (Vq − E[Vq |Y ])rq |Y ].

46

Lemma 1. For any random variables X and Y with density P (Y, X; θ) and P (Y ; θ),




∇`(Y ) = E [∇`(Y, X)|Y ] , ∇2 `(Y ) = E ∇2 `(Y, X) Y + Ec (∇`(Y, X))2 Y ,






∇3 `(Y ) = E ∇3 `(Y, X) Y + 3Ec ∇2 `(Y, X)∇`(Y, X) Y + Ec (∇`(Y, X))3 Y ,






∇4 `(Y ) = E ∇4 `(Y, X) Y + 4Ec ∇3 `(Y, X)∇`(Y, X) Y + 3Ec (∇2 `(Y, X))2 Y




 
 2
+ 6Ec ∇2 `(Y, X)(∇`(Y, X))2 Y + Ec (∇`(Y, X))4 Y − 3 Ec (∇`(Y, X))2 Y
,
 5

 4

 3

5
c
c
2
∇ `(Y ) = E ∇ `(Y, X) Y + 5E ∇ `(Y, X)∇`(Y, X) Y + 10E ∇ `(Y, X)∇ `(Y, X) Y




+ 10Ec ∇3 `(Y, X)(∇`(Y, X))2 Y + 15Ec (∇2 `(Y, X))2 ∇`(Y, X) Y



 

+ 10Ec ∇2 `(Y, X)(`(Y, X))3 Y − 30Ec ∇2 `(Y, X)∇`(Y, X) Y Ec (∇`(Y, X))2 Y



 

+ Ec (∇`(Y, X))5 Y − 10Ec (∇`(Y, X))3 Y Ec (∇`(Y, X))2 Y ,


∇6 `(Y ) = E ∇6 `(Y, X) Y




+ 6Ec ∇5 `(Y, X)∇`(Y, X) Y + 15Ec ∇4 `(Y, X)∇2 `(Y, X) Y




+ 15Ec ∇4 `(Y, X)(∇`(Y, X))2 Y + 60Ec ∇3 `(Y, X)∇2 `(Y, X)∇`(Y, X) Y




+ 10Ec (∇3 `(Y, X))2 Y + 15Ec (∇2 `(Y, X))3 Y

 



+ 20Ec ∇3 `(Y, X)(∇`(Y, X))3 Y − 60Ec ∇3 `(Y, X)∇`(Y, X) Y E (∇`(Y, X))2 Y


 
 2
+ 45Ec (∇2 `(Y, X))2 (∇`(Y, X))2 Y − 90 Ec ∇2 `(Y, X)∇`(Y, X) Y

 

− 45Ec (∇2 `(Y, X))2 Y Ec (∇`(Y, X))2 Y



 

+ 15Ec ∇2 `(Y, X)(∇`(Y, X))4 Y − 90Ec ∇2 `(Y, X)(∇`(Y, X))2 Y Ec (∇`(Y, X))2 Y

 

− 60Ec ∇2 `(Y, X)∇`(Y, X) Y Ec (∇`(Y, X))3 Y



 

+ Ec (∇`(Y, X))6 Y − 15Ec (∇`(Y, X))4 Y Ec (∇`(Y, X))2 Y
 3
 2
 
 
,
+ 30 Ec (∇`(Y, X))2 Y
− 10 Ec (∇`(Y, X))3 Y
provided that the conditional expectation on the right-hand side exists. When P (Y ; θ) on the lefthand side is replaced with P (Y |Z; θ), the stated result holds with P (Y, X; θ) and E[·|Y ] on the
right-hand side replaced with P (Y, X|Z; θ) and E[·|Y, Z].
Proof of Lemma 1. The stated result follows from a direct calculation and relations such as

47

∇jϑ P (Y ; ϑ)/P (Y ; ϑ) = E[∇jϑ P (Y, X; ϑ)/P (Y, X; ϑ)|Y ] and
∇ log f = ∇f /f,

∇2 log f = ∇2 f /f − (∇ log f )2 ,

∇3 log f = ∇3 f /f − 3∇2 f ∇f /f 2 + 2(∇f /f )3 ,
∇4 log f = ∇4 f /f − 4∇3 f ∇f /f 2 − 3(∇2 f /f )2 + 12∇2 f (∇f )2 /f 3 − 6(∇f /f )4 ,
∇5 log f = ∇5 f /f − 5∇4 f ∇f /f 2 − 10∇3 f ∇2 f /f 2 + 20∇3 f (∇f )2 /f 3
+ 30(∇2 f )2 ∇f /f 3 − 60∇2 f (∇f )3 /f 4 + 24(∇f /f )5 ,
∇6 log f = ∇6 f /f − 6∇5 f ∇f /f 2 − 15∇4 f ∇2 f /f 2 + 30∇4 f (∇f )2 /f 3 − 10(∇3 f )2 /f 2
+ 120∇3 f ∇2 f ∇f /f 3 − 120∇3 f (∇f )3 /f 4 + 30(∇2 f )3 /f 3
− 270(∇2 f )2 (∇f )2 /f 4 + 360∇2 f (∇f )4 /f 5 − 120(∇f )6 /f 6 ,
3

3

(91)

3

2

∇ f /f = ∇ log f + 3∇ log f ∇ log f + (∇ log f ) ,
∇4 f /f = ∇4 log f + 4∇3 log f ∇ log f + 3(∇2 log f )2 + 6∇2 log f (∇ log f )2 + (∇ log f )4 ,
∇5 f /f = ∇5 log f + 5∇4 log f ∇ log f + 10∇3 log f ∇2 log f + 10∇3 log f (∇ log f )2
+ 15(∇2 log f )2 ∇ log f + 10∇2 log f (∇ log f )3 + (∇ log f )5 ,
∇6 f /f = ∇6 log f + 6∇5 log f ∇ log f + 15∇4 log f ∇2 log f + 15∇4 log f (∇ log f )2
+ 10(∇3 log f )2 + 60∇3 log f ∇2 log f ∇ log f + 20∇3 log f (∇ log f )3
+ 15(∇2 log f )3 + 45(∇2 log f )2 (∇ log f )2 + 15∇2 log f (∇ log f )4 + (∇ log f )6 .
For example, ∇3 `(Y ) is derived by writing ∇3 `(Y ) as, with suppressing ϑ,
∇3 `(Y )


∇2 P (Y ) ∇P (Y )
∇P (Y ) 3
∇3 P (Y )
−3
+2
=
P (Y )
P (Y ) P (Y )
P (Y )
 3

 2
 

 
3
∇ P (Y, X)
∇ P (Y, X)
∇P (Y, X)
∇P (Y, X)
=E
Y − 3E
Y E
Y +2 E
Y
P (Y, X)
P (Y, X)
P (Y, X)
P (Y, X)


= E ∇3 `(Y, X) + 3∇2 `(Y, X)∇`(Y, X) + (∇`(Y, X))3 Y


− 3E ∇2 `(Y, X) + (∇`(Y, X))2 Y E [∇`(Y, X)|Y ] + 2 {E [∇`(Y, X)|Y ]}3 ,
and collecting terms. ∇4 `(Y ), ∇5 `(Y ), and ∇6 `(Y ) are derived similarly.
11.2.2

Auxiliary lemmas
k

We first collect the notations. Define Zk−1 := (Xk−1 , Yk−1 , Wk , Xk , Yk ) and denote the derivative
of the complete data log-density by
k

φi (ϑ, Zk−1 ) := ∇i log pϑ (Yk , Xk |Yk−1 , Xk−1 , Wk ),

48

i ≥ 1.

(92)

k

We use short-handed notation φiϑk := φi (ϑ, Zk−1 ). We also suppress the superscript 1 from φ1ϑk ,
so that φϑk = φ1ϑk . For random variables V1 , . . . , Vq and a conditioning set F, define the central
conditional moment of (V1 , . . . , Vq ) as
Ecϑ [V1 , . . . , Vq |F] := Eϑ [(V1 − Eϑ [V1 |F]) · · · (Vq − Eϑ [Vq |F])|F] .
For example, Ecϑ [φϑk1 φϑk2 |F] := Eϑ [(φϑk1 − Eϑ [φϑk1 |F]) (φϑk2 − Eϑ [φϑk2 |F])|F].
Let I(j) = (i1 , . . . , ij ) denote a sequence of positive integers with j elements, let σ(I(j))
denote the set of all the unique permutations of (i1 , . . . , ij ), and let |σ(I(j))| denote its cardinality.
For example, if I(3) = (2, 1, 1), then σ(I(3)) = {(2, 1, 1), (1, 2, 1), (1, 1, 2)} and |σ(I(3))| = 3; if
I(3) = (1, 1, 1), then σ(I(3)) = (1, 1, 1) and |I(3)| = 1. Let T (j) = (t1 , . . . , tj ) for j = 1, . . . , 6. For
a conditioning set F, define the symmetrized central conditional moments as
h
i
I(1)
ΦϑT (1) [F] := Eϑ φiϑt1 1 F ,
I(3)

ΦϑT (3) [F] :=
I(4)

ΦϑT (4) [F] :=

1
|σ(I(3))|
1
|σ(I(4))|

I(2)

ΦϑT (2) [F] :=

1
|σ(I(2))|

X

h
i
Ecϑ φ`ϑt1 1 φ`ϑt2 2 F ,

(`1 ,`2 )∈σ(I(2))

h
i
Ecϑ φ`ϑt1 1 φ`ϑt2 2 φ`ϑt3 3 F ,

X

(93)

(`1 ,`2 ,`3 )∈σ(I(3))

X

1 `2 `3 `4
Φ̃`ϑT
(4) ,

(`1 ,...,`4 )∈σ(I(4))

`4
`2
`2
`3
`4
c `3
c `1
c `1
1 `2 `3 `4
where Φ̃`ϑT
(4) := Eϑ [φϑt1 φϑt2 φϑt3 φϑt4 |F] − Eϑ [φϑt1 φϑt2 |F]Eϑ [φϑt3 φϑt4 |F]

− Ecϑ [φ`ϑt1 1 φ`ϑt3 3 |F]Ecϑ [φ`ϑt2 2 φ`ϑt4 4 |F] − Ecϑ [φ`ϑt1 1 φ`ϑt4 4 |F]Ecϑ [φ`ϑt2 2 φ`ϑt3 3 |F], and
I(5)

ΦϑT (5) [F] :=

1
|σ(I(5))|

h
i
Ecϑ φ`θt11 φ`θt22 φ`θt33 φ`θt44 φ`θt55 F

(`1 ,...,`5 )∈σ(I(5))

h

X

−



X

Ecϑ φ`θtaa φ`θtb b φ`θtc c


i
h
F Ecϑ φ`θtdd φ`θtee F  ,
i

({a,b,c},{d,e})∈σ5
I(6)
ΦϑT (6) [F]

:=

Ecϑ [φθt1 φθt2 φθt3 φθt4 φθt5 φθt6 |F]

X

−



Ecϑ [φθta φθtb φθtc φθtd |F] Ecϑ φθte φθtf F

({a,b,c,d},{e,f })∈σ61

X

−

Ecϑ [φθta φθtb φθtc |F] Ecϑ



φθtd φθte φθtf F

({a,b,c},{d,e,f })∈σ62

+2

X



Ecϑ [φθta φθtb |F] Ecϑ [φθtc φθtd |F] Ecϑ φθte φθtf F ,

({a,b},{c,d},{e,f })∈σ63

(94)

49

where
σ5 := the set of
σ61 := the set of
σ62 := the set of
σ63 := the set of

5
3 = 10 partitions of {1, 2, 3, 4, 5} of the form {a, b, c}, {d, e},

6
4 = 15 partitions of {1, 2, 3, 4, 5, 6} of the form {a, b, c, d}, {e, f },

6
3 /2 = 10 partitions of {1, 2, 3, 4, 5, 6} of the form {a, b, c}, {d, e, f },
 
6 4
2 2 /6 = 15 partitions of {1, 2, 3, 4, 5, 6} of the form {a, b}, {c, d}, {e, f }.



(95)

Note that these moments are symmetric with respect to (t1 , . . . , tj ). For j = 1, 2, . . . , 6, k ≥ 1,
I(j)

m ≥ 0, and x ∈ X , define the difference between the sums of the ΦϑT (j) ’s over different time indices
and conditioning sets as
I(j)

h
i
k
I(j)
k
ΦϑT (j) Y−m , W−m
, X−m = x

X

∆j,k,m,x (ϑ) :=

T (j)∈{−m+1,...,k}j

h
i
k−1
I(j)
k−1
ΦϑT (j) Y−m , W−m
, X−m = x ,

X

−

(96)

T (j)∈{−m+1,...,k−1}j

where

P

T (j)∈{−m+1,...,k}j

denotes
I(j)

P
Pk
T (j)∈{−m+1,...,k−1}j
tj =−m+1 , and
t2 =−m+1 · · ·
I(j)
analogously to ∆j,k,m,x (ϑ) by dropping X−m = x from the

Pk

is defined similarly. Define ∆j,k,m (θ)

t1 =−m+1

Pk

conditioning variable.
n
Henceforth, we suppress the conditioning variable W−m
from the conditioning sets and condi-

tional densities unless confusion might arise. The following lemma expresses the derivatives of the
I(j)

log-densities, ∇j `k,m,x (ϑ)’s, in terms of the ∆j,k,m,x (ϑ)’s. The first two equations are also given in
DMR (p. 2272 and pp. 2276–7).
Lemma 2. For all 1 ≤ k ≤ n, m ≥ 0, and x ∈ X ,
∇1 `k,m,x (ϑ) = ∆11,k,m,x (ϑ),

∇2 `k,m,x (ϑ) = ∆21,k,m,x (ϑ) + ∆1,1
2,k,m,x (ϑ),

1,1,1
∇3 `k,m,x (ϑ) = ∆31,k,m,x (ϑ) + 3∆2,1
2,k,m,x (ϑ) + ∆3,k,m,x (ϑ),
2,2
2,1,1
1,1,1,1
∇4 `k,m,x (ϑ) = ∆41,k,m,x (ϑ) + 4∆3,1
2,k,m,x (ϑ) + 3∆2,k,m,x (ϑ) + 6∆3,k,m,x (ϑ) + ∆4,k,m,x (ϑ),
3,2
3,1,1
2,2,1
∇5 `k,m,x (ϑ) = ∆51,k,m,x (ϑ) + 5∆4,1
2,k,m,x (ϑ) + 10∆2,k,m,x (ϑ) + 10∆3,k,m,x (ϑ) + 15∆3,k,m,x (ϑ)
1,1,1,1,1
+ 10∆2,1,1,1
4,k,m,x (ϑ) + ∆5,k,m,x (ϑ),
4,2
3,3
4,1,1
∇6 `k,m,x (ϑ) = ∆61,k,m,x (ϑ) + 6∆5,1
2,k,m,x (ϑ) + 15∆2,k,m,x (ϑ) + 10∆2,k,m,x (ϑ) + 15∆3,k,m,x (ϑ)
2,2,2
3,1,1,1
2,2,1,1
2,1,1,1,1
1,1,1,1,1
+ 60∆3,2,1
3,k,m,x (ϑ) + 15∆3,k,m,x (ϑ) + 20∆4,k,m,x (ϑ) + 45∆4,k,m,x (ϑ) + 15∆5,k,m,x (ϑ) + ∆6,k,m,x (ϑ).
I(j)

Further, the above holds when ∇j `k,m,x (ϑ) and ∆j,k,m,x (ϑ) are replaced with ∇j `k,m (ϑ) and
I(j)

∆j,k,m (ϑ).
Proof of Lemma 2. The stated result follows from writing ∇j `k,m,x (ϑ) =
k−1
k
∇j log pϑ (Y−m+1
|Y−m , X−m = x) − ∇j log pϑ (Y−m+1
|Y−m , X−m = x), applying Lemma 1 to the

50

k
right-hand side, and noting that ∇j log pϑ (Y−m+1
, Xk−m+1 |Y−m , X−m ) =

Pk

t=−m+1 φ

j (ϑ, Zt )
t−1

The result for ∇j `k,m,x (ϑ) with j = 1, 2 is also given in DMR

(see (1) and (92)).

2,1
2272 and pp.
2276–7).
For j = 3, the term ∆2,k,m,x
(ϑ) follows from
Pk
Pk
Pk
k
k
2,1
c
2
1
t2 =−m+1 Eϑ [φϑt1 φϑt2 |Y −m , X−m = x] =
t2 =−m+1 Φϑt1 t2 [Y −m , X−m =
t1 =−m+1
t1 =−m+1

(p.
Pk

k
x]. For j = 4, note that when we apply Lemma 1 to ∇4 log pϑ (Y−m+1
|Y−m , X−m = x), the last two
P
k
1,1,1,1
terms on the right-hand side of Lemma 1 can be written as T (4)∈{−m+1,...,k}4 ΦϑT
(4) [Y −m , X−m =

x]. The result for j = 5 follows from a similar argument. For j = 6, note that when we apply
k
Lemma 1 to ∇6 log pϑ (Y−m+1
|Y−m , X−m = x), the last four terms on the right-hand side of Lemma
P
k
I(6)
1 can be written as T (6)∈{−m+1,...,k}6 ΦϑT (6) [Y−m , X−m = x].
I(j)

The following lemma provides bounds on ΦϑT (j) [F] defined in (93) and (94) and is used in the
proof of Lemma 4. For j = 2, . . . , 6, define kφit k∞ := supϑ∈N ∗ supx,x0 |φi (ϑ, Yt , x, Yt−1 , x0 )| and
P
`
I(j)
kφT (j) k∞ := (`1 ,...,`j )∈σ(I(j)) kφ`t11 k∞ · · · kφtjj k∞ .
Lemma 3. Under Assumptions 1, 2, and 4, there exists a finite non-stochastic constant C that
does not depend on ρ such that, for all m0 ≥ m ≥ 0, all −m < t1 ≤ t2 ≤ · · · ≤ tj ≤ n, all ϑ ∈ N ∗
and all x ∈ X , and j = 2, . . . , 6,
I(j)

n

I(j)

n

I(j)

n

I(j)

n

I(j)

n

I(j)

n

I(j)

n

I(j)

n

I(j)

(a)

|ΦϑT (j) [Y−m ]| ≤ Cρ(t2 −t1 −1)+ ∨(t3 −t2 −1)+ ∨···∨(tj −tj−1 −1)+ kφT (j) k∞ ,

(b)

|ΦϑT (j) [Y−m , X−m = x]| ≤ Cρ(t2 −t1 −1)+ ∨(t3 −t2 −1)+ ∨···∨(tj −tj−1 −1)+ kφT (j) k∞ ,

(c)

|ΦϑT (j) [Y−m , X−m = x] − ΦϑT (j) [Y−m ]| ≤ Cρ(m+t1 −1)+ kφT (j) k∞ ,

(d)

|ΦϑT (j) [Y−m , X−m = x] − ΦϑT (j) [Y−m0 , X−m0 = x]| ≤ Cρ(m+t1 −1)+ kφT (j) k∞ ,

(e)

|ΦϑT (j) [Y−m ] − ΦϑT (j) [Y−m ]| ≤ Cρ(n−1−tj )+ kφT (j) k∞ ,

(f )

|ΦϑT (j) [Y−m , X−m = x] − ΦϑT (j) [Y−m , X−m = x]| ≤ Cρ(n−1−tj )+ kφT (j) k∞ .

I(j)

I(j)

I(j)

I(j)

n−1

I(j)

I(j)

n−1

I(j)

Proof of Lemma 3. Recall supϑ∈N ∗ supx,x0 |φi (ϑ, Yt , x, Yt−1 , x0 ) − Eϑ [φi (ϑ, Yt , x, Yt−1 , x0 )|F]|
≤

2 supϑ∈N ∗ supx,x0 |φi (ϑ, Yt , x, Yt−1 , x0 )| for the conditioning sets F that appear in the

lemma.

t

t

n

`

n

Define φ̃iϑt := φi (ϑ, Zt−1 ) − Eϑ [φi (ϑ, Zt−1 )|Y−m ], so that Ecϑ [φ`ϑt1 1 · · · φϑtj j |Y−m ] =
`

n

Eϑ [φ̃`ϑt1 1 · · · φ̃ϑtj j |Y−m ]. Henceforth, we suppress the subscript ϑ from φiϑt and φ̃iϑt .
t

Recall that φi (ϑ, Zt−1 ) depends on Xt and Xt−1 . Parts (c) and (d) follow from Lemma 11(a)
R
and the fact that, for any two probability measures µ1 and µ2 , supf (x):maxx |f (x)|≤1 | f (x)dµ1 (x) −
R
f (x)dµ2 (x)| = 2kµ1 − µ2 kT V (see, e.g., Levin et al. (2009, Proposition 4.5)). Similarly, parts
(e) and (f) for tj ≤ n − 1 follow from Lemma 11(b), and parts (e) and (f) for tj = n follow from
I(j)

I(j)

|ΦϑT (j) [·]| ≤ 2j kφT (j) k∞ .
We proceed to show parts (a) and (b). The results for j = 2 and j = 3 follow from Lemma

51

11(c) and
E(Xt1 − EXt1 ) · · · (Xtj − EXtj ) = cov[Xt1 , (Xt2 − EXt2 ) · · · (Xtj − EXtj )]

(97)

= cov[(Xt1 − EXt1 ) · · · (Xtj−1 − EXtj−1 ), Xtj ].
n

Before proving the results for j ≥ 4, we collect some results. For a conditioning set F = Y−m or
n

{Y−m , Xm = x}, Lemmas 11(c) and (97) imply that
I(j)

`

|Ecϑ [φ`t11 · · · φtjj |F]| ≤ Cρ(t2 −t1 −1)+ ∨(tj −tj−1 −1)+ kφT (j) k∞ ,
`

(98)

`

`

k+1
· · · φtjj |F]|
|Ecϑ [φ`t11 · · · φtjj |F] − Ecϑ [φ`t11 · · · φ`tkk |F]Ecϑ [φtk+1

I(j)

`

`

k+1
· · · φ̃tjj |F]| ≤ Cρ(tk+1 −tk −1)+ kφT (j) k∞
= |covϑ [φ̃`t11 · · · φ̃`tkk , φ̃tk+1

for any 2 ≤ k ≤ j − 2.

I(4)

(99)

I(4)

Parts (a) and (b) hold for j = 4 because ΦϑT (4) [F] ≤ Cρ(t2 −t1 −1)+ ∨(t4 −t3 −1)+ kφT (4) k∞ from (98)
I(4)

I(4)

`1 `2 `3 `4
1 `2 `3 `4
and we have ΦϑT (4) [F] ≤ Cρ(t3 −t2 −1)+ kφT (4) k∞ from writing Φ̃`ϑT
(4) defined in (93) as Φ̃ϑT (4) =

covϑ [φ̃`t11 φ̃`t22 , φ̃`t33 φ̃`t44 |F] − Ecϑ [φ`t11 φ`t33 |F]Ecϑ [φ`t22 φ`t44 |F] − Ecϑ [φ`t11 φ`t44 |F]Ecϑ [φ`t22 φ`t33 |F] and applying (99).
Parts (a)–(b) for j = 5 follow from a similar argument.
For j
(98).

=

6,

first,

I(6)

I(6)

ΦϑT (6) [F] is bounded by Cρ(t2 −t1 −1)+ ∨(t6 −t5 −1)+ kφT (6) k∞ from
I(6)

Second, write ΦϑT (6) [F]

=

A1 + A2 , where A1

=

Ecϑ [φt1 φt2 φt3 φt4 φt5 φt6 |F] −
I(6)

Ecϑ [φt1 φt2 φt3 |F] Ecϑ [φt4 φt5 φt6 |F] and A2 denotes all the terms on the right-hand side of ΦϑT (6) [F]
I(6)

in (94) except for A1 . A1 is bounded by Cρ(t4 −t3 −1)+ kφT (6) k∞ from (99), and A2 is bounded
I(6)

I(6)

I(6)

by Cρ(t4 −t3 −1)+ kφT (6) k∞ from (98). Therefore, ΦϑT (6) [F] is bounded by Cρ(t4 −t3 −1)+ kφT (6) k∞ .
I(6)

B1 + B2 + B3 , where B1 = Ecϑ [φt1 φt2 φt3 φt4 φt5 φt6 |F] −
P
Ecϑ [φt3 φt4 φt5 φt6 |F]Ecϑ [φt1 φt2 |F], B2 = − ({1,2,c,d},{e,f })∈X61 Ecϑ [φ1 φ2 φtc φtd |F]Ecϑ [φte φtf |F] +

P
2 ({a,b},{c,d},{e,f })∈X63 Ecϑ [φta φtb |F]Ecϑ [φtc φtd |F]Ecϑ [φte φtf |F], where X61 is the set of 42 = 6 parThird,

write ΦϑT (6) [F]

=

titions of {1, 2, 3, 4, 5, 6} of the form of {1, 2, c, d}, {e, f } and
X63 := {({1, 2}, {3, 4}, {5, 6}), ({1, 2}, {3, 5}, {4, 6}), ({1, 2}, {3, 6}, {4, 5})}, and B3 denotes all
I(6)

the terms on the right-hand side of ΦϑT (6) [F] except for B1 + B2 .
I(6)
Cρ(t3 −t2 −1)+ kφT (6) k∞

P

({1,2,c,d},{e,f })∈X61
P

B1 is bounded by

from (99). We can write B2 as

{−Ecϑ [φt1 φt2 φtc φtd |F]Ecϑ [φte φtf |F] + Ecϑ [φt1 φt2 |F]Ecϑ [φtc φtd |F]Ecϑ [φte φtf |F]} =

c
({1,2,c,d},{e,f })∈X61 Eϑ [φte φtf |F]covϑ [φ̃θt1 φ̃θt2 , φ̃θtc φ̃θtd |F], then this is bounded by
I(6)
I(6)
Cρ(t3 −t2 −1)+ kφT (6) k∞ from (99). Finally, B3 is bounded by Cρ(t3 −t2 −1)+ kφT (6) k∞ from (98).
I(6)
I(6)
I(6)
fore, ΦϑT (6) [F] is bounded by Cρ(t3 −t2 −1)+ kφT (6) k∞ . From a similar argument, ΦϑT (6) [F]
I(6)
bounded by Cρ(t5 −t4 −1)+ kφT (6) k∞ , and parts (a) and (b) follow.

−

I(j)

Thereis also

I(j)

We next present the result that bound the difference between ∆j,k,m,x (ϑ) and ∆j,k,m (θ) that
appear on the right-hand side in Lemma 2. This lemma extends Lemmas 13 and 17 of DMR.
Let rI(1) = qi1 ; rI(2) = qi1 /2 if i1 = i2 and (qi1 ∧ qi2 )/2 if i1 6= i2 ; rI(3) = qi1 /3 if i1 = i2 = i3 ,
(qi1 /2∧qi2 /4) if i1 6= i2 = i3 , (qi1 ∧qi2 ∧qi3 )/3 if i1 , i2 , i3 are distinct; rI(4) = qi1 /4 if i1 = i2 = i3 = i4 ,
52

(qi1 ∧ qi3 )/4 if i1 6= i2 = i3 = i4 or i1 = i2 6= i3 = i4 ; rI(5) = qi1 /5 if i1 = i2 = i3 = i4 = i5 ;
(qi1 /3 ∧ qi2 /6) if i1 6= i2 = i3 = i4 = i5 ; rI(6) = q1 /6. Part (d) of this lemma establishes the uniform
I(j)

convergence of {∆j,k,m,x (ϑ)}m≥0 to a random variable that does not depend on x.
Lemma 4. Under Assumptions 1, 2, and 4, for j = 1, . . . , 6, there exist random variables
KI(j) , {MI(j),k }nk=1 ∈ LrI(j) (Pϑ∗ ) such that, for all 1 ≤ k ≤ n and m0 ≥ m ≥ 0,
I(j)

I(j)

I(j)

I(j)

sup sup |∆j,k,m,x (ϑ) − ∆j,k,m (ϑ)| ≤ KI(j) (k + m)7 ρb(k+m−1)/24c

(a)

x∈X ϑ∈N ∗

Pϑ∗ -a.s.,

sup sup |∆j,k,m,x (ϑ) − ∆j,k,m0 ,x (ϑ)| ≤ KI(j) (k + m)7 ρb(k+m−1)/1340c

(b)

x∈X ϑ∈N ∗

Pϑ∗ -a.s.,

I(j)

I(j)

(c) supm≥0 supx∈X supϑ∈N ∗ |∆j,k,m,x (ϑ)| + supm≥0 supϑ∈N ∗ |∆j,k,m (ϑ)| ≤ MI(j),k Pϑ∗ -a.s., (d) UniI(j)

I(j)

formly in ϑ ∈ N ∗ and x ∈ X , ∆j,k,m,x (ϑ) and ∆j,k,m (ϑ) converge Pϑ∗ -a.s. and in LrI(j) (Pϑ∗ ) to
I(j)

∆j,k,∞ (ϑ) ∈ LrI(j) (Pϑ∗ ) as m → ∞.
Proof of Lemma 4. First, we prove parts (a) and (b). Recall T (j) = (t1 , . . . , tj ). For part (a),
define, suppressing the dependence of AT (j) on ϑ and I(j),

AT (j)

i
h
i
h

k
k−1
I(j)
I(j)

Y
,
X
=
x
−
Φ
Y
,
X
=
x
Φ

−m
−m
−m

h
hϑT (j) i −m
i
 ϑT (j)
k
k−1
I(j)
I(j)
− ΦϑT (j) Y−m + ΦϑT (j) Y−m ,
:=

i
h
i
h


ΦI(j) Yk , X−m = x − ΦI(j) Yk
−m
−m ,
ϑT (j)
ϑT (j)

AT (j,`,k) := At1 t2 ···tj−`

k···k
|{z}

,

` times

` times

I(j)

I(j)

∆a :=

otherwise,

where T (j, `, k) := (T (j − `), k, · · · , k ).
| {z }

Then, we can write ∆j,k,m,x (ϑ) − ∆j,k,m (ϑ) =
X

if max{t1 , . . . , tj } < k,

AT (j) ,

∆b :=

T (j)∈{−m+1,...,k−1}j

P

T (j)∈{−m+1,...,k}j

j−1  
X
j
`=1

`

AT (j) = ∆a + ∆b + ∆c , where

X

AT (j,`,k) ,

∆c := A(k,...,k) ,

T (j−`)∈{−m+1,...,k−1}j−`

and ∆b := 0 when j = 1. From Lemma 3 and the symmetry of AT (j) , ∆a is bounded by
I(j)

CBj,k,m Mj,k,m , where
Bj,k,m :=



X

ρ(m+t1 −1)+ ∧ ρ(t2 −t1 −1)+ ∧ · · · ∧ ρ(tj −tj−1 −1)+ ∧ ρ(k−1−tj −1)+



−m+1≤t1 ≤t2 ≤···≤tj ≤k−1



X

=


ρ(t1 −1)+ ∧ ρ(t2 −t1 −1)+ ∧ · · · ∧ ρ(tj −tj−1 −1)+ ∧ ρ(k+m−1−tj −1)+ ,

1≤t1 ≤t2 ≤···≤tj ≤k+m−1
I(j)

Mj,k,m :=

max

−m+1≤t1 ,...,tj ≤k−1

i

kφit11 k∞ kφit22 k∞ · · · kφtjj k∞ .

From (t − 1)+ ≥ bt/2c and Lemma 13, Bj,k,m is bounded by Cj2 (ρ)ρb(k+m−1)/4jc .
P∞
I(j)
−2
i `
We proceed to derive a bound of Mj,k,m . Define kφi k`∞ :=
t=−∞ (|t| ∨ 1) kφt k∞ . When
53

I(j)

i1 = i2 = · · · = ij , it follows from Lemma 14 that Mj,k,m ≤ (k + m)j+1 kφi1 kj∞ , and kφi1 kj∞ ∈
LrI(j) (Pϑ∗ ) from Assumption 4. In the other cases, observe that if x, y, z ≥ 0, we have xy ≤ x2 + y 2 ,
xyz ≤ x3 + y 3 + z 3 , xy ≤ x4 + y 4/3 , and xy ≤ x3 + y 3/2 from Young’s inequality. By using this
I(j)

result and Lemma 14, we can bound Mj,k,m by
j = 2 and i1 6= i2 :

(k + m)2 (kφi1 k2∞ + kφi2 k2∞ ),

j = 3 and i1 6= i2 = i3 :

(k + m)3 (kφi1 k2∞ + kφi2 k4∞ ),

j = 3 and i1 , i2 , i3 are distinct :

(k + m)2 (kφi1 k3∞ + kφi2 k3∞ + kφi3 k3∞ ),

j = 4 and i1 6= i2 = i3 = i4 :

(k + m)3 (kφi1 k4∞ + kφi2 k4∞ ),

j = 4 and i1 = i2 6= i3 = i4 :

(k + m)3 (kφi1 k4∞ + kφi3 k4∞ ),

j = 5 and i1 6= i2 = i3 = i4 = i5 :

(k + m)3 (kφi1 k3∞ + kφi2 k6∞ ).

Therefore, from Assumption 4, ∆a is bounded by the right-hand side of part (a). From
P
P
(m+t1 −1)+ ∧ ρ(t2 −t1 −1)+ ∧
Lemmas 3 and 13, ∆b is bounded by C j−1
−m+1≤t1 ≤···≤tj−` ≤k−1 (ρ
`=1
I(j)

· · · ∧ ρ(k−tj−` −1)+ )Mj,k+1,m
I(j)
Cρb(k+m−1)/4(j−1)c Mj,k+1,m ,

≤

I(j)

Cρb(k+m−1)/4(j−1)c Mj,k+1,m .

Similarly, ∆c is bounded by

and part (a) of the lemma follows.

For part (b), define, for −m0 + 1 ≤ t1 , . . . , tj ≤ k,

DT (j),m0 ,x


ΦI(j) [Yk 0 , X 0 = x] − ΦI(j) [Yk−10 , X 0 = x],
−m
−m
−m
−m
θT (j)
θT (j)
:=
ΦI(j) [Yk 0 , X 0 = x],
−m
−m
θT (j)

if max{t1 , . . . , tj } < k,
otherwise,

P
I(j)
and define DT (j),m,x similarly. Then, we can write ∆j,k,m,x (θ) = T (j)∈{−m+1,...,k}j DT (j),m,x and
P
P
I(j)
∆j,k,m0 ,x (θ) = T (j)∈{−m0 +1,...,k}j DT (j),m0 ,x = ∆d + ∆e , where ∆d := T (j)∈{−m+1,...,k}j DT (j),m0 ,x
and
∆e :=

j  
X
j

`

`=1

−m
X

−m
X

···

t1 =−m0 +1

k
X

k
X

···

t` =−m0 +1 t`+1 =−m+1

DT (j),m0 ,x .

tj =−m+1

I(j)

From the same argument as part (a), ∆j,k,m,x (θ) − ∆d is bounded by the right-hand side of part

(a). For ∆e , observe that, with Mj := max1≤`≤j j` ,
|∆e | ≤ Mj

j
X

−m
X

−m
X

···

`=1 t1 =−m0 +1 t2 =−m0 +1
−m
X

≤ jMj
t1

=−m0 +1

k
X
t2

−m
X

≤ jMj j!
t1

=−m0 +1

k
X

t` =−m0 +1 t`+1 =−m+1
k
X

···

=−m0 +1

−m
X

tj

···

k
X

DT (j),m0 ,x

tj =−m+1

DT (j),m0 ,x

=−m0 +1

X

DT (j),m0 ,x .

t1 ≤t2 ≤···≤tj ≤k

From Lemma 3, if t1 ≤ · · · ≤ tj , we have |DT (j),m0 ,x | ≤ C[I{tj < k}(ρ(t2 −t1 −1)+ ∧ ρ(tj −tj−1 −1)+ ∧

54

I(j)

· · · ∧ ρ(k−1−tj −1)+ ) + I{tj = k}(ρ(t2 −t1 −1)+ ∧ · · · ∧ ρ(tj −tj−1 −1)+ )]kφT (j) k∞ . Hence, part (b) follows
from Lemma 15.
I(j)

For part (c), observe that supm≥0 supx∈X supϑ∈N ∗ |∆j,k,m,x (ϑ)| ≤ A + B, where A :=
I(j)

I(j)

I(j)

supm≥0 supx∈X supϑ∈N ∗ |∆j,k,m,x (ϑ) − ∆j,k,0,x (ϑ)| and B := supx∈X supϑ∈N ∗ |∆j,k,0,x (ϑ)|. A is
bounded by KI(j) k 7 ρb(k−1)/1340c from part (b). B does not depend on m and is distributionally
I(j)

I(j)

equivalent to supx∈X supϑ∈N ∗ |∆j,1,k−1,x (ϑ)|. This is bounded by supx∈X supϑ∈N ∗ |∆j,1,k−1,x (ϑ) −
I(j)

I(j)

∆j,1,0,x (ϑ)| + supx∈X supϑ∈N ∗ |∆j,1,0,x (ϑ)|. The first term is in LrI(j) (Pϑ∗ ) from part (b), and
I(j)

the second term is in LrI(j) (Pϑ∗ ) from the definition of ∆j,k,m,x (ϑ).
rI(j)

MI(j),k ∈ L

Therefore, there exists

(Pϑ∗ ) such that A + B ≤ MI(j),k , and part (c) holds in view of part (a). Part (d)
I(j)

I(j)

follows from parts (a)–(c) because parts (a)–(c) imply that {∆j,k,m,x (ϑ)}m≥0 and {∆j,k,m (ϑ)}m≥0
are uniform LrI(j) (Pϑ∗ )-Cauchy sequences with respect to ϑ ∈ N ∗ that converge to the same limit
and Lq (Pϑ∗ ) is complete.
Lemma 5. Under Assumptions 1, 2, and 4, there exist random variables {Kk }nk=1 ∈ L(1+ε)qϑ /ε (Pϑ∗ )
and ρ ∈ (0, 1) such that, for all 1 ≤ k ≤ n and m0 ≥ m ≥ 0,
k−1

sup
ϑ∈N ∗

pϑ (Yk |Y−m )
k−1
pϑ∗ (Yk |Y−m )

k−1

≤ Kk ,

pϑ (Yk |Y−m , X−m = x)

sup sup
x∈X

k−1
pϑ∗ (Yk |Y−m , X−m

ϑ∈N ∗

k−1

−

= x)

pϑ (Yk |Y−m )
k−1
pϑ∗ (Yk |Y−m )

≤ Kk ρk+m−1 .

k−1

k−1

Furthermore, these bounds hold uniformly in x ∈ X when pϑ (Yk |Y−m ) and pϑ∗ (Yk |Y−m ) are
k−1

k−1

replaced with pϑ (Yk |Y−m0 , X−m0 = x) and pϑ∗ (Yk |Y−m0 , X−m0 = x).
k−1

Proof of Lemma 5. The first result follows from noting that pϑ (Yk |Y−m )
P
k−1
=
(xk−1 ,xk )∈X 2 gϑ (Yk |Y k−1 , xk )qϑx (xk−1 , xk )Pϑ (xk−1 |Y −m ) ∈ [σ− Gϑk , σ+ Gϑk ] and using Ask−1

k−1

sumption 4(b). For the second result, observe that |pϑ (Yk |Y−m , X−m = x) − pϑ (Yk |Y−m )| ≤
P
k−1
k−1
= x) − Pϑ (xk−1 |Y−m )| ≤
(xk−1 ,xk )∈X 2 gϑ (Yk |Y k−1 , xk )qϑx (xk−1 , xk )|Pϑ (xk−1 |Y −m , X−m
ρk+m−1 σ+ Gϑk /σ− , where the second inequality follows from Lemma 11(a). The second result
then follows from writing the left-hand side as
k−1

k−1

pϑ (Yk |Y−m , X−m = x) − pϑ (Yk |Y−m )
k−1

k−1

+

pϑ∗ (Yk |Y−m , X−m = x)

k−1

k−1

pϑ (Yk |Y−m ) pϑ∗ (Yk |Y−m ) − pϑ∗ (Yk |Y−m , X−m = x)
k−1

pϑ∗ (Yk |Y−m )

k−1

,

pϑ∗ (Yk |Y−m , X−m = x)

k−1

noting that pϑ (Yk |Y−m , X−m = x) ≥ σ− Gϑk , and using the derived bounds. The results with
k−1

k−1

pϑ (Yk |Y−m0 , X−m0 = x) and pϑ∗ (Yk |Y−m0 , X−m0 = x) are proven similarly.
The following result originally appeared in equations (59)–(60) of Kasahara and Shimotsu
(2015). We state this as a lemma for ease of reference.

55

Lemma 6. Let f (µ, σ 2 ) denote the density of N (µ, σ 2 ). Then

∇λkµ f (c1 λµ , c2 λ2µ )

λµ =0




c1 ∇µ f (0, 0)




c2 ∇ 2 f (0, 0) + 2c ∇ 2 f (0, 0)
2 σ
1 µ
=


c31 ∇µ3 f (0, 0) + 6c1 c2 ∇µσ2 f (0, 0)




c4 ∇ 4 f (0, 0) + 12c2 c ∇ 2 f (0, 0)∇ 2 f (0, 0) + 12c2 ∇ 4 f (0, 0)
σ
1 µ
1 2 µ
2 σ

if k = 1,
if k = 2,
if k = 3,
if k = 4.

Proof of Lemma 6. Observe that a composite function f (λµ , h(λµ )) satisfies ∇λkµ f (λµ , h(λµ )) =

P
(∇λµ + ∇u )k f (λµ , h(u))|u=λµ = kj=0 kj ∇λk−j uj f (λµ , h(u))|u=λµ . Further, because ∇uj u2 |u=0 = 0
µ

except for j = 2, it follows from Faà di Bruno’s formula that ∇uj f (c1 λµ , c2 u2 )|λµ =u=0 is 0 if
j = 1, 3, is 2c2 ∇h f (0, h(0)) if j = 2, and is 12c22 ∇h2 f (0, h(0)) if j = 4. Therefore, the stated result
follows.
Lemma 7. Suppose the assumptions of Proposition 9 hold. Then, there exist %̄1 , %̄2 , %̄3 ∈ (0, %)
such that, for all k ≥ 1,
k−1

(a)

(b)

∇λ3µ pψ∗ π (Yk |Y0

k−1

)

k−1
pψ∗ π (Yk |Y0 )
k−1
∇λ4µ pψ∗ π (Yk |Y0 )
k−1
pψ∗ π (Yk |Y0 )

=%

∇% ∇λ3µ pψ∗ %̄1 α (Yk |Y0

k−1

− b(α)

∇λ2σ pψ∗ π (Yk |Y0

k−1

pψ∗ π (Yk |Y0
k−1

=%

)
,

k−1
pψ∗ %̄1 α (Yk |Y0 )

∇% ∇λ4µ pψ∗ %̄2 α (Yk |Y0

)

k−1
pψ∗ %̄2 α (Yk |Y0 )

)

)
k−1

−%

∇% ∇λ2σ pψ∗ %̄3 α (Yk |Y0
k−1
pψ∗ %̄3 α (Yk |Y0 )

)

.

Proof of Lemma 7. Part (a) holds if
k−1

∇λ3µ pψ∗ 0α (Yk |Y0
k−1

because (i) ∇λ3µ pψ∗ %α (Yk |Y0

k−1

)/pψ∗ 0α (Yk |Y0
k−1

)−∇λ3µ pψ∗ 0α (Yk |Y0

the mean value theorem and (ii)

k−1
pψ∗ %α (Yk |Y0 )

) = 0,

(100)
k−1

) = ∇% ∇λ3µ pψ∗ %̄α (Yk |Y0

)% for %̄ ∈ (0, %) from

does not depend on the value of %.
k−1

We proceed to show (100). Note that∇λ3µ pψ∗ π (Yk |Y0

k−1

∇λ3µ log pψ∗ π (Y1k−1 |Y0 ) from (91) and ∇λ pψ∗ π (Yk |Y0

56

k−1

)/pψ∗ π (Yk |Y0
) = 0.

) = ∇λ3µ log pψ∗ π (Y1k |Y0 )−

Let ∇i `∗t := ∇λiµ log gt∗ with

∇`∗t = ∇1 `∗t . Observe that
∇λ3µ log pψ∗ 0α (Y1k |Y0 )

=

k
X

k X
k
i
i
h
h
X
k
k
3 ∗
Eψ∗ 0α ∇ `t Y0 + 3
Eψ∗ 0α ∇2 `∗t1 ∇`∗t2 Y0

t=1

t1 =1 t2 =1

+

k
X

k
X

k
X

h
i
k
Eψ∗ 0α ∇`∗t1 ∇`∗t2 ∇`∗t3 Y0

t1 =1 t2 =1 t3 =1

=

k
X

(101)

i
h
k
Eψ∗ 0α ∇3 `∗t + 3∇2 `∗t ∇`∗t + ∇`∗t ∇`∗t ∇`∗t Y0

t=1

=

k
X

h
i
k
Eψ∗ 0α ∇λ3µ gt∗ /gt∗ Y0 ,

t=1

where the first equality follows from Lemma 1, the second equality holds because (i) Xt is serially
independent when % = 0 and (ii) ∇`∗t = d1t ∇µ ft∗ /ft∗ and ∇2 `∗t = d2t ∇2µ ft∗ /ft∗ − (d1t ∇µ ft∗ /ft∗ )2 , and
k

k

(iii) Eψ∗ 0α [d1t |Y0 ] = Eψ∗ 0α [d2t |Y0 ] = 0 from (27), and the third equality follows from (91) The
right-hand side is 0 from (27), and hence part (a) is proven.
For part (b), from a similar argument to part (a), the stated result holds if
k−1

∇λ4µ pψ∗ 0α (Yk |Y0

k−1

)/pψ∗ 0α (Yk |Y0
k−1

Observe that ∇λ4µ pψ∗ 0α (Yk |Y0
k−1

from (91), ∇λ pψ∗ π (Yk |Y0

k−1

) = b(α)∇λ2σ pψ∗ 0α (Yk |Y0
k−1

)/pψ∗ 0α (Yk |Y0

k−1

)/pψ∗ 0α (Yk |Y0

).

(102)

) = ∇λ4µ log pψ∗ 0α (Y0k |Y0 )−∇λ4µ log pψ∗ 0α (Y0k−1 |Y0 )
k−1

) = 0, and ∇λ2µ log pψ∗ 0α (Yk |Y0

) = 0. A similar derivation to (101)

gives
∇λ4µ log pψ∗ 0α (Y0k |Y0 )

=

k
X

h
i
k
Eψ∗ 0α ∇λ4µ gt∗ /gt∗ Y0 .

(103)

t=1
k−1

(102) follows from (103) because (i) ∇λ2σ pψ∗ 0α (Yk |Y0

k−1

)/pψ∗ 0α (Yk |Y0

k

k

) = Eϑ∗ [∇λ2σ gk∗ |Y0 ] from a
k

similar argument to (15) and (ii) Eψ∗ 0α [∇λ4µ gt∗ /gt∗ |Y0 ] = b(α)Eϑ∗ [∇λ2σ gk∗ |Y0 ] from (27). Therefore,
part (b) is proven.
Lemma 8. Suppose the assumptions of Proposition 11 hold. Then, there exist %̄1 , %̄2 ∈ (0, %) such
that, for all k ≥ 1,
k−1

(a)

(b)

∇λ3µ pψ∗ π (Yk |Y0

)

k−1
pψ∗ π (Yk |Y0 )
k−1
∇λ4µ pψ∗ π (Yk |Y0 )
k−1
pψ∗ π (Yk |Y0 )

k−1

∇% ∇λ3µ pψ∗ %̄1 α (Yk |Y0
∇µ3 fk∗
= α(1 − α)(1 − 2α)
+%
∗
k−1
fk
p ∗ (Yk |Y )
ψ %̄1 α

= α(1 − α)(1 − 6α +

∇µ4 fk∗
6α2 )
fk∗

)
,

0

k−1

+%

∇% ∇λ4µ pψ∗ %̄2 α (Yk |Y0
k−1
pψ∗ %̄2 α (Yk |Y0 )

)
.

Proof of Lemma 8. The proof is similar to the proof of Lemma 7(a). From an argument similar to
57

the proof of Lemma 7, the stated results hold if
k−1

)/pψ∗ 0α (Yk |Y0

k−1

)/pψ∗ 0α (Yk |Y0

(A) ∇λ3µ pψ∗ 0α (Yk |Y0
(B) ∇λ4µ pψ∗ 0α (Yk |Y0

k−1

) = α(1 − α)(1 − 2α)∇µ3 fk∗ /fk∗ ,

k−1

) = α(1 − α)(1 − 6α + 6α2 )∇µ4 fk∗ /fk∗ .

Observe that equalities (101) and (103) in the proof of Lemma 7 still hold under the assumptions
of Proposition 11 if we use (41) in place of (27). Consequently, (A) and (B) follow from (40), (41),
and the argument of the proof of Lemma 7, and the stated result follows.
Lemma 9. Suppose that the assumptions of Propositions 8 hold. Let Cη be a set of sequences {ηn }
Q
√
satisfying n(ηn − η ∗ ) → hη for some finite hη . Let Pnηn := nk=1 fk (ηn , 0) denote the probability
measure under ηn with λn = 0. Then, for every sequence {ηn } ∈ Cη , the LRTS under {Pnηn }


converges in distribution sup%∈Θ% t̃0λ% Iλ.η% t̃λ% given in Propositions 8.
√
Proof of Lemma 9. Observe that ϑn := (πn , ηn , λn ) = (π, η ∗ + hη / n, 0) satisfies the assumptions
of Proposition 17. Therefore, Proposition 17 holds under ϑn with νn (s%nk ) →d N (I% h, I% ) with
h = (h0η , 0)0 under Pnϑn . Furthermore, the log-likelihood function of the one-regime model admits
a similar expansion, and log(dPnηn /dPnη∗ ) = h0η νn (sηk ) − (1/2)h0η Iη hη + op (1) holds under Pnηn .


Gh
ηn
h
Therefore, the proof of Proposition 8 goes through by replacing G%n with G%n = Gh
:=
λ%n

G%n + I% h. In view of Ghηn = Gηn + Iη hη and Ghλ%n = Gλ%n + Iλη% hη , we have Ghλ.η%n := Ghλ%n −
Iλη% Iη−1 Ghηn = Gλ%n − Iλη% Iη−1 Gηn = Gλ%n . Therefore, the asymptotic distribution of the LRTS
under Pnηn is the same as that under Pnη∗ , and the stated result follows.
11.2.3

Bounds on difference in state probabilities and conditional moments

Lemma 10. Suppose X1 , . . . , Xn are random variables with max1≤i≤n E|Xi |q < C for some q > 0
and C ∈ (0, ∞). Then, max1≤i≤n |Xi | = op (n1/q ).
P
Proof of Lemma 10. For any ε > 0, we have P(max1≤i≤n |Xi | > εn1/q ) ≤ 1≤i≤n P(|Xi | > εn1/q )
P
≤ ε−q n−1 1≤i≤n E(|Xi |q I{|Xi | > εn1/q }) by a version of Markov inequality. As n → ∞, the
right-hand side tends to 0 by the dominated convergence theorem.
The following lemma extends Corollary 1 and (39) of DMR and an equation on p. 2298 of DMR;
n
DMR derive these results when t1 = t2 and t3 = t4 and W−m
is absent. For the two probability

measures µ1 and µ2 , the total variation distance between µ1 and µ2 is defined as kµ1 − µ2 kT V :=
R
R
supA |µ1 (A)−µ2 (A)|. k·kT V satisfies supf (x):0≤f (x)≤1 | f (x)dµ1 (x)− f (x)dµ2 (x)| = kµ1 −µ2 kT V .
n

n

n

n ), and we let vn
n
In the following, we define V−m := (Y−m , W−m
−m and x−m denote “V−m = v−m ”

and “X−m = x−m .”

58

Lemma 11. Suppose Assumptions 1–2 hold and ϑx ∈ Θx . Then, we have, for all vn−m ,
(a) For all −m ≤ t1 ≤ t2 with −m < n and all probability measures µ1 and µ2 on B(X ),
X

X

Pϑx (Xtt21 ∈ ·|x−m , vn−m )µ1 (x−m ) −

Pϑx (Xtt21 ∈ ·|x−m , vn−m )µ2 (x−m )

x−m ∈X

x−m ∈X

≤ ρt1 +m .
TV

(b) For all −m ≤ t1 ≤ t2 ≤ n − 1,
n−1
Pϑx (Xtt21 ∈ ·|vn−m , x−m ) − Pϑx (Xtt21 ∈ ·|v−m
, x−m )

TV

≤ ρn−1−t2 .

The same bound holds when x−m is dropped from the conditioning variables.
(c) For all −m ≤ t1 ≤ t2 < t3 ≤ t4 with −m < n,
Pϑx (Xtt21 ∈ ·, Xtt43 ∈ ·|vn−m , x−m ) − Pϑx (Xtt21 ∈ ·|vn−m , x−m )Pϑx (Xtt43 ∈ ·|vn−m , x−m )

TV

≤ ρt3 −t2 .

The same bound holds when x−m is dropped from the conditioning variables.
Proof of Lemma 11. We prove part (a) first. We assume t1 > −m because the stated result holds
n
trivially when t1 = −m. Observe that Lemma 1 of DMR still holds when W−m
is added to

the conditioning variable because Assumption 1 implies that {(Xk , Yk )}∞
k=0 is a Markov chain
n

n
given {Wk }∞
k=0 . Therefore, {Xt }t≥−m is a Markov chain when conditioned on {Y −m , W−m }, and
P
hence Pϑx (Xtt21 ∈ A|vn−m , x−m ) = xt ∈X Pϑx (Xtt21 ∈ A|Xt1 = xt1 , vn−m )pϑx (xt1 |vn−m , x−m ) holds.
1

From applying this result and the property of the total variation distance, we can bound the
P
P
left-hand side of the lemma by k x−m ∈X pϑx (Xt1 ∈ ·|vn−m , x−m )µ1 (x−m ) − x−m ∈X pϑx (Xt1 ∈
·|vn−m , x−m )µ2 (x−m )kT V . This is bounded by ρt1 +m from Corollary 1 of DMR, which holds when
n
W−m
is added to the conditioning variable. Therefore, part (a) is proven.

We proceed to prove part (b). Observe that the time-reversed process {Zn−k }0≤k≤n+m is
k−1

n
Markov when conditioned on W−m
and that Wk is independent of (X0k−1 , Y0 ) given W0k−1 .
P
Consequently, for k = n, n − 1, we have Pϑx (Xtt21 ∈ A|vk−m , x−m ) = xt ∈X Pϑx (Xtt21 ∈ A|Xt2 =
2

2
xt2 , vt−m
, x−m )pϑx (xt2 |vk−m , x−m ). Therefore, from the property of the total variation distance, the

n−1
left-hand side of the lemma is bounded by kPϑx (Xt2 ∈ ·|vn−m , x−m ) − Pϑx (Xt2 ∈ ·|v−m
, x−m )kT V .
n
This is bounded by ρn−1−t2 because equation (39) of DMR p. 2294 holds when W−m
is added to the

conditioning variables, and the stated result follows. When x−m is dropped from the conditioning
variables, part (b) follows from a similar argument with using Lemma 9 and an analogue of Corollary
1 of DMR in place of equation (39) of DMR.
Part (c) follows immediately from writing the left-hand side of lemma as supA,B |Pϑx (Xtt21 ∈
A|vn−m , x−m )[Pϑx (Xtt43 ∈ B|vn−m , Xtt21 ∈ A) − Pϑx (Xtt43 ∈ B|vn−m , x−m )]| and applying part (a).

59

11.2.4

The sums of powers of ρ

Lemma 12. For all ρ ∈ (0, 1), c ≥ 1, q ≥ 1, and b > a,
∞ 
X
t=−∞
∞ 
X
t=−∞

 q(c + 1)ρb(b−a)/(c+1)qc
ρb(t−a)/cqc ∧ ρb(b−t)/qc ≤
,
1−ρ
 q(c + 1)ρb(b−a)/(c+1)qc
.
ρb(t−a)/qc ∧ ρb(b−t)/cqc ≤
1−ρ

Proof of Lemma 12. The first result holds because the left-hand side is bounded by
Pb(a+bc)/(c+1)c b(b−t)/qc
P∞
b(t−a)/cqc
≤ qρb{b−b(a+bc)/(c+1)c}/qc /(1 − ρ) +
ρ
+
t=−∞
t=b(a+bc)/(c+1)c+1 ρ
cqρb{b(a+bc)/(c+1)c+1−a}/cqc /(1 − ρ) ≤ q(1 + c)ρb(b−a)/(c+1)qc /(1 − ρ). The second result is proven by
Pb(ac+b)/(c+1)c b(b−t)/cqc P∞
bounding the left-hand side by t=−∞
ρ
+ t=b(ac+b)/(c+1)c+1 ρb(t−a)/qc and proceeding similarly.
The following lemma generalizes the result in the last inequality on p. 2299 of DMR.
Lemma 13. For all ρ ∈ (0, 1), k ≥ 1, q ≥ 1, and n ≥ 0,


X


ρbt1 /qc ∧ ρb(t2 −t1 )/qc ∧ · · · ∧ ρb(tk −tk−1 )/qc ∧ ρb(n−tk )/qc ≤ Ckq (ρ)ρbn/2kqc ,

0≤t1 ≤t2 ≤···≤tk ≤n

where Ckq (ρ) := q k k(k + 1)!(1 − ρ)−k .
Proof of Lemma 13. When k = 1, the stated result follows from Lemma 12 with c = 1. We first
show that the following holds for k ≥ 2:


X
t1 ≤t2 ≤···≤tk ≤n

 q k−1 (k + 1)!ρb(n−t1 )/kqc
.
ρb(t2 −t1 )/qc ∧ · · · ∧ ρb(tk −tk−1 )/qc ∧ ρb(n−tk )/qc ≤
(1 − ρ)k−1

(104)

We prove (104) by induction. When k = 2, it follows from Lemma 12 with c = 1 that
Pn
b(t2 −t1 )/qc ∧ ρb(n−t2 )/qc ) ≤ 2qρb(n−t1 )/2qc /(1 − ρ), giving (104). Suppose (104) holds when
t2 =t1 (ρ
k = `. Then (104) holds when k = ` + 1 because, from Lemma 12,


X

ρb(t2 −t1 )/qc ∧ ρb(t3 −t2 )/qc ∧ · · · ∧ ρb(t`+1 −t` )/qc ∧ ρb(n−t`+1 )/qc



t1 ≤t2 ≤···≤t` ≤t`+1 ≤n

≤

n
X





ρb(t2 −t1 )/qc ∧

≤
≤

(1 − ρ)`−1





ρb(t3 −t2 )/qc ∧ · · · ∧ ρb(t`+1 −t` )/qc ∧ ρb(n−t`+1 )/qc 

t2 ≤···≤t`+1 ≤n

t2 =t1

q `−1 `!

X

n
X



ρb(t2 −t1 )/qc ∧ ρb(n−t2 )/`qc

t2 =t1

q ` (`

+ 1)! b(n−t1 )/(`+1)qc
ρ
,
(1 − ρ)`

60



and hence (104) holds for all k ≥ 2. We proceed to show the stated result. Observe that


X

ρbt1 /qc ∧ ρb(t2 −t1 )/qc ∧ · · · ∧ ρb(tk −tk−1 )/qc ∧ ρb(n−tk )/qc



0≤t1 ≤t2 ≤···≤tk ≤n
n/2
X

≤2

X

n−t
X1



ρbt1 /qc ∧ ρb(t2 −t1 )/qc ∧ · · · ∧ ρb(tk −tk−1 )/qc ∧ ρb(n−tk )/qc



ρb(t2 −t1 )/qc ∧ · · · ∧ ρb(tk −tk−1 )/qc ∧ ρb(n−tk )/qc



t1 =0 t1 ≤t2 ≤···≤tk−1 ≤tk tk =t1
n/2
X

=2

X

n−t
X1



t1 =0 t1 ≤t2 ≤···≤tk−1 ≤tk tk =t1
n/2
X

≤2



X


ρb(t2 −t1 )/qc ∧ · · · ∧ ρb(tk −tk−1 )/qc ∧ ρb(n−tk )/qc ,

t1 =0 t1 ≤t2 ≤···≤tk−1 ≤tk ≤n

where the first inequality holds by symmetry, and the subsequent equality follows from n − tk ≥ t1 .
Pn/2
From (104), the right-hand side is no larger than q k−1 (k + 1)!(1 − ρ)(1−k) t1 =0 ρb(n−t1 )/kqc ≤
q k k(k + 1)!(1 − ρ)−k ρbn/2kqc , giving the stated result.
The next lemma generalizes equation (46) and p. 2294 of DMR, who derive a similar bound
when ` = 1, 2.
Lemma 14. Let aj > 0 for all j. For all positive integer ` ≥ 1 and all k ≥ 1 and m ≥ 0, we have
P
−2 `
max−m+1≤t1 ,...,t` ≤k at1 · · · at` ≤ (k + m)`+1 A` , where A` := ∞
t=−∞ (|t| ∨ 1) at .
P
Proof of Lemma 14. When ` = 1, the stated result follows from max−m+1≤t≤k at ≤ kt=−m+1 at =
P∞
Pk
−2
2
−2
2
t=−∞ (|t|∨1) at . When ` ≥ 2, from Hölder’s inequality,
t=−m+1 (|t|∨1) (|t|∨1) at ≤ (k +m)
P
Pk
we have max−m+1≤t1 ≤...≤t` ≤k at1 at2 · · · at` ≤ ( t=−m+1 at )` = [ kt=−m+1 (|t|∨1)2/` (|t|∨1)−2/` at ]` ≤
P
P
[ kt=−m+1 (|t| ∨ 1)2/(`−1) ](`−1) kt=−m+1 (|t| ∨ 1)−2 a`t ≤ [(k + m)1+2/(`−1) ]`−1 A` = (k + m)`+1 A` .
The following lemma generalizes the bound derived on p. 2301 of DMR.
α
Lemma 15. For α > 0, q > 0, and cjt ≥ 0, define c∞
jq (ρ ) :=

k ≥ 1, and 0 ≤ m ≤
−m
X

P∞

bα|t|/qc c .
jt
t=−∞ ρ

For all ρ ∈ (0, 1),

m0 ,
X



ρb(k−1−t6 )/qc ∧ ρb(t6 −t5 )/qc ∧ ρb(t5 −t4 )/qc ∧ ρb(t4 −t3 )/qc ∧

t1 =−m0 +1 t1 ≤t2 ≤t3 ≤t4 ≤t5 ≤t6 ≤k
b(t3 −t2 )/qc

ρ

b(t2 −t1 )/qc

∧ρ

6
Y

cjtj ≤

ρb(k−1+m)/2qa7 c c∞
1q

j=1



1/2a7

ρ

6
Y

(105)
c∞
jq



1/4aj

ρ



,

j=2

where (aj , bj ) are defined recursively with (a2 , b2 ) = (1, 1) and, for j ≥ 3, aj+1 = 4aj (aj +bj )/(2aj −
1) and bj+1 = aj (4bj − 1)/(2aj − 1). aj and bj satisfy aj , bj ≥ 3/2 for all j. Direct calculations
.
using Matlab produce a7 = 334.5406.

61

Proof of Lemma 15. First, observe that the following result holds for a, b > 1/4, t1 ≤ 0, and
tj , tj+1 ≥ t1 :
atj+1 + t1
,
a+b
atj+1 + t1
(b) if tj ≥
,
a+b

(a) if tj ≤

|tj |
a(4a + 1)tj+1 + (2a − 1)t1
≤
− tj ,
4a
4a(a + b)
|tj |
a(4b − 1)tj+1 + (2a + 4b + 1)t1
b
then
≤ tj −
.
4a
a
4a(a + b)
then

(106)

(a) holds because (i) when tj ≤ 0, we have tj ≤ (atj+1 + t1 )/(a + b) ⇒ (4a − 1)tj /4a ≤ [a(4a −
1)tj+1 + (4a − 1)t1 ]/4a(a + b) ⇒ −tj /4a ≤ [a(4a − 1)tj+1 + (4a − 1)t1 ]/4a(a + b) − tj and a(4a −
1)tj+1 + (4a − 1)t1 ≤ a(4a − 1)tj+1 + (4a − 1)t1 + 2a(tj+1 − t1 ) = a(4a + 1)tj+1 + (2a − 1)t1 ; (ii) when
tj ≥ 0, we have tj ≤ (atj+1 + t1 )/(a + b) ⇒ (4a + 1)tj /4a ≤ [a(4a + 1)tj+1 + (4a + 1)t1 ]/4a(a + b) ⇒
tj /4a ≤ [a(4a + 1)tj+1 + (4a + 1)t1 ]/4a(a + b) − tj and (4a + 1)t1 ≤ (2a − 1)t1 .
(b) holds because (i) when tj ≤ 0, we have tj ≥ (atj+1 + t1 )/(a + b) ⇒ (4b + 1)tj /4a ≥
[a(4b + 1)tj+1 + (4b + 1)t1 ]/4a(a + b) ⇒ −tj /4a ≤ btj /a − [a(4b + 1)tj+1 + (4b + 1)t1 ]/4a(a + b) and
a(4b+1)tj+1 +(4b+1)t1 ≥ a(4b+1)tj+1 +(4b+1)t1 −2a(tj+1 −t1 ) = a(4b−1)tj+1 +(2a+4b+1)t1 ; (ii)
when tj ≥ 0, we have tj ≥ (atj+1 + t1 )/(a + b) ⇒ (4b − 1)tj /4a ≥ [a(4b − 1)tj+1 + (4b − 1)t1 ]/4a(a +
b) ⇒ tj /4a ≤ btj /a − [a(4b − 1)tj+1 + (4b − 1)t1 ]/4a(a + b) and a(4b − 1)tj+1 + (4b − 1)t1 ≥
a(4b − 1)tj+1 + (2a + 4b + 1)t1 .
We proceed to derive the stated bound. It follows from (a) and (b) and bx + yc ≥ bxc + byc
that, with tj = (aj tj+1 + t1 )/(aj + bj ),
k
X




ρb(tj+1 −tj )/qc ∧ ρb(bj tj −t1 )/aj qc cjtj

tj =−m0 +1

≤ρ

a (4b −1)tj+1 −(2aj −1)t1
c
b j j 4a (a
j j +bj )q


X


t
a (4a +1)tj+1 +(2aj −1)t1
− qj
b j j 4a (a
j j +bj )q

ρ

tj ≤tj
a (4b −1)tj+1 −(2aj −1)t1
b j j 4a (a
c
j j +bj )q



1/4aj
c∞
jq ρ
b
t
−t


b j+1a j+1 1 c ∞
j+1
=ρ
cjq ρ1/4aj .

≤ρ

c

+

X

a (4b −1)t
+(2aj +4bj +1)t1
b
c
b a jq tj − j j 4aj+1
j
j (aj +bj )q

ρ


 cjtj

tj ≥tj


(107)

Observe that aj+1 ≥ 2aj ≥ 2 and bj+1 ≥ 2bj − (1/2) ≥ 3/2 for all j ≥ 2. Therefore, we can apply
(106) and (107) to the left-hand side of (105) sequentially for j = 2, 3, . . . , 6. Consequently, the
left-hand side of (105) is no larger than
−m
X

b

ρ

b7 (k−1)−t1
c
a7 q

c1t1

t1 =−m0 +1

6
Y



1/4aj
c∞
.
jq ρ

j=2

Observe that |t1 | ≤ k − 1 − 2t1 − m because t1 ≤ −m ⇒ −t1 ≤ −2t1 − m ≤ k − 1 − 2t1 − m. From

62

b7 (k − 1) ≥ k − 1 and |t1 | ≤ k − 1 − 2t1 − m, the sum is bounded by
−m
X

b

ρ

k−1−t1
c
a7 q

b k−1+m
c
2a q

c1t1 = ρ

7

t1 =−m0 +1

−m
X

b

ρ

k−1−2t1 −m
c
2a7 q

c1t1 ≤

b k−1+m c
ρ 2a7 q c∞
1q



ρ

1/2a7



,

t1 =−m0 +1

and the stated result follows.
11.2.5

0 )0 and π
0 0
Derivation of ϑM0 +1,x = (ϑ0xm , πxm
xm = (%m , αm , φm )

Define J m0 := {1, . . . , M0 } \ Jm , and let pj and p∗j denote PϑM0 +1 (Xk = j) and Pϑ∗M (Xk = j),
0

respectively.
We parameterize the transition probability of Xk in terms of its stationary distribution and the
first to the (m − 1)-th rows and the (m + 1)-th to the (M0 + 1)-th rows of its transition matrix.11
For i ∈ J m , we reparameterize (pim , pi,m+1 ) to piJ = pim + pi,m+1 = PϑM0 +1 (Xk ∈ Jm |Xk−1 = i)
and pim|iJ = pim /(pim + pi,m+1 ). Furthermore, we reparameterize (pm , pm+1 ) in the stationary
distribution to pJ = pm + pm+1 = PϑM0 +1 (Xk ∈ Jm ) and pm|J = pm /(pm + pm+1 ) = PϑM0 +1 (Xk =
m|Xk ∈ Jm ). Therefore, with ∧ and ∨ denoting “and” and “or,” the transition probability of Xk
0
is summarized by ϑM0 +1,x := ({piJ , pim|iJ }i∈J m , {pij }i∈J m ∧j∈J m0 , {pm+1,j }M
j=1 , {pj }j∈J m0 , pJ , pm|J ).

0 )0 , where ϑ
Split ϑM0 +1,x as ϑM0 +1,x = (ϑ0xm , πxm
xm := ({pij }i∈J m ∧j∈J m0 , {piJ }i∈J m , {pj }j∈J m0 , pJ )
0
and πxm := ({pim|iJ }i∈J m , {pm+1,j }M
j=1 , pm|J ). When the m-th and (m+1)-th regimes are combined

into one regime, the transition probability of Xk equals the transition probability of Xk under ϑ∗M0 ,x
if and only if ϑxm = ϑ∗xm := {pij = p∗ij for i ∈ J m ∧(1 ≤ j ≤ m−1); pij = p∗i,j−1 for i ∈ J m ∧(m+2 ≤
j ≤ M0 ); piJ = p∗ for i ∈ J¯m ; pj = p∗ for 1 ≤ j ≤ m − 1; pj = p∗ for m + 2 ≤ j ≤ M0 ; pJ = p∗m }.
im

j

j−1

πxm is the part of ϑM0 +1,x that is not identified under H0m .
We proceed to derive the reparameterization of some elements of πxm in terms of (αm , %m ). First,
map pm+1,m and pm+1,m+1 to pm+1,J := pm+1,m + pm+1,m+1 = PϑM0 +1 (Xk ∈ J|Xk−1 = m + 1) and
pm+1,m|J := pm+1,m /pm+1,J = PϑM0 +1 (Xk = m|Xk ∈ J, Xk−1 = m + 1). Let PJ and πJ denote
the transition matrix and stationary distribution of Xk restricted to lie in Jm . The second row
of PJ is given by (pm+1,m|J , 1 − pm+1,m|J ), and πJ is given by (pm|J , 1 − pm|J ). From the relation
πJ = πJ PJ , we can obtain the first row of PJ as a function of pm+1,m|J and pm|J . Finally, the
elements of PJ are mapped to (%m , αm ) as in Section 6.

11
Suppose a Markov process has a transition probability P and stationary distribution π whose elements are strictly
positive. If π and all the rows of P except for one are identified, then the remaining row of P is identified from the
relation πP = π.

63

References
Andrews, D. W. K. (1999), “Estimation When a Parameter is on a Boundary,” Econometrica, 67,
1341–1383.
— (2001), “Testing when a Parameter is on the Boundary of the Maintained Hypothesis,” Econometrica, 69, 683–734.
Andrews, D. W. K. and Ploberger, W. (1994), “Optimal Tests when a Nuisance Parameter is
Present Only Under the Alternative,” Econometrica, 62, 1383–1414.
— (1995), “Admissibility of the Likelihood Ratio Test When a Nuisance Parameter is Present Only
Under the Alternative,” Annals of Statistics, 23, 1609–1629.
Ang, A. and Bekaert, G. (2002), “International Asset Allocation with Regime Shifts,” Review of
Financial Studies, 15, 1137–1187.
Ang, A. and Timmermann, A. (2012), “Regime Changes and Financial Markets,” Annual Review
of Financial Economics, 4, 313–337.
Bianchi, F. (2013), “Regime Switches, Agents’ Beliefs, and Post-World War II U.S. Macroeconomic
Dynamics,” Review of Economic Studies, 80, 463–490.
Bickel, P. J., Ritov, Y., and Rydén, T. (1998), “Asymptotic normality of the maximum-likelihood
estimator for general hidden Markov models,” Annals of Statistics, 26, 1614–1635.
Carrasco, M., Hu, L., and Ploberger, W. (2014), “Optimal Test for Markov Switching Parameters,”
Econometrica, 82, 765–784.
Carter, A. V. and Steigerwald, D. G. (2012), “Testing for Regime Switching: A Comment,” Econometrica, 80, 1809–1812.
Chen, J. and Li, P. (2009), “Hypothesis Test for Normal Mixture Models: The EM Approach,”
Annals of Statistics, 37, 2523–2542.
Chen, J., Li, P., and Fu, Y. (2012), “Inference on the Order of a Normal Mixture,” Journal of the
American Statistical Association, 107, 1096–1105.
Chen, X., Ponomarevab, M., and Tamer, E. (2014), “Likelihood Inference in Some Finite Mixture
Models,” Journal of Econometrics, 182, 87–99.
Chesher, A. (1984), “Testing for Neglected Heterogeneity.” Econometrica, 52, 865–872.
Cho, J. S. and White, H. (2007), “Testing for Regime Switching,” Econometrica, 75, 1671–1720.
Dannemann, J. and Holtzmann, H. (2008), “Testing for Two States in a Hidden Markov Model,”
Canadian Journal of Statistics, 36, 505–520.
64

Davies, R. (1977), “Hypothesis Testing When a Nuisance Parameter is Present Only Under the
Alternative,” Biometrika, 64, 247–254.
— (1987), “Hypothesis Testing When a Nuisance Parameter is Present Only Under the Alternative,” Biometrika, 74, 33–43.
Douc, R. and Matias, C. (2001), “Asymptotics of the Maximum Likelihood Estimator for General
Hidden Markov Models,” Bernoulli, 7, 381–420.
Douc, R., Moulines, É., and Rydén, T. (2004), “Asymptotic Properties of the Maximum Likelihood
Estimator in Autoregressive Models with Markov Regime,” Annals of Statistics, 32, 2254–2304.
Dufour, J.-M. and Luger, R. (2017), “Identification-Robust Moment-Based Tests for Markov
Switching in Autoregressive Models,” Econometric Reviews, 36, 713–727.
Durrett, R. (2010), Probability: Theory and Examples, Cambridge University Press, 4th ed.
Francq, C. and Roussignol, M. (1998), “Ergodicity of Autoregressive Processes with MarkovSwitching and Consistency of the Maximum-Likelihood Estimator,” Statistics, 32, 151–173.
Garcia, R. (1998), “Asymptotic Null Distribution of the Likelihood Ratio Test in Markov Switching
Models,” International Economic Review, 39, 763–788.
Gassiat, E. and Keribin, C. (2000), “The Likelihood Ratio Test for the Number of Components in
a Mixture with Markov Regime,” ESAIM: Probability and Statistics, 4, 25–52.
Gu, J., Koenker, R., and Volgushev, S. (2017), “Testing for Homogeneity in Mixture Models,”
Forthcoming in Econometric Theory.
Hamilton, J. D. (1989), “A New Approach to the Economic Analysis of Nonstationary Time Series
and the Business Cycle,” Econometrica, 57, 357–384.
— (1994), Time Series Analysis, Princeton University Press.
— (2005), “What’s Real About the Business Cycle?” Federal Reserve Bank of St. Louis Review,
87, 435–452.
— (2008), “Regime Switching Models,” in The New Palgrave Dictionary of Economics, eds. Durlauf,
S. N. and Blume, L. E., Palgrave Macmillan, 2nd ed.
— (2016), “Macroeconomic Regimes and Regime Shifts,” in Handbook of Macroeconomics, eds.
Taylor, J. B. and Uhlig, H., Elsevier, vol. 2, chap. 3.
Hamilton, J. D. and Susmel, R. (1994), “Autoregressive Conditional Heteroskedasticity and
Changes in Regime.” Journal of Econometrics, 64, 307–333.

65

Hansen, B. (1992), “The Likelihood Ratio Test Under Non-Standard Conditions: Testing the
Markov Switching Model of GNP,” Journal of Applied Econometrics, 7, 61–82.
— (1996a), “Inference When a Nuisance Parameter is not Identified under the Null Hypothesis,”
Econometrica, 64, 413–430.
— (1996b), “Stochastic Equicontinuity for Unbounded Dependent Heterogeneous Arrays,” Econometric Theory, 12, 347–359.
Hartigan, J. (1985), “Failure of Log-likelihood Ratio Test,” in Proceedings of the Berkeley Conference in Honor of Jerzy Neyman and Jack Kiefer, eds. Le Cam, L. and Olshen, R., Berkeley:
University of California Press, vol. 2, pp. 807–810.
Ho, N. and Nguyen, X. (2016), “Convergence Rates of Parameter Estimation for Some Weakly
Identifiable Finite Mixtures,” Annals of Statistics, 44, 2726–2755.
Jensen, J. L. and Petersen, N. V. (1999), “Asymptotic normality of the Maximum Likelihood
Estimator in State Space Models,” Annals of Statistics, 27, 514–535.
Kahn, J. A. and Rich, R. W. (2007), “Tracking the New Economy: Using Growth Theory to Detect
Changes in Trend Productivity,” Journal of Monetary Economics, 54, 1670–1701.
Kasahara, H. and Shimotsu, K. (2015), “Testing the Number of Components in Normal Mixture
Regression Models,” Journal of the American Statistical Association, 110, 1632–1645.
— (2017), “Asymptotic Properties of the Maximum Likelihood Estimator in Regime Switching
Econometric Models,” Preprint, University of British Columbia.
Krishnamurthy, V. and Rydén, T. (1998), “Consistent Estimation of Linear and Non-Linear Autoregressive Models with Markov Regime,” Journal of Time Series Analysis, 19, 291–307.
Le Gland, F. and Mevel, L. (2000), “Exponential Forgetting and Geometric Ergodicity in Hidden
Markov Models,” Mathematics of Control, Signals, and Systems, 13, 63–93.
Lee, L.-F. and Chesher, A. (1986), “Specification Testing when Score Test statistics are Identically
Zero.” Journal of Econometrics, 31, 121–149.
Lehmann, E. L. and Romano, J. P. (2005), Testing Statistical Hypotheses, Springer, 3rd ed.
Leroux, B. G. (1992), “Maximum-Likelihood Estimation for Hidden Markov Models,” Stochastic
Processes and their Applications, 40, 127–143.
Levin, D. A., Peres, Y., and Wilmer, E. L. (2009), Markov Chains and Mixing Times, American
Mathematical Society.

66

Liu, X. and Shao, Y. (2003), “Asymptotics for Likelihood Ratio Tests under Loss of Identifiability,”
Annals of Statistics, 31, 807–832.
Louis, T. A. (1982), “Finding the Observed Information Matrix When Using the EM Algorithm,”
Journal of the Royal Statistical Society Series B, 44, 226–233.
Marmer, V. (2008), “Testing the Null Hypothesis of No Regime Switching with an Application to
GDP Growth Rates,” Empirical Economics, 35, 101–122.
Morley, J. and Piger, J. (2012), “The Asymmetric Business Cycle,” Review of Economics and
Statisitcs, 94, 208–221.
Newey, W. K. and McFadden, D. L. (1994), “Large Sample Estimation and Hypothesis Testing,”
in Handbook of Econometrics, Amsterdam: North-Holland, vol. 4, pp. 2111–2245.
Okimoto, T. (2008), “New Evidence of Asymmetric Dependence Structures in International Equity
Markets,” Journal of Financial and Quantitative Analysis, 43, 787–815.
Pollard, D. (1990), Empirical Processes: Theory and Applications, vol. 2 of CBMS Conference
Series in Probability and Statistics, Hayward, CA: Institute of Mathematical Statistics.
Qu, Z. and Zhuo, F. (2017), “Likelihood Ratio Based Tests for Markov Regime Switching,” Preprint,
Boston Unversity.
Rotnitzky, A., Cox, D. R., Bottai, M., and Robins, J. (2000), “Likelihood-based Inference with
Singular Information Matrix,” Bernoulli, 6, 243–284.
Schorfheide, F. (2005), “Learning and Monetary Policy Shifts,” Review of Economic Dynamics, 8,
392–419.
Sims, C. and Zha, T. (2006), “Were There Regime Switches in U.S. Monetary Policy?” American
Economic Review, 96, 54–81.
Woodbury, M. A. (1971), “Discussion of Paper by Hartley and Hocking,” Biometrics, 27, 808–817.

67

Table 1: Rejection frequencies (%) under the null hypothesis at the nominal 10%, 5%, and 1%
levels
H0 : M = 1
Model 1
Model 2
Test
10%
5%
1%
10%
5%
1%
n = 200 LRT
10.43 4.63 1.13 10.17 5.27 1.00
supTS
9.87 5.10 0.93 9.63 4.67 0.90
10.03 4.97 1.03
—
—
—
QLRT
n = 500 LRT
8.80 4.03 0.67 9.13 4.30 1.23
9.50 4.57 0.60 9.23 5.07 0.90
supTS
QLRT
9.07 4.43 0.80
—
—
—
H0 : M = 2
LRT
Model 1
Model 2
5%
1%
10%
5%
1%
(p11 , p22 ) 10%
n = 200 (0.5, 0.5) 12.06 7.16 1.70 10.57 4.87 0.80
(0.7, 0.7) 11.97 6.07 1.70 9.63 4.53 1.07
n = 500 (0.5, 0.5) 9.77 4.43 0.70 8.37 3.90 0.73
(0.7, 0.7) 8.40 4.20 0.70 9.63 4.80 0.63
Notes: We use 199 bootstrap samples and 3000 replications. For testing H0 : M = 2 using Models
1 and 2, we generate the data under (β, µ1 , µ2 , σ) = (0.5, −1, 1, 1) and
(β, µ1 , µ2 , σ1 , σ2 ) = (0.5, −1, 1, 0.9, 1.2), respectively.

Table 2: Rejection frequencies (%) for testing H0 : M = 1 under the alternative hypothesis
Model 1
Model 2
(p11 , p22 )
Test
µ1 = 0.20 µ1 = 0.6 µ1 = 1.0 µ1 = 0.20 µ1 = 0.6 µ1 = 1.0
(0.25, 0.25) LRT
4.87
46.90
99.63
16.40
78.00
99.97
supTS
6.23
56.43
95.90
16.37
70.97
95.37
QLRT
5.10
8.00
55.27
—
—
—
(0.50, 0.50) LRT
3.80
7.03
67.87
13.70
43.77
92.77
supTS
4.07
4.40
4.60
14.70
35.77
35.30
QLRT
4.90
9.40
82.50
—
—
—
(0.70, 0.70) LRT
4.10
10.23
91.07
14.63
51.37
98.17
supTS
4.57
7.40
26.37
14.90
36.20
43.43
QLRT
5.13
8.53
58.73
—
—
—
(0.90, 0.90) LRT
5.33
46.87
99.97
23.27
79.87
100.00
supTS
6.77
13.90
4.40
19.10
41.17
35.30
QLRT
4.83
5.63
5.97
—
—
—
Notes: Nominal level of 5% and n = 500. We use 199 bootstrap samples and 3000 replications.
We set µ2 = −µ1 for both models, (β, σ) = (0.5, 1.0) for Model 1, and (β, σ1 , σ2 ) = (0.5, 1.1, 0.9)
for Model 2.

68

Table 3: Rejection frequencies (%) for testing H0 : M = 2 under the alternative hypothesis
Model 1
Model 2
(µ1 , µ2 , µ3 ) (µ1 , µ2 , µ3 ) (µ1 , µ2 , µ3 ) (µ1 , µ2 , µ3 )
(p11 , p22 , p33 ) = (1, 0, −1) = (2, 0, −2) = (1, 0, −1) = (2, 0, −2)
(0.5, 0.5, 0.5)
5.23
30.80
10.33
60.03
(0.7, 0.7, 0.7)
8.47
94.03
23.10
99.33
Notes: Nominal level of 5% and n = 500. We use 199 bootstrap samples and 3000 replications.
We set (β, σ) = (0.5, 1.0) for Model 1 and (β, σ1 , σ2 , σ3 ) = (0.5, 0.6, 0.9, 1.2) for Model 2. For both
Models 1 and 2, we set pij = (1 − pii )/2 for j 6= i, so that, for example, (p12 , p13 ) = (0.15, 0.15)
when p11 = 0.7.

69

Table 4: Parameter estimates: U.S. GDP per capita growth, 1960Q1–2014Q4

µ1
µ2
µ3
µ4
σ
β

µ1
µ2
µ3
µ4
σ1
σ2
σ3
σ4
β

Panel A: Model 1 with common variance
M =2
M =3
M =4
coeff.
s.e.
coeff.
s.e.
coeff.
s.e.
-0.634 0.200 -0.823 0.151 -2.348 0.649
0.951 0.176 0.692 0.172 -0.330 0.179
–
–
2.023 0.236 0.532 0.161
–
–
–
–
2.025 0.184
0.913 0.053 0.752 0.052 0.832 0.040
0.787 0.041 0.773 0.046 0.639 0.053
Panel B: Model 2 with switching variance
M =2
M =3
M =4
coeff.
s.e.
coeff.
s.e.
coeff.
s.e.
0.370 0.123 -0.643 0.308 -0.698 0.359
0.426 0.178 0.618 0.179 0.580 0.192
–
–
1.826 0.325 1.569 0.523
–
–
–
–
2.218 0.830
0.655 0.063 1.091 0.167 1.041 0.197
1.495 0.138 0.605 0.058 0.578 0.073
–
–
0.892 0.154 0.670 0.282
–
–
–
–
0.879 0.336
0.867 0.036 0.784 0.050 0.783 0.050

Table 5: Selection of the number of regimes: U.S. GDP per capita growth, 1960Q1–2014Q4

M0
1
2
3
4

Model 1 with common variance
LRT
log-like.
AIC
BIC
LRn p-val.
-331.70 669.39 679.58 20.86 0.000
-321.27 656.54
680.29 27.77 0.000
-307.39 640.77 684.89 15.23 0.020
-299.77 641.54
712.81 6.57 0.523

70

Model 2 with switching variance
LRT
log-like.
AIC
BIC
LRn p-val.
-331.70 669.39
679.58 47.25 0.000
-308.07 632.15 659.29 22.14 0.000
-297.01 624.01 674.91 4.87 0.392
-294.57 637.14
718.59
3.01
0.397

Figure 1: The posterior probabilities of each regime (Model 1 with common variance): U.S. GDP
per capita growth, 1960Q1–2014Q4
M=2

M=3

M=4

71

Figure 2: The posterior probabilities of each regime (Model 2 with switching variance): U.S. GDP
per capita growth, 1960Q1–2014Q4
M=2

M=3

M=4

72

