Locally Robust Semiparametric Estimation
Juan Carlos Escanciano†

Hidehiko Ichimura‡

MIT

Indiana University

University of Tokyo

arXiv:1608.00033v1 [] 29 Jul 2016

Victor Chernozhukov∗

Whitney K. Newey§
MIT
July 27, 2016

Abstract
This paper shows how to construct locally robust semiparametric GMM estimators,
meaning equivalently moment conditions have zero derivative with respect to the first
step and the first step does not affect the asymptotic variance. They are constructed by
adding to the moment functions the adjustment term for first step estimation. Locally
robust estimators have several advantages. They are vital for valid inference with machine
learning in the first step, see Belloni et. al. (2012, 2014), and are less sensitive to the
specification of the first step. They are doubly robust for affine moment functions, so
moment conditions continue to hold when one first step component is incorrect. Locally
robust moment conditions also have smaller bias that is flatter as a function of first step
smoothing leading to improved small sample properties. Series first step estimators confer
local robustness on any moment conditions and are doubly robust for affine moments, in
the direction of the series approximation. Many new locally and doubly robust estimators
are given here, including for economic structural models. We give simple asymptotic theory
for estimators that use cross-fitting in the first step, including machine learning.
Keywords: Local robustness, double robustness, semiparametric estimation, bias, GMM.
JEL classification: C13; C14; C21; D24.
∗

Department of Economics, MIT, Cambridge, MA 02139, U.S.A E-mail: vchern@mit.edu.
Department of Economics, Indiana University, Bloomington, IN 47405–7104, U.S.A E-mail: jescanci@indiana.edu.
‡
Faculty of Economics, The University of Tokyo, 7-3-1 Hongo, Bunkyo-ku, Tokyo, 113-0033. E-mail:
ichimura@e.u-tokyo.ac.jp.
§
Department of Economics, MIT, Cambridge, MA 02139, U.S.A E-mail: wnewey@mit.edu.
†

1

1

Introduction

There are many economic parameters that depend on nonparametric or large dimensional first
steps. Examples include games, dynamic discrete choice, average consumer surplus, and treatment effects. This paper shows how to construct GMM estimators that are locally robust to the
first step, meaning equivalently that moment conditions have a zero derivative with respect to
the first step and that estimation of the first step does not affect their influence function.
Locally robust moment functions have several advantages. Belloni, Chen, Chernozhukov,
and Hansen (2012) and Belloni, Chernozhukov, and Hansen (2014) showed that local robustness, also referred to as orthogonality, is important for correct inference about parameters of
interest when machine learning is used in the first step. Locally robust moment conditions
are also nearly correct when the nonparametric part is approximately correct. This robustness
property is appealing in many settings where it may be difficult to get the first step completely
correct. Furthermore, local robustness implies the small bias property analyzed in Newey, Hsieh,
and Robins (1998, 2004; NHR henceforth). As a result asymptotic confidence intervals based
on locally robust moments have actual coverage probability closer to nominal than for other
moments. Also, bias is flatter as a function of first step bias for locally robust estimators than
for other estimators. This tends to make their mean-square error (MSE) flatter as a function
of smoothing also, so their performance is less sensitive to smoothing. In addition, by virtue
of their smaller bias, locally robust estimators have asymptotic MSE that is smaller than other
estimators in important cases and undersmoothing is not required for root-n consistency. Finally, asymptotic variance estimation is straightforward with locally robust moment functions,
because the first step is already accounted for.
Locally robust moment functions are constructed by adding to the moment functions the
terms that adjust for (or account for) first step estimation. This construction gives moment
functions that are locally robust. It leads to new estimators for games, dynamic discrete choice,
average surplus, and other important economic parameters. Also locally robust moments that
are affine in a first step component are globally robust in that component, meaning the moments
continue to hold when that component varies away from the truth. This result allows construction of doubly robust moments in the sense of Scharfstein, Rotnitzky, and Robins (1999) and
Robins, Rotnitzky, and van der Laan (2000) by adding to affine moment conditions an affine
adjustment term. Here we construct many new doubly robust estimators, e.g. where the first
step solves a conditional moment restriction or is a density.
Certain first step estimators confer the small bias property on moment functions that are
not locally robust, including series estimators of mean-square projections (Newey, 1994), sieve
2

maximum likelihood estimators (Shen, 1996, Chen and Shen, 1997), bootstrap bias corrected
first steps (NHR), and higher-order kernels (Bickel and Ritov, 2003). Consequently the inference
advantages of locally robust estimators may be achieved by using one of these first steps. These
first steps only make moment conditions locally robust in certain directions. Locally robust
moments have the small bias property in a wider sense, that the moments are nearly zero as the
first step varies in a general way. This property is important when the first step is chosen by
machine learning or in other very flexible, data based ways; see Belloni et. al. (2014).
First step series estimators have some special robustness properties. Moments without the
adjustment term can be interpreted as locally robust because there is an estimated adjustment
term with average that is identically zero. This property corresponds to first step series estimators conferring local robustness in the direction of the series approximation. Also, first step
series estimators are doubly robust in those directions when the moment functions and first step
estimating equations are affine.
The theoretical and Monte Carlo results of NHR show the bias and MSE advantages of locally robust estimators for linear functionals of a density. The favorable properties of a twicing
kernel first step versus a standard first step found there correspond to favorable properties of
locally robust moments versus original moments, because a twicing kernel estimator is numerically equivalent to adding an estimated adjustment term. The theoretical results show that
using locally robust moment conditions increases the rate at which bias goes to zero but only
raises the variance constant, and so leads to improved asymptotic MSE. The Monte Carlo results
show that the MSE of the locally robust estimator is much flatter as a function of bandwidth
and has a smaller minimum than the original moment functions, even with quite small samples.
Advantages have also been found in the literature on doubly robust estimation of treatment
effects, as in Bang and Robins (2005) and Firpo and Rothe (2016). These results from earlier work suggest that locally robust moments provide a promising approach to improving the
properties of semiparametric estimators.
This paper builds on other earlier work. Locally robust moment conditions are semiparametric versions of Neyman (1959) C(α) test moments for parametric models, with parametric
extensions to nonlikelihood settings given by Wooldridge (1991), Lee (2005), Bera et. al. (2010),
and Chernozhukov, Hansen, and Spindler (2015). Hasminskii and Ibragimov (1978) suggested
an estimator of a functional of a nonparametric density estimator that can be interpreted as
adding the first step adjustment term. Newey (1990) derived the form of the adjustment term
in some cases. Newey (1994) showed local robustness of moment functions that are derivatives
of an objective function where the first step has been ”concentrated out,” derived the form
of the adjustment term for many important cases, and showed that moment functions based
on series nonparametric regression have small bias. General semiparametric model results on
doubly robust estimators were given in Robins and Rotnitzky (2001).

3

NHR showed that adding the adjustment term gives locally robust moments for functionals
of a density integral and showed the important bias and MSE advantages for locally robust
estimators mentioned above. Robins et. al. (2008) showed that adding the adjustment term
gives local robustness of explicit functionals of nonparametric objects, characterized some doubly
robust moment conditions, and considered higher order adjustments that could further reduce
bias. The form of the adjustment term for first step estimation has been derived for a variety
of first step estimators by Pakes and Olley (1995), Ai and Chen (2003), Bajari, Chernozhukov,
Hong, and Nekipelov (2009), Bajari, Hong, Krainer, and Nekipelov (2010), Ackerberg, Chen,
and Hahn (2012), Ackerberg, Chen, Hahn, and Liao (2014), and Ichimura and Newey (2016),
among others. Locally and doubly robust moments have been constructed for a variety of
estimation problems by Robins, Rotnitzky, and Zhao (1994, 1995), Robins and Rotnitzky (1995),
Scharfstein, Rotznitzky, and Robins (1999), Robins, Rotnitzky, and van der Laan (2000), Robins
and Rotnitzky (2001), Belloni, Chernozhukov, and Wei (2013), Belloni, Chernozhukov, and
Hansen (2014), Ackerberg, Chen, Hahn, and Liao (2014), Firpo and Rothe (2016), and Belloni,
Chernozhukov, Fernandez-Val, and Hansen (2016).
Contributions of this paper are a general construction of locally robust estimators in a
GMM setting, a general nonparametric construction of doubly robust moments, and deriving
bias and other large sample properties. The special robustness properties of first step series
estimators are also shown here. We use these results to obtain many new locally and doubly
robust estimators, such as those where the first step allows for endogeneity or is a conditional
choice probability in an economic structural model. We expect these estimators to have the
advantages mentioned above, that machine learning can be used in the first step, the estimators
have appealing robustness properties, smaller bias and MSE, are less sensitive to bandwidth,
have closer to nominal coverage for confidence intervals, and standard errors that can be easily
computed.
Section 2 describes the general construction of locally robust moment functions for semiparametric GMM. Section 3 shows how the first step adjustment term can be derived and shows the
local robustness of the adjusted moments. Section 4 introduces local double robustness, shows
that affine, locally robust moment functions are doubly robust, and gives new classes of doubly
robust estimators. Section 5 describes how locally robust moment functions have the small bias
property and a smaller remainder term. Section 6 considers first step series estimation. Section
7 characterizes locally robust moments based on conditional moment restrictions. Section 8
gives locally robust moment conditions for conditional choice probability estimation of discrete
game and dynamic discrete choice models. Section 9 gives asymptotic theory based on cross
fitting with easily verifiable regularity conditions for the first step, including machine learning.

4

2

Constructing Locally Robust Moment Functions

The subject of this paper is GMM estimators of parameters where the sample moment functions
depend on a first step nonparametric or large dimensional estimator. We refer to these estimators
as semiparametric. We could also refer to them as GMM where first step estimators are “plugged
in” the moments. This terminology seems awkward though, so we simply refer to them as
semiparametric GMM estimators. We denote such an estimator by β̂, which is a function of the
data z1 , ..., zn where n is the number of observations. Throughout the paper we will assume that
the data observations zi are i.i.d. We denote the object that β̂ estimates as β0 , the subscript
referring to the parameter value under the distribution that generated the data.
To describe the type of estimator we consider let m(z, β, γ) denote an r×1 vector of functions
of the data observation z, parameters of interest β, and a function γ that may be vector valued.
The function γ can depend on β and z through those arguments of m. Here the function γ
represents some possible first step, such as an estimator, its limit, or a true function. A GMM
estimator can be based on a moment condition where β0 is the unique parameter vector satisfying
E[m(zi , β0 , γ0 )] = 0,

(2.1)

and γ0 is the true γ. Here it is assumed that this moment condition identifies β. Let γ̂ denote
some first step estimator of γ0 . Plugging in γ̂ to obtain m(zi , β, γ̂) and averaging over zi gives the
P
estimated sample moments m̂(β) = ni=1 m(zi , β, γ̂)/n. For Ŵ a positive semi-definite weighting
matrix a semiparametric GMM estimator is
β̂ = arg min m̂(β)T Ŵ m̂(β),
β∈B

where AT denotes the transpose of a matrix A and B is the parameter space for β.
√
As usual a choice of Ŵ that minimizes the asymptotic variance of n(β̂ − β0 ) will be
√
a consistent estimator of the inverse of the asymptotic variance of nm̂(β0 ). Of course that
efficient Ŵ may include adjustment terms for the first step estimator γ̂. This optimal Ŵ also
gives an efficient estimator in the wider sense shown in Ackerberg, Chen, Hahn, and Liao (2014).
The optimal Ŵ makes β̂ efficient in a semiparametric model where the only restrictions imposed
are equation (2.1).
To explain and analyze local robustness we consider limits when the true distribution of a
single observation zi is F , and how those limits vary with F over a general class of distributions.
This kind of analysis can be used to derive the asymptotic variance of semiparametric estimators,
as in Newey (1994), and is also useful here. Let γ(F ) denote the limit of γ̂ when F is the true
distribution of zi . Here γ(F ) is understood to be the limit of γ̂ under general misspecification
where F need not satisfy the conditions used to construct γ̂. We also consider parametric models
Fτ where τ denotes a vector of parameters, with Fτ equal to the true distribution F0 at τ = 0.
5

We will restrict each parametric model to be regular in the sense used in the semiparametric
efficiency bounds literature, so that Fτ has a score S(z) (derivative of the log-likelihood in many
cases, e.g. see Van der Vaart, 1998, p. 362) at τ = 0 and possibly other conditions are satisfied.
We also require that the set of scores over all regular parametric family has mean square closure
that includes all functions with mean zero and finite variance. Here we are assuming that the
set of scores for regular parametric models is unrestricted, the precise meaning of the domain of
γ(F ) being a general class of distributions. We define local robustness in terms of such families
of regular parametric models.
Definition 1: The moment functions m(z, β, γ) are locally robust if and only if for all
regular parametric models
∂E[m(zi , β0 , γ(Fτ ))]
= 0.
∂τ
τ =0
This zero pathwise derivative condition means that moment conditions are nearly zero as the
first step limit γ(F ) departs from the truth γ0 along any path γ(Fτ ). Below we use a functional
derivative condition, but for now this pathwise derivative definition is convenient. Throughout
the remainder of the paper we evaluate derivatives with respect to τ at τ = 0 unless otherwise
specified.
In general, locally robust moment functions can be constructed by adding to moment functions the term that adjusts for (or accounts for) first step estimation. Under conditions discussed
below there is a unique vector of functions φ(z, β, γ) such that E[φ(zi , β0 , γ0 )] = 0 and
√

n

n

1 X
1 X
nm̂(β0 ) = √
m(zi , β0 , γ̂) = √
{m(zi , β0 , γ0) + φ(zi , β0 , γ0)} + op (1).
n i=1
n i=1

(2.2)

Here φ(z, β0 , γ0 ) adjusts for the presence of γ̂ in m(z, β0 , γ̂). Locally robust moment functions
can be constructed by adding φ(z, β, γ) to m(z, β, γ) to obtain new moment functions
g(z, β, γ) = m(z, β, γ) + φ(z, β, γ).
For ĝ(β) =

Pn

i=1

(2.3)

g(zi , β, γ̂)/n, a locally robust semiparametric GMM estimator is obtained as
β̂ = arg min ĝ(β)′Ŵ ĝ(β).
β

In a parametric setting it is easy to see how adding the adjustment term for first step
estimation gives locally robust moment conditions. Suppose that the first step estimator γ̂ is
a function of a finite dimensional vector of parameters λ where there is a vector of functions
h(z, λ) satisfying E[h(zi , λ0 )] = 0 and the first step parameter estimator λ̂ satisfies
n

1 X
√
h(zi , λ̂) = op (1).
n i=1
6

(2.4)

For H = ∂E[h(zi , λ)]/∂λ|λ=λ0 the usual expansion gives
√

n

1 X
n(λ̂ − λ0 ) = −H −1 √
h(zi , λ0 ) + op (1).
n i=1

For notational simplicity let the moment functions depend directly on λ (rather than γ (λ)) and
so take the form m(z, β, λ). Let Mλ = ∂E[m(zi , β0 , λ0 )]/∂λ. Another expansion gives
√

n

√
1 X
m(zi , β0 , λ0 ) + Mλ n(λ̂ − λ0 ) + op (1)
nm̂(β0 ) = √
n i=1
n

1 X
{m(zi , β0 , λ0 ) − Mλ H −1 h(zi , λ0 )} + op (1).
=√
n i=1

Here we see that the adjustment term is
φ(z, β, λ) = −Mλ H −1 h(z, λ).

(2.5)

We can add this term to the original moment functions to produce new moment functions of
the form
g(z, β, λ) = m(z, β, λ) + φ(z, β, λ) = m(z, β, λ) − Mλ H −1 h(z, λ).
Local robustness of these moment functions follows by the chain rule and
∂E[g(zi , β0 , λ)]
∂λ

=
λ=λ0

∂E[m(zi , β0 , λ) − Mλ H −1 h(zi , λ)]
∂λ

λ=λ0

= Mλ − Mλ H −1 H = 0.

Neyman (1959) used scores and the information matrix to form such g(z, β, λ) in a parametric
likelihood setting, where g(z, β0 , λ0 ) has an orthogonal projection interpretation. There the
purpose was to construct tests, based on g(z, β, λ), where estimation of the nuisance parameters
λ did not affect the distribution of the tests. The form here was given in Wooldridge (1991) for
nonlinear least squares and Lee (2005), Bera et. al (2010), and Chernozhukov et. al. (2015)
for GMM. What appears to be new here is construction of locally robust moment functions by
adding the adjustment term to original moment functions.
In general the adjustment term φ(z, β, γ) may depend on unknown components that are
not present in the original moment functions m(z, β, γ). In the above parametric example the
matrix Mλ H −1 is unknown so its elements should really be included in γ, along with λ. If we
do this local robustness will continue to hold with these additional components of γ because
E[h(zi , λ0 )] = 0. For notational simplicity we will take γ to be the first step for the locally
robust moment functions g(z, β, γ) = m(z, β, γ) + φ(z, β, γ), with the understanding φ(z, β, γ)
will generally depend on first step functions that are not included in m(z, β, γ).
In general semiparametric settings the form of the adjustment term φ(z, β, γ) and local
robustness of g(z, β, γ) can be explained in terms of influence functions. We will do so in the
7

next Section. In many interesting cases the form of the adjustment term φ(z, β, γ) is already
known, allowing construction of locally robust estimators. We conclude this section with an
important class of examples.
The class of examples we consider is one where the first step γ̂1 is based on a conditional
moment restriction E[ρ(zi , γ10 )|xi ] = 0 for a residual ρ(z, γ1 ) and instrumental variables x. The
conditional mean or median of yi given xi are included as special cases where ρ(z, γ1 ) = y −γ1 (x)
and ρ(z, γ1 ) = 2·1(y < γ1 (x))−1 respectively, as are versions that allow for endogeneity where γ1
depends on variables other than x. We take γ̂1 to have the same limit as the nonparametric twostage least squares (NP2SLS) estimator of Newey and Powell (1989, 2003) and Newey (1991).
Thus, γ̂1 has limit γ1 (F ) satisfying
γ1 (F ) = arg min EF [{EF [ρ(zi , γ1 )|xi ]}2 ],
γ1 ∈Γ

and EF denotes the expectation under the distribution F . Suppose that there is γ20 (x) in the
mean square closure of the set of derivatives ∂E[ρ(zi , γ1 (Fτ ))|xi ]/∂τ as Fτ varies over regular
parametric models such that
∂E[m(zi , β0 , γ1(Fτ ))]
∂E[ρ(zi , γ1 (Fτ ))|xi ]
= −E[γ20 (xi )
].
∂τ
∂τ

(2.6)

Then from Ichimura and Newey (2016) the adjustment term is
φ(z, β, γ) = γ2 (x, β)ρ(z, γ1 ).

(2.7)

A function γ20 (x) satisfying equation (2.6) exists when the set of derivatives ∂E[ρ(zi , γ1 (Fτ ))|xi ]/∂τ
is linear as Fτ varies over parametric models, ∂E[m(zi , β0 , γ1 (Fτ ))]/∂τ is a linear functional
of ∂E[ρ(zi , γ1(Fτ ))|xi ]/∂τ, and that functional is continuous in mean square. Existence of
γ20 (x) then follows from the Riesz representation theorem. Special cases of this characterization of γ20 (x) are in Newey (1994), Ai and Chen (2007), and Ackerberg, Chen, Hahn, and
Liao (2014). When ∂E[m(zi , β0 , γ1 (Fτ ))]/∂τ is not a mean square continuous functional of
∂E[ρ(zi , γ1 (Fτ ))|xi ]/∂τ then first step estimation should make the moments converge slower
√
than 1/ n, as shown by Newey and McFadden (1994) and Severini and Tripathi (2012) for
special cases. The adjustment term given here includes Santos (2011) as a special case with
R
m(z, β, γ1 ) = v(x)γ1 (x)dx − β, though Santos (2011) is more general in allowing for nonidentification of γ10 .
There are a variety of ways to construct an estimator φ(zi , β, γ̂) of the adjustment term to
be used in forming locally robust moment functions, see NHR and Ichimura and Newey (2016).
A relatively simple and general one when the first step is a series or sieve estimator is to treat
the first step as if it were parametric and use the parametric formula in equation (2.5). This
approach to estimating the adjustment term is known to be asymptotically valid in a variety
8

of settings, see Newey (1994), Ackerberg, Chen, and Hahn (2012), and Ichimura and Newey
(2016). For completeness we give a brief description here.
We parameterize an approximation to γ1 as γ1 = γ(λ) where λ is a finite dimensional vector
of parameters as before. Let m(zi , β, λ) = m(zi , β, γ1(λ)) and λ̂i denote an estimator of λ0
solving E[h(zi , λ0 )] = 0, that is allowed to depend on observation i. Being a series or sieve
estimator the dimension of λ and hence of h(z, λ) will increase with sample size. Also, let I(i)
be a set of observation indices that can also depend on i. An estimator of the adjustment term
is give by
φ(zi , β, γ̂) = −M̂iλ (β)Ĥi−1 h(zi , λ̂i ), M̂iλ (β) =

X ∂h(zj , λ̂i )
X ∂m(zj , β, λ̂i )
, Ĥi =
.
∂λ
∂λ

j∈I(i)

j∈I(i)

This estimator allows for cross fitting where λ̂i , M̂iλ , and Ĥi depend only on observations other
than the ith . cross fitting is known to improve performance in some settings, such as in ”leave
one out” kernel estimators of averages, e.g. see NHR. This adjustment term will lead to locally
robust moments in a variety of settings, as further discussed below.

3

Influence Functions, Adjustment Terms, and Local Robustness

Influence function calculations can be used to derive the form of the adjustment term φ(z, β, γ)
and show local robustness of the adjusted moment functions g(z, β, γ) = m(z, β, γ) + φ(z, β, γ).
To explain influence functions note that many estimators are asymptotically equivalent to a
sample average. The object being averaged is the unique influence function. For example, in
equation (2.2) we are assuming that the influence function of m̂(β0 ) is g(z, β0 , γ0 ) = m(z, β0 , γ0 )+
φ(z, β0 , γ0). This terminology is widely used in the semiparametric estimation literature.
In general an estimator µ̂ of a true value µ0 and its influence function ψ(z) satisfy
√

n

1 X
ψ(zi ) + op (1), E[ψ(zi )] = 0, E[ψ(zi )ψ(zi )′ ] exists.
n(µ̂ − µ0 ) = √
n i=1

The function ψ(z) can be characterized in terms of the functional µ(F ) that is the limit of µ̂
under general misspecification where F need not satisfy the conditions used to construct µ̂. As
before, we allow F to vary over a family of regular parametric models where the set of scores for
the family has mean square closure that includes all mean zero functions with finite variance. As
shown by Newey (1994) the influence function ψ(z) is then the unique solution to a derivative
equation of Van der Vaart (1991),
∂µ(Fτ )
= E[ψ(zi )S(zi )], E[ψ(zi )] = 0,
∂τ
9

(3.1)

as Fτ (and hence S(z)) varies over the general family of regularly parametric models. Ichimura
and Newey (2016) also showed that when ψ(z) has certain continuity properties it can be
computed as
∂µ(Fτh ) h
, Fτ = (1 − τ )F0 + τ Ghz ,
(3.2)
ψ(z) = lim
h−→0
∂τ
where Ghz is constructed so that Fτh is in the domain of µ(F ) and Ghz approaches the point mass
at z as h −→ 0.
These results can be used to derive the adjustment term φ(z, β, γ) and to explain local
robustness. Let γ(F ) denote the limit of the first step estimator γ̂ under general misspecification
when a single observation has CDF F, as discussed above. From Newey (1994, pp. 1356-1357) we
know that the adjustment term φ(z, β0 , γ0 ) is the influence function of µ(F ) = E[m(zi , β0 , γ(F ))]
where E[·] denotes the expectation at the truth. Thus φ(z, β, γ) can be calculated as in equation
(3.2) for that µ(F ). Also φ(z, β, γ) satisfies equation (3.1) for µ(F ) = E[m(zi , β0 , γ(F ))], i.e. for
the score S(z) at τ = 0 for any regular parametric model {Fτ },
∂E[m(zi , β0 , γ(Fτ ))]
= E[φ(zi , β0 , γ0 )S(zi )], E[φ(zi , β0 , γ0 )] = 0.
∂τ

(3.3)

Also, φ(z, β0 , γ0) can be computed as
∂E[m(zi , β0 , γ(Fτh ))] h
, Fτ = (1 − τ )F0 + τ Ghz ,
h−→0
∂τ
The characterization of φ(z, β0 , γ0) in equation (3.3) can be used to specify another local
robustness property that is equivalent to Definition 1. We have defined local robustness as the
derivative on the left of the first equality in equation (3.3) being zero for all regular parametric
models. If that derivative is zero for all parametric models then φ(z, β0 , γ0 ) = 0 is the unique
solution to this equation, by the set of scores being mean square dense in the set of mean zero
random variables with finite variance. Also, if φ(z, β0 , γ0) = 0 then the derivative on the left is
always zero. Therefore we have
φ(z, β0 , γ0 ) = lim

Proposition 1: φ(z, β0 , γ0 ) = 0 if and only if m(z, β, γ) is locally robust.
Note that φ(z, β, γ) is the term in the influence function of m̂(β0 ) that accounts for the first
step estimator γ̂. Thus Proposition 1 gives an alternative characterization of local robustness,
that first step estimation does not affect the influence function of m̂(β0 ). This result is a
semiparametric version of Theorem 6.2 of Newey and McFadden (1994). It also formalizes the
discussion in Newey (1994, pp. 1356-1357).
Local robustness of the adjusted moment function g(z, β, γ) = m(z, β, γ) + φ(z, β, γ) follows from Proposition 1 and φ(z, β, γ) being a nonparametric influence function. Because
def
φ(z, β, γ) is an influence function it has mean zero at all true distributions, i.e. µ(F ) =
10

R

φ(z, β0 , γ(F ))F (dz) ≡ 0 identically in F. Consequently the derivative in equation (3.1) is
zero, so that (like Proposition 1) the influence function of µ(F ) is zero. Consequently, under
P
appropriate regularity conditions φ̄ = ni=1 φ(zi , β0 , γ̂)/n has a zero influence function and so
√
nφ̄ = op (1). It then follows that
n

n

√
√
√
1 X
1 X
√
g(zi , β0 , γ̂) = nm̂(β0 ) + nφ̄ = nm̂(β0 ) + op (1) = √
g(zi , β0 , γ0 ) + op (1), (3.4)
n i=1
n i=1
where the last equality follows by equation (2.2). Here we see that the adjustment term is zero
for the moment functions g(z, β, γ). From Proposition 1 with g(z, β, γ) replacing m(z, β, γ) it
then follows that g(z, β, γ) is locally robust.
Proposition 2: For the influence function φ(z, β0 , γ0 ) of µ(F ) = E[m(zi , β0 , γ(F ))] the
adjusted moment function g(z, β, γ) = m(z, β, γ) + φ(z, β, γ) is locally robust.
R
Local robustness of g(z, β, γ) also follows directly from the identity φ(z, β0 , γ(F ))F (dz) ≡ 0
as discussed in the Appendix. Also, the adjusted moments ĝ(β0 ) have the same asymptotic
variance as the original moments, as in the second equality of equation (3.4). That is, adding
φ(z, β, γ) to m(z, β, γ) does not affect the asymptotic variance. Thus the asymptotic benefits
of the locally robust moments are in their higher order properties. Other modifications of
the moments may also improve higher-order properties of estimators, such as the cross fitting
described above (like ”leave on out” in NHR) and the higher order bias corrections in Robins
et. al. (2008) and Cattaneo and Jansson (2014).

4

Local and Double Robustness

The zero derivative condition in Definition 1 is an appealing robustness property in and of itself.
Mathematically a zero derivative is equivalent to the moments remaining closer to zero than τ as
τ varies away from zero. This property can be interpreted as local robustness of the moments to
the value of γ being plugged in, with the moments remaining close to zero as γ varies away from
its true value. Because it is difficult to get nonparametric functions exactly right, especially in
high dimensional settings, this property is an appealing one.
Such robustness considerations, well explained in Robins and Rotnitzky (2001), have motivated the development of doubly robust estimators. For our purposes doubly robust moments
have expectation zero if just one first stage component is incorrect. When there are only two first
stage components this means that the moment conditions hold when only one of the first stage
components is correct. Doubly robust moment conditions allow two chances for the moment
conditions to hold.

11

It turns out that locally robust moment functions are automatically doubly robust in a local
sense that the derivative with respect to each individual, distinct first stage component is zero. In
that way the moment conditions nearly hold as each distinct component varies in a neighborhood
of the truth. Furthermore, when locally robust moment functions are affine functions of a distinct
first step component they are automatically globally robust in that component. Thus, locally
robust moment functions that are affine in each distinct first step are doubly robust.
These observations suggest a way to construct doubly robust moment functions. Starting
with any two step semiparametric moment function we can add the adjustment term to get a
locally robust moment function. When we can choose a first step of that moment function so
that it enters in an affine way the new moment function will be doubly robust in that component.
To give these results we need to define distinct components of γ. A distinct component is
one where there are parametric models Fτ with that component varying in an unrestricted way
but the other components of γ not varying. For a precise definition we will focus on the first
component γ1 of γ = (γ1 , ..., γJ ).
Definition 2: A component γ1 of γ is distinct if and only if there is Fτ such that
γ(Fτ ) = (γ1 (Fτ ), γ20 , ..., γJ0 ),
and γ1 (Fτ ) is unrestricted as Fτ varies across parametric models.
An example is the moment function g(z, β, γ1, γ2 ) = m(z, β, γ1 ) + γ2 (x, β)ρ(z, γ1 ), where
E[ρ(zi , γ10 )|xi ] = 0. In that example the two components γ1 and γ2 are often distinct because γ1
depends only on the conditional distribution of zi given xi and γ20 (x, β) depends on the marginal
distribution of xi in an unrestricted way.
Local robustness means that the derivative must be zero for any model, so in particular it
must be zero for any model where only the distinct component is varying. Thus we have
Proposition 3: If γ1 is distinct then for g(z, β, γ) = m(z, β, γ) + φ(z, β, γ) and regular
parametric models Fτ as in Definition 2,
∂E[g(zi , β0 , γ1(Fτ ), γ20 , ..., γJ0)]
= 0.
∂τ
This result is an application of the simple fact that when a multivariable derivative is zero
the partial derivative must be zero when the variables are allowed to vary in an unrestricted
way. Although this fact is simple, it is helpful in understanding when local robustness holds
for individual components. This means that locally robust moment functions automatically
have a local double robustness property, that the expectation of the moment function remains
nearly zero as each distinct first stage component varies away from the truth. For example, for
12

a first step conditional moment restriction where g(z, β, γ) = m(z, β, γ1 ) + γ2 (x, β)ρ(z, γ1 ), the
conclusion of Proposition 3 is
∂E[m(zi , β0 , γ1 (Fτ )) + γ20 (xi )ρ(zi , γ1 (Fτ ))]
= 0.
∂τ
In fact, this result is implied by equation (2.6), so by construction g(z, β, γ) is already locally robust in γ1 alone. Local robustness in γ2 follows by the conditional moment restriction
E[ρ(zi , γ10 )|xi ] = 0.
Moments that are locally robust in a distinct component γ1 will be globally robust in γ1 if
γ1 enters the moments in an affine way, meaning that for any γ1 and γ = (γ1 , γ20 , ..., γJ0 )′ and
any z,
g(z, β, (1 − τ )γ0 + τ γ) = (1 − τ ) · g(z, β, γ0) + τ · g(z, β, γ).
(4.1)
Global robustness holds because an affine function with zero derivative is constant. For simplicity
we state a result when Fτ can be chosen so that γ(Fτ ) = (1 − τ )γ0 + τ γ though it will hold more
generally. Note that here
E[g(zi , β0 , γ(Fτ ))] = (1 − τ )E[g(zi , β0 , γ0 )] + τ · E[g(zi , β0 , γ)] = τ · E[g(zi , β0 , γ)].
Here the derivative of the moment condition with respect to τ is just E[g(zi , β0 , γ)] so Proposition
3 gives the following result:
Proposition 4: If equation (4.1) is satisfied and there is Fτ with γ(Fτ ) = ((1 − τ )γ10 +
τ γ1 , γ20 , ..., γJ0)′ then E[g(zi , β0 , γ1 , γ20 , ..., γJ0 )] = 0.
Thus we see that locally robust moment functions that are affine in a distinct first step
component are globally robust in that component. This result includes many existing examples
of doubly robust moment functions and can be used to construct new ones.
A general class of doubly robust moment functions that appears to be new and includes
many new and previous examples has first step satisfying a conditional moment restriction
E[ρ(zi , γ10 )|xi ] = 0 where ρ(z, γ1 ) and m(z, β0 , γ1 ) are affine in γ1 . Suppose that E[m(zi , β0 , γ1 )]
is a mean-square continuous linear functional of E[ρ(zi , γ1 )|xi ] for γ1 in a linear set Γ. Then by
the Riesz representation theorem there is γ ∗ (x) in the mean square closure Π of the image of
E[ρ(zi , γ1 )|xi ] such that
E[m(zi , β0 , γ1 )] = −E[γ ∗ (xi )E[ρ(zi , γ1 )|xi ]] = −E[γ ∗ (xi )ρ(zi , γ1 )], γ1 ∈ Γ.

(4.2)

Let γ20 (x) be any function such that γ20 (xi ) − γ ∗ (xi ) is orthogonal to Π and g(z, β, γ) =
m(z, β, γ1 ) + γ2 (x, β)ρ(z, γ1 ). Then E[g(zi , β0 , γ1 , γ20 )] = 0 by the previous equation. It also
follows that E[g(zi , β0 , γ10 , γ2 )] = 0 by E[ρ(zi , γ10 )|xi ] = 0. Therefore g(z, β, γ1, γ2 ) is doubly
robust, showing the following result:
13

Proposition 5: If m(zi , β0 , γ1 ) and ρ(zi , γ1) are affine in γ1 ∈ Γ with Γ linear and
E[m(zi , β0 , γ1 )] is a linear, mean square continuous functional of E[ρ(zi , γ1 )|xi ] then there is
γ20 (x) such that g(z, β, γ1, γ2 ) = m(z, β, γ1 ) + γ2 (x, β)ρ(z, γ1 ) is doubly robust.
Section 3 of Robins et. al. (2008) gives necessary and sufficient conditions for a moment
function to be doubly robust when γ1 and γ2 enter the moment functions as functions evaluated
at observed x. Proposition 5 is complementary to that work in deriving the form of doubly robust
moment functions when the first step satisfies a conditional moment restriction and m(z, β, γ1 )
can depend on the entire function γ1 .
It is interesting to note that γ20 such that E[g(z, β0 , γ1, γ20 )] = 0 for all γ1 ∈ Γ is not unique
when Π does not include all functions of x, the overidentified case of Chen and Santos (2015).
This nonuniqueness can occur when there are multiple ways to estimate the first step γ10 using
the conditional moment restrictions E[ρ(zi , γ10 )|xi ] = 0. As discussed in Ichimura and Newey
(2016), the different γ20 (xi ) correspond to different first step estimators, with γ20 (xi ) = γ ∗ (xi )
corresponding to the NP2SLS estimator.
An important case is a linear conditional moment restrictions setup up like Newey and Powell
(1989, 2003) and Newey (1991) where
ρ(z, γ1 ) = y − γ1 (w), E[yi − γ10 (wi )|xi ] = E[ρ(zi , γ10 )|xi ] = 0.

(4.3)

Consider a moment function equal to m(z, β, γ1 ) = v(w)γ1(w) − β for some known function
v(w), where the parameter of interest is β0 = E[v(wi )γ10 (wi )]. If there is γ̄(x) such that v(wi ) =
E[γ̄(xi )|wi ] then we have
E[m(zi , β0 , γ1 )] = E[v(wi ){γ1(wi ) − γ10 (wi )}] = E[E[γ̄(xi )|wi ]{γ1 (wi ) − γ10 (wi )}]
= E[γ̄(xi ){γ1(wi ) − γ10 (wi )}] = −E[γ̄(xi )ρ(zi , γ1 )].

It follows that g(z, β, γ) = m(z, β, γ1 ) + γ2 (x)ρ(z, γ1 ) is doubly robust for γ20 (x) = γ̄(xi ). Interestingly, the existence of γ̄ with v(wi ) = E[γ̄(xi )|wi ] is a necessary condition for root-n consistent
estimability of β0 as in Severini and Tripathi’s (2012, Lemma 4.1). We see here that a doubly
robust moment condition can always be constructed when this necessary condition is satisfied.
Also, similarly to the above, the γ20 (x) may not be unique.
Corollary 6: If m(z, β, γ1 ) = v(w)γ1(w) − β, equation ( 4.3) is satisfied, and there is γ̄(x)
such that v(w) = E[γ̄(x)|w] then g(z, β, γ1, γ2 ) = v(w)γ1(w) − β + γ2 (x)[y − γ1 (w)] is doubly
robust for γ20 (x) − γ̄(x) orthogonal to Π.
A new example of a doubly robust moment condition corresponds to the weighted average
derivative of γ10 (w) of Ai and Chen (2007). Here m(z, β, γ1 ) = v̄(w)∂γ1 (w)/∂w − β for some
14

function v̄(w). Let f0 (w) be the pdf of wi . Assuming that v̄(w)γ1(w)f0 (w) is zero on the
boundary of the support of wi , integration by parts gives
E[m(zi , β0 , γ1 )] = E[v(wi ){γ1(wi ) − γ10 (wi )}], v(w) = f0 (w)−1 ∂[v̄(w)f0 (w)]/∂w.
Assume that there exists γ̄(x) such that v(wi ) = E[γ̄(xi )|wi ]. Then as in Proposition 5 a doubly
robust moment function is
g(z, β, γ) = v̄(w)

∂γ1 (w)
− β + γ2 (x)[y − γ1 (w)].
∂w

A special case of this example is the doubly robust moment condition for the weighted average
derivative in the exogenous case where wi = xi given in Firpo and Rothe (2016).
Doubly robust moment conditions can be used to identify parameters of interest. In general,
if g(z, β, γ1, γ2 ) is doubly robust and γ20 is identified then β0 may be identified from
E[g(zi , β0 , γ̄1 , γ20 )] = 0,
for any fixed γ̄1 when the solution β0 to this equation is unique.
Proposition 7: If g(z, β, γ1, γ2 ) is doubly robust, γ20 is identified, and for some γ̄1 the
equation E[g(zi , β, γ̄1, γ20 )] = 0 has a unique solution then β0 is identified as that solution.
Applying this result to the NPIV setting gives an explicit formula for certain functionals of
γ10 (w) without requiring that the completeness identification condition of Newey and Powell
(2003) be satisfied, similarly to Santos (2011). Suppose that v(w) is identified, e.g. as for the
weighted average derivative. Since both w and x are observed it follows that a solution γ20 (x)
to v(w) = E[γ20 (x)|w] will be identified if such a solution exists. Plugging in γ̄1 = 0 in the
equation E[g(zi , β0 , γ̄1 , γ20 )] = 0 gives
Corollary 8: If v(wi ) is identified and there exists γ20 (xi ) such that v(wi ) = E[γ20 (xi )|wi ]
then β0 = E[v(wi )γ10 (wi )] is identified as β0 = E[γ20 (xi )yi ].
Note that this result holds without the completeness condition. Identification of β0 =
E[v(wi )γ10 (wi )] for known v(wi ) with v(wi ) = E[γ20 (xi )|wi ] follows from Severini and Tripathi
R
(2006). Santos (2011) gives a related formula for a parameter β0 = ṽ(w)γ20(w)dw. The
formula here differs from Santos (2011) in being an expectation rather Lebesgue integral. An
estimator of β0 could be constructed similarly to Santos (2011), though that is beyond the scope
of this paper.
Another new example of a doubly robust estimator is a weighted average over income values
of an average (across heterogenous individuals) of exact consumer surplus bounds, as in Hausman
and Newey (2016). Here y is quantity consumed, w = x = (x1 , x2 )′ , x1 is price, x2 is income,
15

γ10 (xi ) = E[yi |xi ], price is changing between x̆1 and x̄1 , and B is a bound on the income
effect. Let v2 (x2 ) be some weight function and v1 (x1 ) = 1(x̆1 ≤ x1 ≤ x̄1 )e−B(x1 −x̆1 ) . For the
R
moment function m(z, β, γ1 ) = v2 (x2 ) v1 (u)γ1 (u, x2 )du − β the true parameter β0 is a bound
on the average of equivalent variation over unobserved individual heterogeneity and income. Let
f10 (x1 |x2 ) denote the conditional pdf of x1 given x2 . Note that
Z
E[m(zi , β0 , γ1)] = E[v2 (x2i ) v1 (u)[γ1(u, x2i ) − γ10 (u, x2i )]du]
= E[f10 (x1i |x2i )−1 v1 (x1i )v2 (x2i ){γ1 (xi ) − γ10 (xi )}]

= −E[γ20 (xi )E[yi − γ1 (xi )|xi ]], γ20 (x) = f10 (x1 |x2 )−1 v1 (x1 )v2 (x2 ).

Then it follows by Proposition 5 that a doubly robust moment function is
Z
g(z, β, γ) = v2 (x2 ) v1 (u)γ1 (u, x2 )du − β + γ2 (x)[y − γ1 (x)].
When the moment conditions are formulated so that they are affine in the first step Proposition 4 applies to many previously developed doubly robust moment conditions. Data missing
at random is a leading example. Let β0 be the mean of a variable of interest w where w is not
always observed, y ∈ {0, 1} denote an indicator for w being observed, and x a vector of covariates. Assume w is mean independent of y conditional on covariates x. We consider estimating
β0 using the propensity score P0 (xi ) = Pr(yi = 1|xi ). We specify an affine conditional moment
restriction by letting γ1 (x) = 1/P (x) and ρ(z, γ1 ) = γ1 (x)y − 1. We have β0 = E[γ10 (xi )yi wi ],
as is well known. An affine moment function is then m(z, β, γ1 ) = γ1 (x)yw − β. Note that
E[m(zi , β0 , γ1 )] = E[E[yi wi |xi ]{γ1 (xi ) − γ10 (xi )}] = −E[γ20 (xi )ρ(zi , γ1 )],
γ20 (xi ) = −γ10 (xi )E[yi wi |xi ].

Then Proposition 5 implies that a doubly robust moment function is given by
g(z, β, γ) = γ1 (x)yw − β − γ2 (x)[γ1 (x)y − 1].
This is the well known doubly robust moment function of Robins, Rotnitzky, and Zhao (1994).
This example illustrates how applying Propositions 4 and 5 require specifying the first step
so that the moment functions are affine. These moment conditions were originally shown to be
doubly robust when the first step is taken to be the propensity score P (x). Propositions 4 and
5 only apply when the first step is taken to be 1/P (x). More generally we expect that particular
formulations of the first step may be needed to make the moment functions affine in the first
step and so use Propositions 4 and 5 to derive doubly robust moment functions.
Another general class of doubly robust moment functions depend on the pdf γ1 of a subset of variables xi and are affine in γ1 . An important example of such a moment function
16

R
is the average where β0 = f0 (x)2 dx and m(z, β, γ1 ) = γ1 (x) − β. Another is the density
weighted average derivative (WAD) of Powell, Stock, and Stoker (1989) where m(z, β, γ1 ) =
−2y · ∂γ1 (x)/∂x − β. Assume that E[m(zi , β0 , γ1 )] is a function of γ1 − γ10 that is continuous in
R
1/2
the norm [γ1 (u) − γ10 (u)]2du
. Then by the Riesz representation theorem there is γ20 (x)
with
Z
E[m(zi , β0 , γ1 )] = γ20 (u)[γ1 (u) − γ10 (u)]du.
(4.4)
The adjustment term for m(z, β, γ), as in Proposition 3 of Newey (1994), is φ(z, β, γ) = γ2 (x) −
R
γ2 (u)γ1(u)du. The corresponding locally robust moment function is
Z
g(z, β, γ1, γ2 ) = m(z, β, γ1 ) + γ2 (x) − γ2 (u)γ1(u)du.
(4.5)
This function is affine in γ1 and γ2 separately so when they are distinct Proposition 4 implies
double robustness. Double robustness also follows directly from
Z
Z
Z
E[g(zi , β0 , γ)] = γ20 (u)[γ1(u) − γ10 (u)]du + γ2 (u)γ10(u)du − γ2 (u)γ1(u)du
Z
= − [γ2 (u) − γ20 (u)][γ1 (u) − γ10 (u)]du.
Thus we have the following result:
Proposition 9: If m(zi , β, γ1) is affine in γ1 and E[m(zi , β0 , γ1)] is a linear function of
R
1/2
, then for γ20 (x) from equation
γ1 −γ10 that is continuous in the norm {γ1 (x) − γ10 (x)}2 dx
R
(4.4), g(z, β, γ) = m(z, β, γ1 ) + γ2 (x) − γ2 (u)γ1(u)du is doubly robust.

We can use this result to derive doubly robust moment functions for the WAD. Let δ(xi ) =
E[yi |xi ]γ10 (xi ). Assuming that δ(u)γ1(u) is zero on the boundary, integration by parts gives
Z
E[m(zi , β0 , γ1 )] = −2E[yi ∂γ1 (xi )/∂x] − β0 = 2 [∂δ(u)/∂u]{γ1 (u) − γ10 (u)}du,
so that γ20 (x) = 2∂δ(x)/∂x. A doubly robust moment condition is then
Z
∂γ1 (x)
∂δ(x)
∂δ(u)
g(z, β, γ) = −2y
−β+2
− 2
γ1 (u)du.
∂x
∂x
∂u

The double robustness of this moment condition appears to be a new result. As shown in Newey,
Hsieh, and Robins (1998), a ”delete-one” symmetric kernel estimator based on this moment
function gives the twicing kernel estimator of NHR. Consequently the MSE comparisons of
NHR for twicing kernel estimators with the original kernel estimator correspond to comparison
of a doubly (and locally) robust estimator with one based on unadjusted moment conditions, as
discussed in the introduction.
17

It is interesting to note that Proposition 9 does not require that γ1 and γ2 are distinct first
step components. For the average density γ1 (x) and γ2 (x) both represent the marginal density
of x and so are not distinct. Nevertheless the moment function g(z, β0 , γ) = γ1 (x) − β0 +
R
γ2 (x) − γ1 (u)γ2 (u)du is doubly robust, having zero expectation if either γ1 or γ2 is correct.
This example shows a moment function may be doubly robust even though γ1 and γ2 are not
distinct. Thus, there are doubly robust moment functions that cannot be constructed using
Proposition 4.
All of the results of this Section continue to hold with cross fitting. That is true because the
results of this Section concern the moment and their expectation at various values of the first
step, and not the particular way in which the first step is formed.

5

Small Bias of Locally Robust Moment Conditions

Adding the adjustment term improves the higher order properties of the estimated moments
though it does not change their asymptotic variance. An advantage of locally robust moment
functions is that the effect of first step smoothing bias is relatively small. To describe this
advantage it is helpful to modify the definition of local robustness. In doing so we allow F to
represent a more general object, an unsigned measure (charge). Let k·k denote a seminorm on
F (a seminorm has all the properties of a norm but may be zero when F is not zero). Also, let
F be a set of charges where m(z, β0 , γ(F )) is well defined.
Definition 3: m(z, β, γ) is locally robust if and only if E[m(zi , β0 , γ(F ))] = o(kF − F0 k)
for F ∈ F .
Definition 1 requires that µ(F ) have a zero pathwise derivative. Definition 3 requires a zero
Frechet derivative for the semi-norm k·k, generally a stronger condition than a zero pathwise
derivative. The zero Frechet derivative condition is helpful in explaining the bias properties of
locally robust moment functions.
Generally a first estimator will depend on some vector of smoothing parameters b. This
b could be the bandwidth in a kernel estimator or the inverse of number of terms in a series
estimator. Suppose that the limit of γ̂ for fixed b is γ(Fb ) where Fb is a ”smoothed” version of
the true distribution that approaches the truth F0 as b −→ 0. Then under regularity conditions
m̂(β0 ) will have limit E[m(zi , β0 , γ(Fb ))]. We can think of kFb − F0 k as a measure of the smoothing bias of Fb . Similarly E[m(zi , β0 , γ(Fb ))] is a measure of the bias in the moment conditions
caused by smoothing. The small bias property (SBP) analyzed in NHR is that the expectation
of the moment functions vanishes faster than the nonparametric bias as b −→ 0.
Definition 4: m(z, β, γ) and Fb have the small bias property if and only if E[m(zi , β0 , γ(Fb ))] =
o(kFb − F0 k) as b −→ 0.
18

As long as Fb ∈ F , the set F in Definition 3, locally robust moments will have bias that
vanishes faster than the nonparametric bias kFb − F0 k as b −→ 0. Thus locally robust moment
functions have the small bias property.
Proposition 10: If m(z, β, γ) is locally robust then m(z, β, γ) has the small bias property
for any Fb ∈ F .
Note that the bias of locally robust moment conditions will be flat as a function of the first
step smoothing bias kFb − F0 k as that goes to zero. This flatter moment bias can also make the
MSE flatter, meaning the MSE of the estimator does not depend as strongly on kFb − F0 k for
locally robust moments as for other estimators.
By comparing Definitions 3 and 4 we see that the small bias property is a form of directional
local robustness, with the moment being locally robust in the direction Fb . If the moments are
not locally robust then there will be directions where the bias of the moments is not smaller
than the smoothing bias Fb . Being locally robust in all directions can be important when the
first step is allowed to be very flexible, such as when machine learning is used to construct the
first step. There the first step can vary randomly across a large class of functions making the
use of locally robust moments important for correct inference, e.g. see Belloni, Chernozhukov,
and Hansen (2014).
This discussion of smoothing bias is based on sequential asymptotics where we consider
limits for fixed b. This discussion provides useful intuition but it is also important to consider
asymptotics where b could be changing with the sample size. We can analyze the precise effect
of using locally robust moments by considering an expansion of the average moments. Let m̄ =
Pn
Pn
i=1 m(zi , β0 , γ0 )/n, ḡ =
i=1 g(zi , β0 , γ0 )/n, φ(z) = φ(z, β0 , γ0 ), and F̃ denote the empirical
distribution. We suppose that γ̂ = γ(F̂ ) for some estimator F̂ of the true distribution F0 . Let
µ(F ) = E[m(zi , β0 , γ(F ))]. By adding and subtracting terms we have
Z
m̂(β0 ) = ḡ + R̃1 + R̂2 + R̂3 , R̃1 = φ(z)[F̂ − F̃ ](dz),
(5.1)
Z
R̂2 = µ(F̂ ) − φ(z)F̂ (dz), R̂3 = m̂(β0 ) − m̄ − µ(F̂ ).
R
R
The object φ(z)F̂ (dz) = φ(z)[F̂ − F0 ](dz) is a linearization of µ(F̂ ) = µ(F̂ ) − µ(F0 ) so R̂2 is
a nonlinearity remainder that is second order. Also R̂3 is a stochastic equicontinuity remainder
of a type familiar from Andrews (1994) that is also second order.
The locally robust counterpart ĝ(β0 ) to m̂(β0 ) has a corresponding remainder term that
is asymptotically smaller than R̃1 . To see this let φ̂(z) = φ(z, β0 , γ(F̂ )) and note that the
R
mean zero property of an influence function will generally give φ̂(z)F̂ (dz) = 0. Then by
R
ĝ(β0 ) = m̂(β0 ) + φ̂(z)F̃ (dz) we have
19

Proposition 11: If

R

φ̂(z)F̂ (dz) = 0 then ĝ(β0 ) = ḡ + R̂1 + R̂2 + R̂3 ,
Z
R̂1 = − [φ̂(z) − φ(z)][F̂ − F̃ ](dz).

Comparing this conclusion with equation (5.1) we see that locally robust moments have the
same expansion as the original moments except that the remainder R̃1 has been replaced by
the remainder R̂1 . The remainder R̂1 will be asymptotically smaller than R̃1 under sufficient
regularity conditions. Consequently, depending on cross correlations with other terms, the
locally robust moments ĝ(β0 ) can be more accurate than m̂(β0 ). For instance, as shown by
NHR the locally robust moments for linear kernel averages have a higher order bias term that
converges to zero at a faster rate than the original moments, while only the constant term in
the higher order variance is larger. Consequently, the locally robust estimator will have smaller
MSE asymptotically for appropriate choice of bandwidth. In nonlinear cases the use of locally
robust moments may not lead to an improvement in MSE because nonlinear remainder terms
may be important, see Robins et. al. (2008) and Cattaneo and Jansson (2014). Nevertheless,
using locally robust moments does make smoothing bias small, which can be an important
improvement.
In some settings it is possible to obtain a corresponding improvement by changing the first
step estimator. For example, as mentioned earlier, for linear kernel averages the locally robust
estimator is identical to the original estimator based on a twicing version of the original kernel
(see NHR). The improvement from changing the first step can be explained in relation to the
remainder R̃1 , that is the difference of the integral of φ(z) over the estimated distribution F̂ and
its sample average. Note that F̂ − F̃ will be shrinking to zero so that R̃1 − E[R̃1 ] should be a
second order (stochastic equicontinuity) term. E[R̃1 ] is the most interesting term. If E[F̂ ] = Fb
and integrals can be interchanged then
Z
Z
E[R̃1 ] = φ(z)Fb (dz) = φ(z)[Fb − F0 ](dz).
When a twicing kernel or any other higher order kernel is used this remainder becomes second
order, depending on the smoothness of both the true distribution F0 and the influence function
φ(z), see NHR and Bickel and Ritov (2003). Thus, by using a twicing or higher order kernel
we obtain a second order bias, so all of the remainder terms are second order. Furthermore,
series estimator automatically have a second order bias term, as pointed out in Newey (1994).
Consequently, for all of these first steps the remainders are all second order even though the
moment function is not locally robust.
The advantage of locally robust moments are that the improvement applies to any first step
estimator. One does not have to depend on the particular structure of the estimator, such
as having a kernel of sufficiently high order. This feature is important when the first step is
20

complicated so that it is hard to analyze the properties of terms that correspond to E[R̃1 ].
Important examples are first steps that use machine learning. In that setting locally robust
moments are very important for obtaining root-n consistency; see Belloni et. al. (2014). Locally
robust moments have the advantages we have discussed even for very complicated first steps.

6

First Step Series Estimators

First step series estimators have certain automatic robustness properties. Moment conditions
based on series estimators are automatically locally robust in the direction of the series approximation. We also find that affine moment functions are automatically doubly robust in these
directions. In this Section we present these results.
It turns out that for certain first step series estimators there is a version of the adjustment
term that has sample mean zero, so that ĝ(β) = m̂(β). That is, locally robust moments are
numerically identical to the original moments. This version of the adjustment term is constructed by treating the first step as if it were parametric with parameters given by those of the
series approximation, and calculating a sample version of the adjustment described in Section
P
2. Suppose that the coefficients λ̂ of the first step estimator satisfy ni=1 h(zi , λ̂)/n = 0. Let
P
P
M̂λ (β) = n−1 ni=1 ∂m(zi , β, λ̂)/∂λ, Ĥ = n−1 ni=1 ∂h(zi , λ̂)/∂λ, and
φ(z, β, γ̂) = −M̂λ (β)Ĥ −1h(z, λ̂)

(6.1)

be the parametric adjustment term described in Section 2, where γ̂ includes the elements of
M̂γ (β) and Ĥ and there is no cross fitting. Note that
n

n

1X
1X
φ(zi , β, γ̂) = −M̂λ (β)Ĥ −1
h(zi , λ̂) = 0.
n i=1
n i=1

It follows that ĝ(β) = m̂(β), i.e. the locally robust moments obtained by adding the adjustment
P
term are identical to the original moments. Thus, if ni=1 h(zi , λ̂)/n = 0, we treat the first
step series estimator as parametric, and use the parametric adjustment term the locally robust
moments are numerically identical to the original moments. This numerical equivalence results is
an exact version of local robustness of the moments in the direction of the series approximation.
In some settings it is known that φ(z, β, γ̂) in equation (6.1) is an estimated approximation to
φ(zi , β, γ0), justifying its use. Newey (1994, p. 1369) showed that this approximation property
holds when the first step is a series regression. Ackerberg, Chen, and Hahn (2012) showed
that this property holds when the first step satisfies certain conditional moment restrictions
or is part of a sieve maximum likelihood estimator. It is also straightforward to show that
this approximation holds when the first step is a series approximates to the solution to the
conditional moment restriction E[ρ(zi , γ10 )|xi ] = 0. We expect that in general φ(z, β, γ̂) is an
estimator of φ(z, β, γ0 ).
21

We note that the result that ĝ(β) = m̂(β) is dependent on λ̂ not varying with the observations
and on being constructed from the whole sample. If we use cross fitting in any form then the
numerical equivalence of the original moments with their locally robust counterpart will generally
not hold. Also ĝ(β) 6= m̂(β) will generally occur when different models are used for different
elements of γ. Such different models will often be present when machine learning is used for
constructing the estimators of the different elements of γ. See for example Chernozhukov et. al.
(2016).
There are interesting cases where the original moment functions m(z, β, γ) with a series
estimator for γ are doubly robust in certain directions, with E[m(zi , β0 , γ1)] = 0 when γ1 is
a series approximation to γ10 . Here we show this directional double robustness property for
series estimators of solutions to conditional moment restrictions and orthogonal series density
estimators. Consider first a conditional moment restriction where the residual ρ(z, γ1 ) is affine
in γ1 , m(z, β, γ1 ) is also affine in γ1 , and the first step is a linear series estimator. Suppose
that the series estimator approximates γ10 by a linear combination pK′λ of a vector of functions
pK (w) = (p1K (w), ..., pKK (w))′. Let q K (xi ) be a K × 1 vector of instrumental variables and λ̂
P
be the instrumental variables estimator solving a moment condition ni=1 q K (xi )ρ(zi , pK′λ̂) = 0.
Under standard regularity conditions the limit λ∗ of λ̂ will solve the corresponding population
moment condition E[q K (xi )ρ(zi , pK′λ∗ )] = 0. Let γ20 (x) satisfy equation (4.2). Then if γ20 (xi ) =
θ′ q K (xi ) for some θ it follows that
E[m(zi , β0 , pK′λ∗ )] = −E[γ20 (xi )ρ(zi , pK′λ∗ )] = −θ′ E[q K (xi )ρ(zi , pK′λ∗ )] = 0.
Thus we have the result
Proposition 12: If m(zi , β, γ1) and ρ(zi , γ1) are affine in γ1 ∈ Γ with Γ linear, E[m(zi , β0 , γ1 )]
is a mean square continuous functional of E[ρ(zi , γ1)|xi ], and γ20 (xi ) satisfying E[m(zi , β0 , γ1 )] =
−E[γ20 (xi )ρ(zi , γ1 )] also satisfies γ20 (xi ) = θ′ q K (xi ) for some θ then E[m(zi , β0 , pK′λ∗ )] = 0.
The property shown in the conclusion of Proposition 12 is a directional double robustness
condition that depends on γ1 being equal to a series approximation to γ10 and on γ20 (x) being
restricted. These restrictions are not required for double robustness of g(z, β, γ) = m(z, β, γ1 ) +
γ2 (x)ρ(z, γ1 ). We will have E[g(zi , β0 , γ20 , γ1 )] = 0 for all γ1 , and not just for the γ1 that are a
series approximation to γ10 , and for any γ20 (x) and not just one that is a linear combination of
q K (x). For series first steps the the original moment functions will be doubly robust in certain
directions just as they are locally robust in certain directions.
Previous examples of Proposition 12 are given in Newey (1990, p. 116), Newey (1999), and
Robins et. al. (2007). Proposition 12 allows for endogeneity and m(z, β, γ1 ) to depend on the
entire function γ1 . The condition that the instruments q K (xi ) have the same dimension as the
approximating functions pK (w) allows for more than K instrumental variables. As is well known,
22

any IV estimator of λ can be viewed as having only K instruments q K (x), each one of which
is equal to a linear combination of all the instrumental variables. Here the existence of θ such
that γ20 (xi ) = θ′ q K (xi ) is restrictive. It is not sufficient that γ20 (xi ) be any linear combination
of all the instrumental variables. We must have γ20 (xi ) equal to a linear combination of the
instruments q K (xi ) used in estimating λ∗ . This result also extends to the case where an infinite
number of instrumental variables are used in the limit. In that case q K (xi ) can also be interpreted
as an infinite dimensional linear combination of instrumental variables.
To illustrate, consider again the weighted average derivative example discussed above where
m(z, β, γ1 ) = v(w)∂γ1 (w)/∂w − β, ρ(z, γ1 ) = y − γ1 (w), and there is γ20 (x) such that
E[γ20 (x)|w] = −f0 (w)−1∂[f0 (w)v(w)]/∂w.

(6.2)

Suppose that the first step is a linear instrumental variables (IV) estimator with right-hand side
variables pK (wi ) and instruments q K (xi ) and let λ∗ be the limit of the IV coefficients. From
Proposition 12 it follows that if there is a θ such that θ′ q K (x) = γ20 (x) then

E[v(wi ) ∂pK (wi )′ λ∗ /∂w] − β0 = E[m(zi , β0 , pK′ λ∗ )] = 0.

Thus, the weighted average derivative of the linear IV estimator will be consistent when γ20 (x)
is a linear combination of q K (x).
The case where v(w) is 1, wi is Gaussian, and E[xi |wi ] is linear in wi is interesting. Partial
out constants and means so that (wi′ , x′i )′ has mean zero. Let pK (wi ) = wi and let qi = q K (xi ) be
any linear combination xi such that ζ = E[qi wi′ ] is nonsingular. Normalize wi and qi so that each
have an identity variance matrix. Then f0 (w)−1 ∂f0 (w)/∂w = −w. Note that E[qi |wi ] = ζwi , so
that equation (6.2) is satisfied with γ20 (x) = −ζ −1 q K (x).Thus the conditions of Proposition 12
are satisfied, giving the following result
Corollary 13: If yi = γ10 (wi ) + εi , E[xi εi ] = 0, E[yi2] < ∞, wi is Gaussian, and E[xi |wi ]
is linear in wi then for instruments equal to any linear combination qi of xi with cov(qi , wi )
nonsingular ,
E[∂γ10 (wi )/∂w] = cov(qi , wi )−1 cov(qi , yi ).
We can give a simple, direct proof that only uses qi and εi uncorrelated, as is assumed in
Corollary 11, rather than the conditional moment restriction we have been focusing on. With
means partialed out we have E[wi ] = E[qi ] = 0 so that
cov(qi , wi )−1 cov(qi , yi ) = (E[qi wi′ ])−1 E[qi yi ] = (E[qi wi′ ])−1 E[qi γ10 (wi )]
= (E[E[qi |wi ]wi′ ])−1 E[E[qi |wi ]γ10 (wi )] = (E[ζwi wi′ ])−1 E[ζwi γ10 (wi )]
= (E[wi wi′ ])−1 E[wi γ10 (wi )] = E[∂γ10 (wi )/∂w],
23

where the fourth equality follows by E[qi |wi ] = ζwi and the last equality holds by Stoker (1986)
and wi Gaussian. This result generalizes that of Stoker (1986) to NPIV models where the right
hand variables are Gaussian. Further generalizations to non Gaussian cases can be obtained by
letting q K (x) and pK (w) be nonlinear in x and w.
Orthogonal series density estimators have a property analogous to Proposition 12. Suppose now that pK (x) is orthonormal with respect to Lebesgue measure on (−∞, ∞) so that
R K
p (u)pK (u)′ du = I. An orthogonal series pdf estimator is γ̂1 (x) = pK (x)′ λ̂, where λ̂ =
R K
Pn K
p
(x
)/n
has
limit
λ
=
p (u)γ10 (u)du. Suppose that E[m(zi , β0 , γ1 )] is a continuous
i
∗
i=1
linear functional of γ1 − γ10 so that by the Riesz representation theorem there is γ20 (x) with
R
E[m(zi , β0 , γ1 )] = γ20 (u)[γ1(u) − γ10 (u)]du. If there is θ with θ′ pK (x) = γ20 (x) then by pK (u)
orthonormal and equation (4.4) we have
Z
Z
K′
K
′
E[m(zi , β0 , p λ∗ )] = γ20 (u)[p (u) λ∗ − γ10 (u)]du = θ′ pK (u)[pK (u)′ λ∗ − γ10 (u)]du
Z
Z
′
K
K
′
′
pK (u)γ10 (u)du = θ′ λ∗ − θ′ λ∗ = 0.
= θ [ p (u)p (u) du]λ∗ − θ
Thus we have the following result:
Proposition 14: If i) m(z, β0 , γ1 ) is affine in γ1 , ii) E[m(zi , β0 , γ1 )] − β0 is a functional
R
1/2
of γ1 (x) − γ10 (x) that is continuous in the norm
[γ1 (u) − γ10 (u)]2 du
, and iii) γ20 (xi ) =
′ K
K′
θ p (xi ) for some θ then E[m(zi , β0 , p λ∗ )] = 0.
The orthogonal series estimators of linear functionals of a pdf discussed in Bickel and Ritov (2003) are examples. Those estimators are special cases of the estimator above where
R
m(z, β, γ1 ) = γ20 (u)γ1 (u)du − β for prespecified γ20 (u). Proposition 14 implies that the orthogonal series estimator of β0 will be consistent if γ20 (u) is a linear combination of the approximating functions. For example, if pK (x) is vector of polynomials of order K − 1 then the
orthogonal series estimator of moments of up to order K − 1 is consistent for fixed K.

7

Conditional Moment Restrictions

Conditional moment restrictions are widely used in econometrics to identify parameters of interest. In this Section we expand upon the cases already considered to construct a wide variety of
locally robust moment conditions. In particular we extend above results to residuals that may
depend on parameters of interest with instrumental variables that can differ across residuals.
Here we depart from deriving locally robust moments from the adjustment term for first step
estimation. Instead we extend the form of previously derived locally robust moments to the
more general setting of this Section.
24

To describe these results let j = 2, ..., J index conditional moment restrictions, ρj (z, β, γ1 )
denote a corresponding residual, and xj be corresponding conditioning variables. We will consider construction of locally robust moment conditions when the true parameters of interest β0
and a first step γ10 satisfy conditional moment restrictions
E[ρj (zi , β0 , γ10 )|xji ] = 0, (j = 2, ..., J).

(7.1)

Here γ1 is specified to include all functions that affect any of the residuals ρj (zi , β, γ1). We
continue to assume that the unconditional moment restriction in equation (2.1) holds, though
m(z, β, γ1 ) could be zero, with identification of β0 coming from the conditional moment restrictions of equation (7.1). We will discuss this case below.
In this setting we consider locally robust moment conditions having the form
g(z, β, γ) = m(z, β, γ1 ) +

J
X

γj (xj , β)ρj (z, β, γ1 ),

(7.2)

j=2

where γj (x, β), (j = 2, ..., J) are unknown functions satisfying properties discussed below. These
moment functions depend on J first step components γ = (γ1 , ..., γJ ). By virtue of the conditional moment restrictions these moment functions will be doubly robust in (γ2 , ..., γJ ), meaning
that E[g(zi , β0 , γ10 , γ2 , ..., γJ ) = 0. They will be locally robust in γ1 if for the limit γ1 (F ) of γ̂1
and all regular parametric models Fτ as discussed in Section 2,
J
X
∂
∂
E[m(zi , β0 , γ1 (Fτ ))] + E[
γj0 (xji ) E[ρj (zi , β0 , γ1 (Fτ ))|xji ]] = 0.
∂τ
∂τ
j=2

(7.3)

If ∂E[m(zi , β0 , γ1(Fτ ))]/∂τ |τ =0 is a linear mean-square continuous function of
(∂E[ρ2 (zi , β0 , γ1 (Fτ ))|xji ]/∂τ, ..., ∂E[ρJ (zi , β0 , γ1 (Fτ ))|xji ]/∂τ )|τ =0
and the mean-square closure of the set of such vectors over all parametric submodels is linear
then existence of γj0 (xj ), j ≥ 2 satisfying equation (7.3) will follow by the Riesz representation
theorem. In addition, if m(z, β0 , γ1 ) and ρj (z, β0 , γ1), (j ≥ 2), are affine in γ1 then we will have
double robustness in γ1 similarly to Proposition 12. Summarizing we have
Proposition 15: If equation (7.3) is satisfied then g(z, β, γ) from equation (7.2) is locally
robust. Furthermore, if i) m(z, β0 , γ1 ) and ρj (z, β0 , γ1 ), (j ≥ 2) are affine in γ1 ∈ Γ with Γ
linear; ii) E[m(zi , β0 , γ1)] is a mean square continuous functional of E[ρj (zi , β0 , γ1 )|xij ], (j ≥ 2)
then there is γj0 (x), (j ≥ 2), such that g(z, β, γ) is doubly robust.
For local identification of β we also require that
rank( ∂E[g(zi , β, γ0)]/∂β|β=β0 ) = dim(β).
25

(7.4)

A model where β0 is identified from semiparametric conditional moment restrictions with
common instrumental variables x is a special case where m(z, β, γ) is zero and xj = x, (j ≥ 2).
In this case let ρ(z, β, γ1 ) = (ρ2 (z, β, γ1 ), ..., ρJ (z, β, γ1 ))′ . The conditional moment restrictions
of equation (7.1) can be summarized as
E[ρ(zi , β0 , γ10 )|xi ] = 0.
This model is considered by Chamberlain (1992) and Ai and Chen (2003, 2007, 2012). We allow
the residual vector ρ(z, β, γ1 ) to depend on the entire function γ1 and not just its value at some
function of the observed data zi . Also let ϕ(x) = [γ2 (x), ..., γJ (x)] denote an r ×(J −1) matrix of
functions of x. A locally robust moment function g(z, β, γ) = ϕ(x)ρ(z, β, γ1 ) will be one which
satisfies Definition 1 with g(z, β, γ) replacing m(z, β, γ), i.e. where


∂E[g(zi , β0 , γ(Fτ ))]
∂E[ρ(zi , β0 , γ1 (Fτ ))|xi ]
= 0,
= E ϕ(xi )
∂τ
∂τ
for all regular parametric models. We also require that equation (7.4) is satisfied.
To characterize local robustness here it is helpful to assume that the set of pathwise derivatives of E[ρ(zi , β0 , γ)|xi ] varies over a linear set as the regular parametric model Fτ varies. To
be precise we will assume that γ1 ∈ Γ for Γ linear and for ∆ ∈ Γ we let
mγ (x, ∆) =

∂E[ρ(zi , β0 , γ10 + τ ∆)|xi ]
∂τ

τ =0

denote the (J − 1) × 1 random vector that is Gateaux derivative of the conditional expectation
E[ρ(zi , β0 , γ1 )|xi ] with respect to the first step γ1 in the direction ∆. We assume that mγ (x, ∆)
is linear in ∆ and that the mean square closure Mγ of the set {mγ (x, ∆) : ∆ ∈ Γ} equals
the mean-square closure of the set {∂E[ρ(zi , β0 , γ1(Fτ ))|xi ]/∂τ } as Fτ varies over all regular
parametric models. The local robustness condition can then be interpreted as orthogonality of
each row ϕ(xi )′ ej of ϕ(x) with Mγ in the Hilbert space of functions of x with inner product
ha, bi = E[a(xi )′ b(xi )], where ek is the k th unit vector. Thus the condition for locally robust
g(z, β, γ) = ϕ(x)ρ(z, β, γ1 ) is that
E[ϕ(xi )mγ (xi , ∆)] = 0 for all ∆ ∈ Γ.
We refer to such ϕ(xi ) as being orthogonal. They can be interpreted as instrumental variables
where the effect of estimation of γ1 has been partialed out.
There are many ways to construct orthogonal instruments. For instance, given a r × (J − 1)
matrix of instrumental variables A(x) one could construct corresponding orthogonal ones ϕ(xi )
as the matrix where each row is the residual of the least squares projection of the corresponding
row of A(x) on Mγ . We focus on another way of constructing orthogonal instruments that leads
26

to an efficient estimator of β0 . Let Σ(x) denote some positive definite matrix with smallest eigenvalue bounded away from zero, so that Σ(xi )−1 is bounded. Let ha, biΣ = E[a(xi )′ Σ(xi )−1 b(xi )]
denote an inner product and note that Mγ is closed in this inner product by Σ(xi )−1 bounded.
Let Ãk (xi , A, Σ) denote the residual from the least squares projection of the k th row A (x)′ ek
of A(x) on Mγ with the inner product ha, biΣ . Also let ϕ(xi , A, Σ) be the matrix with k th row
Ãk (xi , A, Σ)′ Σ(xi )−1 , (k = 1, ..., r). Then for all ∆ ∈ Γ,
A (xi )′ ek − Ãk (xi , A, Σ) ∈ Mγ , E[ϕ(xi , A, Σ)mγ (xi , ∆)] = E[Ã(xi , A, Σ)Σ(xi )−1 mγ (xi , ∆)] = 0,
so that ϕ(xi , A, Σ) are orthogonal instruments. Also, Ã(x, A, Σ) can be interpreted as the
solution to
min

{M (x):M (x)′ ek ∈Mγ ,k=1,...,r}

E[{A(xi ) − M(xi )}Σ(xi )−1 {A(xi ) − M(xi )}′ ]

where the minimization is in the positive semidefinite sense.
The orthogonal instruments that minimize the asymptotic variance of GMM in the class of
GMM estimators with orthogonal instruments are given by
∂E[ρ(zi , β, γ10 )|xi ]
ϕ (xi ) = ϕ(xi , A , Σ ), A (xi ) =
∂β
∗

∗

∗

′

, Σ∗ (xi ) = V ar(ρ(zi , β0 , γ10 )|xi ).

∗

β=β0

To see that ϕ∗ (xi ) minimizes the asymptotic variance note that for any orthogonal instrumental
variable matrix ϕ(x)
G = E[ϕ(xi )A∗ (xi )′ ] = E[ϕ(xi )Ã(xi , A∗ , Σ∗ )′ ] = E[ϕ(xi )ρ(zi , β0 , γ10 )ρ(zi , β0 , γ10 )′ ϕ∗ (xi )′ ],
where the first equality defines G and the second equality holds by ϕ(xi ) orthogonal. Since the
p
instruments are orthogonal the asymptotic variance matrix of GMM estimator with Ŵ −→ W
is the same as if γ̂1 = γ10 . Define mi = G′ W ϕ(xi )ρ(zi , β0 , γ10 ) and m∗i = ϕ∗ (xi )ρ(zi , β0 , γ10 ). The
asymptotic variance of the GMM estimator for orthogonal instruments ϕ(x) is
(G′ W G)−1 G′ W E[ϕ(xi )ρ(zi , β0 , γ10 )ρ(zi , β0 , γ10 )′ ϕ(xi )′ ]W G(G′ W G)−1
−1
′
∗ −1′
= (E[mi m∗′
.
i ]) E[mi mi ](E[mi mi ])

The fact that this matrix is minimized in the positive semidefinite sense for ϕ(x) = ϕ∗ (x) follows
from Theorem 5.3 of Newey and McFadden (1994) and can also be shown using the argument
in Chamberlain (1987).
Proposition 16: The instruments ϕ∗ (xi ) give an efficient estimator in the class of IV
estimators with orthogonal instruments.
The asymptotic variance of the GMM estimator with optimal orthogonal instruments is
−1
(E[m∗i m∗′
= E[Ã(xi , A∗ , Σ∗ )Σ∗ (xi )−1 Ã(xi , A∗ , Σ∗ )′ ])−1 .
i ])

27

This matrix coincides with the semiparametric variance bound of Ai and Chen (2003). Estimation of the optimal orthogonal instruments is beyond the scope of this paper. The series
estimator of Ai and Chen (2003) could be used for this.

8

Structural Economic Examples

Estimating structural models can be difficult when that requires computing equilibrium solutions. Motivated by this difficulty there is increasing interest in two step semiparametric
methods based on first step estimation of conditional choice probabilities (CCP). This two step
approach was pioneered by Hotz and Miller (1993). In this Section we show how locally robust
moment conditions can be formed for two kinds of structural models, the dynamic discrete choice
model of Rust (1987) and the static model of strategic interactions of Bajari, Hong, Krainer,
and Nekipelov (2010, BHKN). It should be straightforward to extend the construction of locally robust moments to other more complicated structural economic models. The use of such
moment conditions will allow for conditional choice probabilities that are estimated by modern,
machine learning methods.

8.1

Static Models of Strategic Interactions

We begin with a static model of interactions where results are relatively simple. To save space we
describe the estimator of BHKN while only describing a small part of the motivational economic
structure. Let x denote a vector of state variables for a fixed set of individuals and let y denote
a vector of binary variables, each one representing a choice of an alternative by an individual.
Let the observations zi = (yi , xi ) represent repeated plays of a static game of interaction and
γ10 (x) = E[yi |xi = x] the vector of conditional choice probabilities given a value x of the state.
In the semiparametric estimation problem described in Section 4.2 of BHKN there is a known
function r(x, β, γ1(x)) of the state variable x, a vector of parameters β, and a possible value
γ(x) of the conditional choice probabilities such that the true parameter β0 satisfies
E[yi |xi = x] = r (x, β0 , γ10 (x)) ,
This model can be used to form moment functions
m(z, β, γ1 ) = A(x)[y − r (x, β, γ1 (x))],
where A(x) is a matrix of instrumental variables; see equation (17) of BHKN.
To describe locally robust moment functions in this example, let rγ (x, β, γ1) = ∂r(x, β, γ1 )/∂γ1
where γ1 here denotes a real vector representing a possible value of γ10 (x). Then, it follows from

28

Proposition 4 of Newey (1994), as discussed in BHKN, that the adjustment term for first step
estimation of γ10 (x) = E[yi |xi = x] is
φ(z, β, γ1 ) = −A(x)rγ (x, β, γ1)[y − γ1 (x)].
This expression differs from BHKN in the appearance of γ1 (x) at the end of the expression
rather than rγ (x, β, γ1 (x)), which is essential for local robustness. The locally robust moment
conditions are then
g(z, β, γ1) = A(x){y − r(x, β, γ1 (x)) − rγ (x, β, γ1(x))[y − γ1 (x)]}.
For a first step estimator γ̂(x) of the conditional choice probabilities, the locally robust sample
moments will be
n

1X
A(xi ){yi − r(xi , β, γ̂1(xi )) − rγ (xi , β, γ̂1(xi ))[yi − γ̂1 (xi )]}
ĝ(β) =
n i=1
Here the locally robust moments are constructed by subtracting from the structural residuals r
a linear combination of the first step residuals. Using these moment functions should result in
an estimator of the structural parameters with less bias and the other improved properties of
locally robust estimators mentioned above.
The optimal instruments here are the same as discussed in BHKN. Let I denote the identity
matrix, set H(x) = I − ∂r(xi , β0 , γ10 (xi ))/∂γ1 , and let Ω(xi ) = H(xi )V ar(yi |xi )H(xi )T denote
the conditional variance of H(xi )(yi − γ10 (xi )). The optimal instruments are given by
∂r(xi , β0 , γ10 (xi )) ′
Ω(x)−
A (xi ) =
∂β
∗

where A− denotes a generalized inverse of a positive semi-definite matrix A.
This model can also be viewed as a special case of the conditional moment restrictions
framework with residual vector ρ(z, β, γ1 ) = (y − γ1 (x), y − r(x, β, γ1 (x)))T . An orthogonal
instrument that gives the above locally robust moment function is A(x)[−rγ (xi , β, γ1(xi )), I].
Here the locally robust moment function only depend on one first step function γ1 (x). This
feature is shared by all setups where the second step residual r(x, β, γ1) depends only on regressors that are included in the first step γ1 (x). The static model of strategic interactions leads
to this structure. The situation is not so simple in other structural economic models, as we see
next.

8.2

Dynamic Discrete Choice

Dynamic discrete choice estimation is important for modeling economic decisions, Rust (1987).
In this setting we find it helpful to describe the underlying economic model in order to explain
29

the form of the moment conditions. Here we give locally robust moment conditions moment
conditions that depend on first step estimation of the conditional choice probabilities. We do
this for the infinite horizon, stationary, dynamic discrete choice model of Rust (1987). It is
straightforward to derive locally robust moment conditions for other structural econometric
models. We also focus here on the case of data on many homogenous individuals, but discuss
how the approach extends to time series on one individual.
Suppose that the per-period utility function for an agent making choice j in period t is given
by
Ujt = vj (xt , β0 ) + ǫjt , (j = 1, ..., J; t = 1, 2, ...)
where we suppress the individual subscript i for notational convenience. The vector xt is the
observed state variables of the problem (e.g. work experience, number of children, wealth) and
the vector β is unknown parameters. The disturbances ǫt = {ǫ1t , ..., ǫJt } are not observed by the
econometrician. As in the majority of the literature we assume that ǫt is i.i.d. over time with
known CDF that has support RJ and is independent of the state process xt and that xt is Markov
of order 1. Let δ denote a time discount parameter, v̄(x) the expected value function, yjt ∈ {0, 1}
the indicator that choice j is made and v̄j (xt ) = vj (xt , β0 ) + δE[v̄(xt+1 )|xt , ytj = 1] the expected
value function for choice j. Also let ṽj denote a possible realization of ṽj (xt ) = v̄j (xt ) − v̄1 (xt ),
so that ṽ1 ≡ 0. Let ṽ = (ṽ2 , ..., ṽJ ) and Pj (ṽ) = Pr(ṽj + ǫjt ≥ ṽk + ǫkt ; ṽ1 = 0; k = 1, ..., J),
(j = 1, ..., J) denote the choice probabilities associated with the distribution of ǫt . Here we
normalize to focus on the difference with v̄1 (x) throughout. Let ṽ(x) = (ṽ2 (x), ..., ṽJ (x))′ . Let
γa (x) = (γa1 (x), ..., γaJ (x))T be a vector of first step functions with true values γaj0(xt ) =
Pr(ytj = 1|xt ). From Rust (1987) we know that for
γaj0 (xt ) = Pj (ṽ(xt )), (j = 1, ..., J).
v̄(xt ) = E[max{v̄j (xt ) + ǫjt }|xt ] = v̄1 (xt ) + E[max{ṽj (xt ) + ǫjt }|xt ].
j

j

From Hotz and Miller (1993) we know that P (ṽ) = (P1 (ṽ), P2 (ṽ), ..., PJ (ṽ))′ is a one-to-one
function of ṽ, so that inverting this relationship it follows that E[maxj {ṽj (xt ) + ǫjt }|xt ] is a
function of γa0 (xt ), say E[maxj {ṽj (xt ) + ǫjt }|xt ] = H(γa0 (xt )), for some function H(·) (e.g. for
binary logit H(γa0 (xt )) = .5772 − ln(γa10 (xt ))). Then the expected value function is given by
v̄(xt ) = v̄1 (xt ) + H(γa0 (xt ))
To use these relationships to construct semiparametric moment conditions we normalize
v1 (x, β) = 0 and make an additional assumption for v̄1 (x). The additional assumption is that
E[v̄(xt+1 )|xt , 1] does not depend on xt . With this normalization and assumption we have a
constant choice specific value function for j = 1, that is v̄1 (x) = v̄1 , with
v̄1 (xt ) = 0 + δE[v̄(xt+1 )|xt , yt1 = 1] = δE[v̄(xt+1 )|yt1 = 1] = v̄1 .
30

A sufficient condition for constant v̄1 (xt ) is that j = 1 is a ”renewal” choice where the distribution
of the future state does not depend on the current state. In the Rust (1987) example this state
is the one where the bus engine is replaced.
With this normalization and assumption we now have
v̄j (x) = vj (x, β0 ) + δE[v̄1 + H(γa0 (xt+1 ))|xt = x, ytj = 1]
= vj (x, β0 ) + δv̄1 + δE[H(γa0 (xt+1 ))|xt = x, ytj = 1],
ṽj (x) = vj (x, β0 ) + δE[H(γa0 (xt+1 ))|xt = x, ytj = 1] − δE[H(γa0 (xt+1 ))|yt1 = 1], (j = 2, ..., J).
The choice specific expected value differences ṽj (x) have a parametric part vj (x, β0 ) and a
nonparametric part that depends on J − 1 additional nonparametric regressions γb (x, γa ) =
(γb1 (x, γa ), ..., γb,J−1 (x, γa ))T and an unknown parameter γc where
γbj0 (x, γa ) = E[H(γa (xt+1 ))|xt = x, yt,j+1 = 1], j = 1, ..., J − 1; γc0(γa ) = E[H(γa (xt+1 ))|yt1 = 1].
Let γ1 (x) = (γa (x)T , γb(x, γa )T , γc (γa ))T be a vector of first step objects and
ṽj (x, β, γ1) = vj (x, β) + δ[γb,j (x, γa ) − γc (γa )], ṽ(x, β, γ1) = (ṽ2 (x, β, γ1), ..., ṽJ (x, β, γ1))T
denote the semiparametric choice specific expected value differences. Semiparametric moment
conditions can then be formed by plugging in a nonparametric γ̂1 estimator of the first step into
the expected differences and plugging those into the choice probability. Let yt = (yt1 , ..., ytJ )T
denote the vector of choice indicators for period t and z = (y1T , xT1 , ..., yTT , xTT )T be the vector
consisting of the observations on choice and state variables xt for each time period t. Also
let At (xt ) be an r × 1 vector of functions of xt where r is the dimension of β. Then for each
t = 1, ..., T − 1 we can form semiparametric moment conditions as
mt (z, β, γ1 ) = At (xt )[yt − P (ṽ(xt , β, γ1))].
To derive locally robust moment functions we can derive the adjustment term for estimation
of γ1 . The first step function γ1 is more complicated than previously considered. It depends on
two unknown conditional expectations, E[yt |xt ] and E[·|xt , j], (j = 2, ..., J). From Newey (1994,
p. 1357) we know that the adjustment term will be the sum of two terms, each adjusting for
one of the two conditional expectations while treating the other as if it was equal to the truth.
In the Appendix we give a general form for each of these adjustment terms. Here we apply that
general form to derive the corresponding locally robust moment functions for dynamic discrete
choice.
We begin by deriving the adjustment terms for γb and γc because they are simpler than
for γa . The adjustment term for γb and γc are obtained by applying Proposition A1 in the

31

Appendix. Let γ̃b (F ), γ̃c (F ) have the same form as γb and γc except that E[·|xt , yt ] is replaced
by EF [·|xt , j]. For Ht+1 = H (γa0 (xt+1 )) and π1 = E[yt1 ] let
λbj (z, β, γ1) = [yt,j+1/γa,j+1(xt )]{H(γa (xt+1 )) − γb,j+1(xt , γa )},
λb (z, β, γ1) = (λb1 (z, β, γ1 ), ..., λb,J−1(z, β, γ1 ))T ,

λc (z, β, γ1 , π1 ) = π1−1 yt1 H(γa (xt+1 )) − γc (γa ).
Then for e a (J − 1) × 1 vector of 1′ s we have

∂E[mt (zi , β0 , γa0 , γ̃b (Fτ ), γ̃c (Fτ ))]
= E[φtb (zi , β0 , γ1 , γ2)S(zi )],
∂τ
∂P
(ṽ(xt , β, γ1))λb (z, β, γ1)
φtb (z, β, γ1, π) = −δA(xt )
∂ṽ
∂P
+ δE[A(xt )
(ṽ(xt , β, γ1))]e · λc (z, β, γ1 , π1 )}
∂ṽ
The adjustment term for estimation of γa (x) is obtained by applying Proposition A2. This
term is somewhat complicated because γa (x) is evaluated at x = xt+1 rather than the conditioning argument xt of its true value We assume that xt is stationary over time so that xt+1 and xt
have the same pdf, eliminating the ratio of pdf’s in the conclusion of Proposition A2. Let γ̄a (F )
have the same form as γ10 except that γ10 (x) is replaced by EFt [yt |xt = x]. Also let


∂P (ṽ(xt , β, xt ))
ytj
A(xt )
|xt+1 = x , (j = 1, ..., J − 1),
γ2j0(x, β, γ1 ) = E
γaj (xt )
∂ṽ
γ30 (x, π1 ) = π1−1 E[yt1 |xt+1 = x]|x=xt .

Then we have
∂E[mt (zi , β0 , γa (Fτ ), γb0 , γc0 )]
= E[φta (zi , β0 , γ1 , γ2)S(zi )],
∂τ
φta (z, β, γ1 , γ2, γ3 , π1 ) = −δ{γ2 (x, β, γ1) − E[A(xt )Pṽ (ṽ(xt , β, γ1))]γ3 (xt , π1 )}e
×

∂H(γa (xt ))
[yt − γa (xt )].
∂γa

We can now form locally robust moment conditions as
gt (z, β, γ1 , γ2 , γ3, π1 ) = mt (z, β, γ1 ) + φta (z, β, γ1 , γ2 , γ3, π1 ) + φtb (z, β, γ1 , π1 ).
With data zi that is i.i.d. over individuals these moment functions can be used for any t to
estimate the structural parameters β. Also, for data for a single individual we could use a time
P −1
gt (z, β, γ1 , γ2 , γ3, π1 )/(T − 1) to estimate β, although the asymptotic theory we
average Tt=1
give does not apply to this estimator.
Bajari, Chernozhukov, Hong, and Nekipelov (2009) derived the adjustment term for the more
complicated dynamic discrete game of imperfect information. Locally robust moment conditions
for such games could be formed using their results. We leave that formulation to future work.
32

9

Asymptotic Theory for Locally Robust Moments

In this Section we give asymptotic theory for locally robust estimators. In keeping with the
general applicability of locally robust moments to a variety of first steps we consider estimation
and conditions that have the most general and simplest conditions we can find for the first step.
In particular, the construction here only requires that the first step converge at rate slightly
faster than n−1/4 in norms specified below, a simpler condition than in most of the literature.
This formulation allows the results to be applied in settings where it is challenging to say much
about the first step other than its convergence rate, such as when machine learning is used in
the first step. The locally robust form of the moment conditions is essential for this formulation,
as previously discussed.
We use cross fitting in the first step to obtain an estimator that is root-n consistent and
asymptotically normal under such simple conditions. Chernozhukov et. al. (2016) gives results
with cross fitting that allow for moment functions that are not smooth in parameters. Here we
focus on the smooth in parameters case. Cross fitting has been previously used in the literature
on semiparametric estimation. See Bickel, Klaasen, Ritov, and Wellner (1993) for discussion.
This approach is different than that some previous work in semiparametric estimation, as in
Andrews (1994), Newey (1994), Chen, Linton, and van Keilegom (2003), Ichimura and Lee
(2010), where cross fitting was not used and the moment conditions need not be locally robust.
The approach adopted here leads to general and simple conditions.
The estimator is formed by grouping observations into L distinct groups. Let Iℓ , (ℓ = 1, ..., L)
partition the set of observation indices {1, ..., n}. Let γ̂−ℓ be the first step constructed from all
observations not in Iℓ . Consider sample moment conditions of the form
L

1 XX
g(zi , β, γ̂−ℓ).
g̃(β) =
n ℓ=1 i∈I
ℓ

We consider GMM estimators based on these moment functions. This is a special case of the
cross fitting described earlier. Also, leave one out moment conditions are a further special case
where each Iℓ consists of a single observation. We focus here on the case where the number of
groups L is fixed to keep the conditions as simple as possible.
An important intermediate result is that the adjustment term for the first step is zero by
virtue of g(z, β, γ) being locally robust, that is
√

n

1 X
ng̃(β0 ) = √
g(zi , β0 , γ0) + op (1).
n i=1

(9.1)

With cross fitting this result holds under relatively weak and simple conditions:
R
p
Assumption 1: For each ℓ = 1, ..., L, i) kg(z, β0 , γ̂−ℓ ) − g(z, β0 , γ0 )k2 F0 (dz) −→ 0, ii) for
R
√
p
ζ > 1, C > 0 we have
g(z, β0 , γ̂−ℓ )F0 (dz) ≤ C kγ̂−ℓ − γ0 kζ , and iii) n kγ̂−ℓ − γ0 kζ −→ 0.
33

Lemma 17: If Assumption 1 is satisfied then equation (9.1) is satisfied.
This Lemma is proved in the Appendix. There are two important components to this result.
One component is a stochastic equicontinuity result
√

n

L

1 X
1 X
ng̃(β0 ) − √
nℓ
g(zi , β0 , γ0) − √
n i=1
n ℓ=1

Z

p

g(z, β0 , γ̂−ℓ )F0 (dz) −→ 0,

where nℓ is the number of observations with i ∈ Iℓ . Assumption 1 i) is sufficient for this result.
Assumption 1 i) is a much weaker stochastic equicontinuity condition than appears in much of
the literature, e.g. Andrews (1994). Those other conditions generally involve boundedness of
some derivatives of γ̂. In contrast Assumption 1 i) only requires that γ̂−ℓ have a mean square
convergence property. The cross fitting is what makes this condition sufficient. Cattaneo and
Jansson (2014) have also previously weakened the stochastic equicontinuity condition and established the validity of the bootstrap for kernel estimators under substantially weaker bandwidth
conditions than usually imposed.
The second component of the result is that
Z
√
p
nḡ(γ̂−ℓ ) −→ 0, ḡ(γ) = g(z, β0 , γ)F0 (dz).
This component follows from Assumptions 1 ii) and iii). By comparing Assumption 1 ii) with
Definition 2 we see that this condition implies local robustness in the sense that the Frechet
derivative of ḡ(γ) is zero at γ0 . Assumption 1 ii) will generally hold with ζ = 2 if ḡ(γ) is twice
continuously Frechet differentiable. In that case Assumption 1 iii) become the n1/4 rate condition
familiar from Newey and McFadden (1994) and other works. The more general 1 < ζ < 2 case
allows for the first Frechet derivative of ḡ(γ) to satisfy a Lipschitz condition. In this case
Assumption 1 iii) will require a convergence rate of γ̂ that is faster than n1/4 .
We note that previous results suggest that n1/4 convergence of γ̂ may be stronger than is
needed. As shown in Robins et. al. (2008) and Cattaneo and Jansson (2014) the variance terms
√
in nḡ(γ̂) are of the same order as the variance term of a nonparametric estimator, rather
√
than being the order of n times those variance terms. The arguments for these weaker results
are quite complicated so we do not attempt to give an account here. Instead we focus on the
relatively simple conditions of Assumption 1.
Another component of an asymptotic normality result is convergence of the Jacobian term
∂g̃(β)/∂β. The conditions we impose to account for the Jacobian term are standard. Let G̃(β) =
∂g̃(β)/∂β denote the derivative of the moment function.
Assumption 2: There is a neighborhood N of β0 such that i) g(zi , β, γ) is differentiable in
β on N with probability approaching 1 ii) there is ζ ′ > 0 and d(zi ) with E[d(zi )] < ∞ such that
34

for β ∈ N and kγ − γ0 k small enough
′
′
∂g(zi , β, γ) ∂g(zi , β0 , γ0)
≤ d(zi )(kβ − β0 kζ + kγ − γ0 kζ ),
−
∂β
∂β

p

iii) E[k∂g(zi , β0 , γ0 )/∂βk] < ∞; iv) kγ̂−ℓ − γ0 k −→ 0, (ℓ = 1, ..., L).
Define
G = E[∂g(zi , β, γ0)/∂β|β=β0 ]
p

Lemma 18: If Assumption 2 is satisfied then for any β̄ −→ β0 , g̃(β) is differentiable at β̄
p
with probability approaching one and ∂g̃(β̄)/∂β −→ G.
With Lemmas 17 and 18 in place the asymptotic normality of semiparametric GMM follows
in a standard way.
p

p

Theorem 19: If Assumptions 1 and 2 are satisfied, β̂ −→ β0 , Ŵ −→ W , G′ W G is
nonsingular, and E[kg(zi , β0 , γ0 )k2 ] < ∞ then for Ω = E[g(zi , β0 , γ0)g(zi , β0 , γ0 )′ ],
√

d

n(β̂ − β0 ) −→ N(0, V ), V = (G′ W G)−1G′ W ΩW G(G′W G)−1 .

It is also useful to have a consistent estimator of the asymptotic variance of β̂. As usual such
an estimator can be constructed as
V̂ = (Ĝ′ Ŵ Ĝ)−1 Ĝ′ Ŵ Ω̂Ŵ Ĝ(Ĝ′ Ŵ Ĝ)−1 ,
L

∂g̃(β̂)
1 XX
Ĝ =
g(zi , β̂, γ̂−ℓ )g(zi , β̂, γ̂−ℓ )′ .
, Ω̂ =
∂β
n ℓ=1 i∈I
ℓ

Note that this variance estimator ignores the estimation of γ, which works here because the
moment conditions are locally robust. Its consistency will follow under the conditions of Theorem
19 and one additional condition that accounts for the presence of β̂ in Ω̂.
˜ with E[d(z
˜ i )2 ] <
Theorem 20: If the conditions of Theorem 19 are satisfied and there is d(z)
∞ such that for kβ − β0 k and kβ − β0 k small enough
˜ i )(kβ − β0 kζ ′ + kγ − γ0 kζ ′ ),
kg(zi , β, γ) − g(zi , β0 , γ0 )k ≤ d(z
p

then V̂ −→ V.
In this Section we have used cross fitting to obtain relatively simple conditions for asymptotic
normality of locally robust semiparametric estimators. It is also known that in some settings
35

some kinds of cross fitting improves the properties of semiparametric estimators. For linear
kernel averages it is known that the leave one out method eliminates a bias term and leads to
a reduction in asymptotic mean square error; e.g. see NHR and the references therein. Also
Robins et. al. (2008) use cross fitting in higher order bias corrections. These results indicate
the some kind of cross fitting can lead to estimators with improved properties. For reducing
higher order bias and variance it may be desirable to let the number of groups grow with the
sample size. That case is beyond the scope of this paper.

10

APPENDIX

We first give an alternative argument for Proposition 2 that is a special case of the proof of
Theorem 2.2 of Robins et. al. (2008). As discussed above, φ(z, β0 , γ0 ) is the influence function
of the functional µ(F ) = E[m(zi , β0 , γ(F ))]. Because it is an influence function it has mean
R
zero at all true distributions, i.e. φ(z, β0 , γ(F0 ))F0 (dz) ≡ 0 identically in F0 . Since a regular
parametric model {Fτ } is just a subset of all true models, we have
Z
φ(z, β0 , γ(Fτ ))Fτ (dz) ≡ 0,
identically in τ . Differentiating this identity at τ = 0 and applying the chain rule gives
∂E[φ(zi , β0 , γ(Fτ ))]
∂τ

τ =0

= −E[φ(zi , β0 , γ0 )S(zi )].

(10.1)

Summing equations (3.3) and (10.1) we obtain
∂E[g(zi , β0 , γ(Fτ ))]
∂τ

=
τ =0

∂E[m(zi , β0 , γ(Fτ ))]
∂τ

+
τ =0

∂E[φ(zi , β0 , γ(Fτ ))]
∂τ

τ =0

= E[φ(zi , β0 , γ0 )S(zi )] − E[φ(zi , β0 , γ0 )S(zi )] = 0.
Thus we see that the adjusted moment functions g(z, β, γ) are locally robust.
Next we derive the form of the adjustment term when a first step is E[·|x, y = 1] for some
binary variable y with where yi ∈ {0, 1}. Consider a first step function of the form γ1 (x) =
E[wi |xi = x, yi = 1]. Let π(xi ) = E[yi |xi ]. Note that E[wi |xi , yi = 1] = E[yi wi |xi ]/E[yi |xi ] so
that
∂EFτ [wi |xi , yi = 1]
= E[π(xi )−1 {yi wi − E[yi wi |xi ]} − E[wi |xi , yi = 1](yi − π(xi ))}S(zi )|xi ]
∂τ
= E[π(xi )−1 yi {wi − E[wi |xi , yi = 1]}S(zi )|xi ].
Suppose that there is δ(xi ) such that
∂EFτ [m(zi , β0 , γ1(Fτ ))]
∂EFτ [wi |xi , yi = 1]
= E[δ(xi )
]
∂τ
∂τ
= E[δ(xi )π(xi )−1 yi {wi − E[wi |xi , yi = 1]}S(zi )].
36

Then taking the limit gives the following result:
Proposition A1: If there is δ(x) such that ∂E[m(zi , β0 , γ1(Fτ ))]/∂τ = E[δ(xi )∂EFτ [wi |xi , yi =
1]/∂τ ] then the adjustment term is
φ(z, β, γ) = δ(x)π(x)−1 y{w − E[wi |xi = x, yi = 1]}.
Next we derive the adjustment term when a nonparametric regression is evaluated at a
variable different than the one being conditioned on in the regression. Note that for δ(v) with
E[E[δ(vi )|wi ]EFτ [yi |xi = x]|x=wi ]
∂E[δ(vi )EFτ [yi |xi = x]|x=wi ]
=
∂τ
∂τ
Z
= ∂ E[δ(vi )|wi = w]EFτ [yi |xi = w]fw (w)dw/∂τ
Z
= ∂ [fw (x)/fx (x)]E[δ(vi )|wi = x]EFτ [yi |xi = x]fx (dx)/∂τ

= E[{fx (xi )−1 fw (xi )E[δ(vi )|wi = w]|w=xi (yi − E[yi |xi ])}S(zi )]

Taking limits gives
Proposition A2: If there is δ(v) such that ∂E[m(zi , β0 , γ1 (Fτ ))]/∂τ = ∂E[δ(vi )EFτ [yi |xi =
x]|x=wi ]/∂τ then the adjustment term is
φ(z, β, γ) = fx (x)−1 fw (x)E[δ(vi )|wi = x](yi − E[yi |xi = x]).
Next we give the proofs for the asymptotic normality results.
Proof of Lemma 17: Let
Z
X
ˆ iℓ .
ˆ iℓ = g(zi , β0 , γ̂−ℓ ) − ḡ(γ̂−ℓ ) − g(zi , β0 , γ0 ), (i ∈ Iℓ ), ∆
¯ℓ = 1
∆
ḡ(γ) = g(z, β0 , γ)F0 (dz), ∆
n i∈I
ℓ

Also, let Z−ℓ denote a vector of all observations zi for i ∈
/ Iℓ . Note that by construction
ˆ
E[∆iℓ |Z−ℓ ] = 0, so for any i, j ∈ Iℓ , i 6= j, it follows by zi and zj independent conditional
ˆ′ ∆
ˆ
ˆ′
ˆ
on Z−ℓ that E[∆
iℓ jℓ |Z−ℓ ] = E[∆iℓ |Z−ℓ ]E[∆jℓ |Z−ℓ ] = 0 Furthermore
Z
2
ˆ iℓ |Z−ℓ ] ≤ kg(z.β0 , γ̂−ℓ ) − g(z, β0 , γ0 )k2 F0 (dz).
E[ ∆
Therefore, for nℓ equal to the number of observations in group ℓ, Assumption 1 i) implies
Z
X
2
1
n
ℓ
′
¯ ∆
¯
ˆ iℓ |Z−ℓ ] ≤
E[∆
E[ ∆
kg(z.β0 , γ̂−ℓ ) − g(z, β0 , γ0 )k2 F0 (dz) = op (nℓ /n2 ).
ℓ ℓ |Z−ℓ ] = 2
2
n i∈I
n
ℓ

37

¯ ℓ = op (√nℓ /n). It then follows that
Standard arguments then imply that for each ℓ we have ∆
#
"
n
L
p
√
√ X
1X
p
¯ ℓ = op ( nℓ /n) −→
g(zi , β0 , γ0 ) − ḡ(γ̂) = n
∆
n g̃(β0 ) −
0.
n i=1
ℓ=1
It also follows by Assumption 1 ii) and iii) that
√

n kḡ(γ̂)k ≤

√

p

nC kγ̂ − γ0 kζ −→ 0.

The conclusion then follows by the triangle inequality. Q.E.D.
Proof of Lemma 18: Let G̃(β) = ∂g̃(β)/∂β when the derivative exists and Ĝ = ∂g(zi , β0 , γ0)/∂β.
p
By the law of large numbers and Assumption 2 iii), Ĝ −→ G. Also, by Assumption 2 i), ii),
P
iii) G̃(β̄) is well defined with probability approaching one ni=1 d(zi )/n = Op (1) by the Markov
inequality, and by the triangle inequality,
L

1 XX
{d(zi )( β̄ − β0
G̃(β̄) − Ĝ ≤
n ℓ=1 i∈I
ℓ
!
n
1X
≤
d(zi ) ( β̄ − β0
n i=1

ζ′

ζ′

′

+ kγ̂ℓ − γ0 kζ )}
+

L
X
ℓ=1

′

p

kγ̂ℓ − γ0 kζ ) = Op (1)op (1) −→ 0.

The conclusion then follows by the triangle inequality. Q.E.D.
The proofs of Theorems 19 and 20 are standard and so we omit them.

Acknowledgements
Whitney Newey gratefully acknowledges support by the NSF. Helpful comments were provided
by M. Cattaneo, J. Hahn, M. Jansson, Z. Liao, J. Robins, R. Moon, A. de Paula, J.M. Robin,
participants in seminars at Cornell, Harvard-MIT, UCL, and USC.

REFERENCES
Ackerberg, D., X. Chen, and J. Hahn (2012): ”A Practical Asymptotic Variance Estimator
for Two-step Semiparametric Estimators,” The Review of Economics and Statistics 94: 481–498.
Ackerberg, D., X. Chen, J. Hahn, and Z. Liao (2014): ”Asymptotic Efficiency of Semiparametric Two-Step GMM,” The Review of Economic Studies 81: 919–943.
Ai, C. and X. Chen (2003): “Efficient Estimation of Models with Conditional Moment Restrictions Containing Unknown Functions,” Econometrica 71, 1795-1843.
38

Ai, C. and X. Chen (2007): ”Estimation of Possibly Misspecified Semiparametric Conditional
Moment Restriction Models with Different Conditioning Variables,” Journal of Econometrics
141, 5–43.
Ai, C. and X. Chen (2012): ”The Semiparametric Efficiency Bound for Models of Sequential
Moment Restrictions Containing Unknown Functions,” Journal of Econometrics 170, 442–457.
Andrews, D.W.K. (1994): “Asymptotics for Semiparametric Models via Stochastic Equicontinuity,” Econometrica 62, 43-72.
Bajari, P., V. Chernozhukov, H. Hong, and D. Nekipelov (2009): ”Nonparametric and
Semiparametric Analysis of a Dynamic Discrete Game,” working paper, Stanford.
Bajari, P., H. Hong, J. Krainer, and D. Nekipelov (2010): ”Estimating Static Models of
Strategic Interactions,” Journal of Business and Economic Statistics 28, 469-482.
Bang, and J.M. Robins (2005): ”Doubly Robust Estimation in Missing Data and Causal Inference Models,” Biometrics 61, 962–972.
Belloni, A., D. Chen, V. Chernozhukov, and C. Hansen (2012): “Sparse Models and
Methods for Optimal Instruments with an Application to Eminent Domain,” Econometrica 80,
2369–2429.
Belloni, A., V. Chernozhukov, and Y. Wei (2013): “Honest Confidence Regions for Logistic
Regression with a Large Number of Controls,” arXiv preprint arXiv:1304.3969.
Belloni, A., V. Chernozhukov, I. Fernandez-Val, and C. Hansen (2016): ”Program
Evaluation and Causal Inference with High-Dimensional Data,” Econometrica, forthcoming..
Bera, A.K., G. Montes-Rojas, and W. Sosa-Escudero (2010): ”General Specification
Testing with Locally Misspecified Models,” Econometric Theory 26, 1838–1845.
Bickel, P.J., C.A.J. Klaassen, Y. Ritov, and J.A. Wellner (1993): Efficient and Adaptive
Estimation for Semiparametric Models, Springer-Verlag, New York.
Bickel, P.J. and Y. Ritov (2003): ”Nonparametric Estimators Which Can Be ”Plugged-in,”
Annals of Statistics 31, 1033-1053.
Cattaneo, M.D., and M. Jansson (2014): ”Bootstrapping Kernel-Based Semiparametric Estimators,” working paper, Berkeley.
Chamberlain, G. (1987): “Asymptotic Efficiency in Estimation with Conditional Moment Restrictions,” Journal of Econometrics 34, 1987, 305–334.
Chamberlain, G. (1992): “Efficiency Bounds for Semiparametric Regression,” Econometrica 60,
567–596.
Chen, X. and X. Shen (1997): “Sieve Extremum Estimates for Weakly Dependent Data,”
Econometrica 66, 289-314.

39

Chen, X., O.B. Linton, and I. van Keilegom (2003): “Estimation of Semiparametric Models
when the Criterion Function Is Not Smooth,” Econometrica 71, 1591-1608.
Chen, X., and A. Santos (2015): “Overidentification in Regular Models,” working paper.
Chernozhukov, V., C. Hansen, and M. Spindler (2015): ”Valid Post-Selection and PostRegularization Inference: An Elementary, General Approach,” Annual Review of Economics 7:
649–688.
Chernozhukov, V., D. Chetverikov, M. Demirer, E. Duflo, C. Hansen, W. Newey
(2016): ”Double Machine Learning: Improved Point and Interval Estimation of Treatment and
Causal Parameters,” MIT working paper.
Firpo, S. and C. Rothe (2016): ”Semiparametric Two-Step Estimation Using Doubly Robust
Moment Conditions,” working paper.
Hasminskii, R.Z. and I.A. Ibragimov (1978): ”On the Nonparametric Estimation of Functionals,” Proceedings of the 2nd Prague Symposium on Asymptotic Statistics, 41-51.
Hausman, J.A., and W.K. Newey (2016): ”Individual Heterogeneity and Average Welfare,”
Econometrica 84, 1225-1248.
Hotz, V.J. and R.A. Miller (1993): ”Conditional Choice Probabilities and the Estimation of
Dynamic Models,” Review of Economic Studies 60, 497-529.
Ichimura, H., and S. Lee (2010): “Characterization of the Asymptotic Distribution of Semiparametric M-Estimators,” Journal of Econometrics 159, 252–266.
Ichimura, H. and W.K. Newey (2016): ”The Influence Function of Semiparametric Estimators,” CEMMAP working paper.
Lee, Lung-fei (2005): “A C(α)-type Gradient Test in the GMM Approach,” working paper.
Newey, W.K. (1990): ”Semiparametric Efficiency Bounds,” Journal of Applied Econometrics 5,
99-135.
Newey, W.K. (1991): ”Uniform Convergence in Probability and Stochastic Equicontinuity,”
Econometrica 59, 1161-1167.
Newey, W.K. (1994): ”The Asymptotic Variance of Semiparametric Estimators,” Econometrica
62, 1349-1382.
Newey, W.K. (1999): ”Consistency of Two-Step Sample Selection Estimators Despite Misspecification of Distribution,” Economics Letters 63, 129-132.
Newey, W.K., and D. McFadden (1994): “Large Sample Estimation and Hypothesis Testing,”
in Handbook of Econometrics, Vol. 4, ed. by R. Engle, and D. McFadden, pp. 2113-2241. North
Holland.
Newey, W.K., and J.L. Powell (1989) ”Instrumental Variable Estimation of Nonparametric
Models,” presented at Econometric Society winter meetings, 1989.
40

Newey, W.K., and J.L. Powell (2003) ”Instrumental Variable Estimation of Nonparametric
Models,” Econometrica 71, 1565-1578.
Newey, W.K., F. Hsieh, and J.M. Robins (1998): “Undersmoothing and Bias Corrected
Functional Estimation,” MIT Dept. of Economics working paper 72, 947-962.
Newey, W.K., F. Hsieh, and J.M. Robins (2004): “Twicing Kernels and a Small Bias Property
of Semiparametric Estimators,” Econometrica 72, 947-962.
Neyman, J. (1959): “Optimal Asymptotic Tests of Composite Statistical Hypotheses,” Probability
and Statistics, the Harald Cramer Volume, ed., U. Grenander, New York, Wiley.
Pakes, A. and G.S. Olley (1995): ”A Limit Theorem for a Smooth Class of Semiparametric
Estimators,” Journal of Econometrics 65, 295-332.
Powell, J.L., J.H. Stock, and T.M. Stoker (1989): ”Semiparametric Estimation of Index
Coefficients,” Econometrica 57, 1403-1430.
Robins, J.M., A. Rotnitzky, and L.P. Zhao (1994): ”Estimation of Regression Coefficients
When Some Regressors Are Not Always Observed,” Journal of the American Statistical Association 89: 846–866.
Robins, J.M. and A. Rotnitzky (1995): ”Semiparametric Efficiency in Multivariate Regression
Models with Missing Data,” Journal of the American Statistical Association 90:122–129.
Robins, J.M., A. Rotnitzky, and L.P. Zhao (1995): ”Analysis of Semiparametric Regression
Models for Repeated Outcomes in the Presence of Missing Data,” Journal of the American
Statistical Association 90,106–121.
Robins, J.M.,and A. Rotnitzky (2001): Comment on “Semiparametric Inference: Question
and an Answer Likelihood” by P.A. Bickel and J. Kwon, Statistica Sinica 11, 863-960.
Robins, J.M., A. Rotnitzky, and M. van der Laan (2000): ”Comment on ’On Profile
Likelihood’ by S. A. Murphy and A. W. van der Vaart, Journal of the American Statistical
Association 95, 431-435.
Robins, J., M. Sued, Q. Lei-Gomez, and A. Rotnitzky (2007): ”Comment: Performance of
Double-Robust Estimators When Inverse Probability’ Weights Are Highly Variable,” Statistical
Science 22, 544–559.
Robins, J.M., L. Li, E. Tchetgen, and A. van der Vaart (2008) ”Higher Order Influence
Functions and Minimax Estimation of Nonlinear Functionals,” IMS Collections Probability and
Statistics: Essays in Honor of David A. Freedman, Vol 2, 335-421.
Rust, J. (1987): ”Optimal Replacement of GMC Bus Engines: An Empirical Model of Harold
Zurcher,” Econometrica 55, 999-1033.
Santos, A. (2011): ”Instrumental Variable Methods for Recovering Continuous Linear Functionals,” Journal of Econometrics, 161, 129-146.
41

Scharfstein D.O., A. Rotnitzky, and J.M. Robins (1999): Rejoinder to “Adjusting For
Nonignorable Drop-out Using Semiparametric Non-response Models,” Journal of the American
Statistical Association 94, 1135-1146.
Severini, T. and G. Tripathi (2006): ”Some Identification Issues in Nonparametric Linear
Models with Endogenous Regressors,” Econometric Theory 22, 258-278.
Stoker, T. (1986): ”Consistent Estimation of Scaled Coefficients,” Econometrica 54, 1461-1482.
Tamer, E. (2003): ”Incomplete Simultaneous Discrete Response Model with Multiple Equilibria,”
Review of Economic Studies 70, 147-165.
van der Vaart, A.W. (1991): “On Differentiable Functionals,” The Annals of Statistics, 19,
178-204.
van der Vaart, A.W. (1998): Asymptotic Statistics, Cambride University Press, Cambridge,
England.
Wooldridge, J.M. (1991): “On the Application of Robust, Regression-Based Diagnostics to
Models of Conditional Means and Conditional Variances,” Journal of Econometrics 47, 5-46.

42

