Submitted to the Annals of Applied Statistics

ESTIMATING AVERAGE CAUSAL EFFECTS UNDER
GENERAL INTERFERENCE, WITH APPLICATION TO A
SOCIAL NETWORK EXPERIMENT

arXiv:1305.6156v3 [] 14 Jul 2017

By Peter M. Aronow∗ and Cyrus Samii†
Yale University∗ and New York University†
This paper presents a randomization-based framework for estimating causal effects under interference between units, motivated by
challenges that arise in analyzing experiments on social networks. The
framework integrates three components: (i) an experimental design
that defines the probability distribution of treatment assignments,
(ii) a mapping that relates experimental treatment assignments to
exposures received by units in the experiment, and (iii) estimands
that make use of the experiment to answer questions of substantive
interest. We develop the case of estimating average unit-level causal
effects from a randomized experiment with interference of arbitrary
but known form. The resulting estimators are based on inverse probability weighting. We provide randomization-based variance estimators
that account for the complex clustering that can occur when interference is present. We also establish consistency and asymptotic normality under local dependence assumptions. We discuss refinements
including covariate-adjusted effect estimators and ratio estimation.
We evaluate empirical performance in realistic settings with a naturalistic simulation using social network data from American schools.
We then present results from a field experiment on the spread of
anti-conflict norms and behavior among school students.

1. Introduction. We develop methods for analyzing an experiment in
which treatments are applied to individuals in a social network and causal
effects are hypothesized to transmit to peers through the network. Experimental and observational studies often involve treatments with effects that
“interfere” (Cox, 1958) across units through spillover or other forms of dependency. Such interference is sometimes considered a nuisance, and researchers may strive to design studies that isolate units as much as possible
from interference. However, such designs are not always possible. Furthermore, researchers may be interested in estimation of the spillover effects
themselves, as these effects may be of substantive importance. Other applications share structural similarities to the social network case. For example,
AMS 2000 subject classifications: Primary 60K35, 60K35; secondary 60K35
Keywords and phrases: interference, potential outcomes, causal inference, randomization inference, SUTVA, networks

1

2

ARONOW AND SAMII

an urban renewal program applied to one town may divert capital from
other towns, in which case the overall effect of the program may be ambiguous. Treatment effects may carry over from one time period to another, and
units have some chance of receiving treatment at any one of a set of points in
time. In these cases, we need methods to estimate effects of both direct and
indirect exposure to a treatment. Moreover, researchers may be interested
in understanding how such indirect effects vary depending on individuals’
characteristics.
This paper presents a general, randomization-based framework for estimating causal effects under these and other forms of interference. Interference represents a departure from the traditional causal inference scenario
wherein units are assigned directly to treatment or control, and the potential outcomes that would be observed for a unit in either the treatment
or control condition are fixed (Cole and Frangakis, 2009) and do not depend on the overall set of treatment assignments. The latter condition is
what Rubin (1990) refers to as the “stable unit treatment value assumption” (SUTVA). In the examples above, the traditional scenario is clearly
an inadequate characterization, as SUTVA would be violated. A more sophisticated characterization of treatment exposure and associated potential
outcomes must be specified. For the school field experiment, program participation was randomly assigned, but encouragement to support anti-conflict
norms could come from direct participation in the program as well as indirect
exposure via social network peers that participated in the program.
We start with theoretical results for an estimation framework that consists
of three components: (i) the experimental (or quasi-experimental) “design,”
which characterizes precisely the probability distribution of treatments assigned; (ii) an “exposure mapping,” which relates treatments assigned to
exposures received; and (iii) a set of causal estimands selected to make use
of the experiment to answer questions of substantive interest. For the case of
a randomized experiment under arbitrary but known forms of interference,
we provide unbiased estimators of average unit-level causal effects induced
by treatment exposure. We also provide estimators for the randomization
variance of the estimated average causal effects. These variance estimators
are assured of being conservative (that is, nonnegatively biased). We establish conditions for consistency and large-N confidence intervals based on
a normal approximation. We propose ratio-estimator-based and covariateadjusted refinements for increased efficiency. We assess finite-sample empirical performance with a naturalistic simulation on real-world school social
network data. The results demonstrate the reliability of the proposed methods in a realistic sample.

CAUSAL EFFECTS UNDER INTERFERENCE

3

We then present our analysis of a field experiment on the effects of a
program meant to promote anti-conflict norms and behavior among middle school students. In the experiment, schools were first randomly assigned
to host the anti-conflict program and then sets of students within the host
schools were randomly assigned to participate directly in the program. The
goal was to understand how attitudinal and behavioral effects on participants might transmit through their social network and affect their peers’
behavior.
2. Related Literature. Our framework extends from the foundational
work of Hudgens and Halloran (2008), who study two-stage, hierarchical
randomized trials in which some groups are randomly assigned to host
treatments; treatments are then assigned at random to units within the
selected groups, and interference is presumed to operate only within groups.
Hudgens and Halloran provide randomization-based estimators for groupaverage causal effects, conditional on assignment strategies that determine
the density of treatment within groups. Tchetgen-Tchetgen and VanderWeele (2012) extend Hudgens and Halloran’s results, providing conservative variance estimators, a framework for finite sample inference with binary outcomes, and extensions to observational studies. Liu and Hudgens
(2014) develop asymptotic results for such two-stage designs. Related to
these contributions is work by Rosenbaum (2007), which provides methods
for inference with exact tests under partial interference. Under hierarchical
treatment assignment and partial interference, estimation and inference can
proceed assuming independence across groups. In some settings, however,
the hierarchical structuring may not be valid, as with experiments carried
out over networks of actors that share links as a result of a complex, endogenous process. Bowers, Fredrickson and Panagopolous (2013) apply exact
tests to evaluate parameters in models of spillover processes. Such a testing
approach differs in its aims from ours, which focuses on estimating averages
of potentially heterogenous unit-level causal effects.
A key contribution of this paper is to go beyond the setting of hierarchical experiments with partial interference, and to generalize estimation
and inference theory to settings that exhibit arbitrary forms of interference and treatment assignment dependencies. In addition, our framework
allows the analyst to work with different estimands, including both types of
group-average causal effects defined by the authors above as well as average unit-level causal effects. Average unit-level causal effects are often the
estimand of primary interest, as is the case, for example, when exploring
unit-level characteristics that moderate the magnitude of treatment effects.

4

ARONOW AND SAMII

3. Treatment Assignment and Exposure Mappings. In this section, we define the first two components of our analytical framework: the
experiment design and exposure mapping. We focus on the case of a randomized experiment with an arbitrary but known exposure mapping. The
first step is to distinguish between (i) treatment assignments over the set
of experimental units and (ii) each unit’s treatment exposure under a given
assignment. Treatment assignments can be manipulated arbitrarily with the
experimental design. However, treatment-induced exposures may be constrained on the basis of the varying potential for interference of different
experimental units. For example, interference or spillover effects may spread
over a spatial gradient. If so, different treatment assignments may result in
different patterns of interference depending on where treatments are applied
on the spatial plane.
Formally, suppose we have a finite population U of units indexed by
i = 1, ..., N on which a randomized experiment is performed. Define a treatment assignment vector, z = (z1 , ..., zN )0 , where zi ∈ {1, .., M } specifies
which of M possible treatment values that unit i receives. An experimental
design contains a plan for randomly selecting a particular value of z from
the M N different possibilities with predetermined probability pz . Restricting our attention only to treatment assignments that can be generated by a
given experimental design, define Ω = {z : pz > 0}, so that Z = (Z1 , ..., ZN )0
is a random vector with support Ω and Pr(Z = z) = pz . Our analysis below focuses on the case where the design is known in the following sense:
Pr(Z = z) for all z ∈ Ω is known.
We define an exposure mapping as a function that maps an assignment
vector and unit-specific traits to an exposure value: f : Ω × Θ → ∆, where
θi ∈ Θ quantifies relevant traits of unit i. The exposure mapping construction is functionally equivalent to the “effective treatments” function used
by Manski (2013), though we find it helpful to denote separately the unitspecific attributes, θi , that feed into the exposure mapping, f (·). In applications we consider below, θi is unit i’s row in a network adjacency matrix.
More complex exposure mappings could take in θi s that encode other traits
of units and their peers—not only network ties, but also differences in age,
gender, or other unit-level characteristics. Or, θi could encode not only firstdegree peer connections, but second-degree connections, third-degree, and
so on. The codomain ∆ contains all of the possible treatment-induced exposures that might be induced in the experiment. The contents of ∆ depend on
the nature of interference. These exposures may be represented as vectors,
discrete classes, or scalar values. As we will show formally below, each of
the distinct exposures in ∆ may give rise to distinct potential outcomes for

CAUSAL EFFECTS UNDER INTERFERENCE

5

each unit in U . The estimation of causal effects under interference amounts
to using information about treatment assignments, which come from the experiment’s design, to estimate effects defined in terms of treatment-induced
exposures, which result from the interaction of the design (captured by Z)
and other underlying features of the population (captured by f and the θi s).
To make things more concrete, consider some examples of exposure mappings. The Neyman-Rubin causal model typically considers inference under
an exposure mapping in which we set ∆ = {1, ..., M } and f (z, θi ) = f (z) =
zi for all i. This model has been a workhorse for much of the causal inference literature (Neyman, 1990 [1923]; Rubin, 1978; Holland, 1986; Imbens
and Rubin, 2015). An exposure mapping that allowed for completely arbitrary interference would be one for which |∆| = |Ω| × N , in which case each
unit has a unique type of exposure under each treatment assignment, and
f (z, θi ) would be unique for each z. If such an exposure mapping were valid,
then it is clear that there would be no meaningful way to use the results of
the experiment to estimate average exposure-specific effects (although other
types of causal effects may admit well-behaved estimators). Instead, the analyst must use substantive judgment about the extent of interference to fix
a mapping somewhere between the traditional randomized experiment and
completely arbitrary exposure mappings in order to carry out analyses under
interference. For example, Hudgens and Halloran (2008) consider a setting
that allows unit i’s exposure to vary with each possible treatment assignment
within i’s group, but, where conditional on the assignment for i’s group, i’s
exposure does not vary in the treatment assignments of other groups. Then,
θi would be unit i’s group index, |∆| would equal the largest number of
assignment possibilities for any group, and f (.) would map each assignment
possibility for units in unit i’s group to a separate exposure condition. The
types of effects that Hudgens and Halloran (2008) construct are ones that
average over these exposures for each unit. Below we discuss implications of
using an exposure model that does not fully account for interference, drawing
connections back to the estimators in Hudgens and Halloran (2008). In the
simulation study below and in the application, we provide more examples of
exposure mappings. Finally, our characterization of exposures is “reduced
form” in that it does not distinguish between the channels through which interference occurs (Ogburn and VanderWeele, 2014). The exposure mapping
does not distinguish between effects that emanate directly from treatments
being assigned to peers or that are mediated by changes in peers’ outcomes
(Eckles, Karrer and Ugander, 2014, pp. 8-9; Manski, 1995, Ch. 7).
Units’ probabilities of falling into one or another exposure condition are
crucial for the estimation strategy that we develop below. Define the expo-

6

ARONOW AND SAMII

sure that unit i receives as Di = f (Z, θi ), a random variable with support
∆i ⊆ ∆ and for which Pr(Di = d) = πi (d). Note that because |∆| ≤ |Ω|×N ,
∆ is a finite set of K ≤ |Ω| × N values, such that ∆ = {d1 , ..., dK }. Then
for each unit, i, we have a vector of probabilities, (πi (d1 ), ..., πi (dK ))0 = π i .
Invoking Imbens (2000)’s generalized propensity score, we call π i the generalized probability of exposure for i. A unit i’s generalized probability of
exposure tells us the probability of i being subject to each of the possible
exposures in {d1 , ..., dK }. We have
X
X
πi (dk ) =
I(f (z, θi ) = dk ) Pr(Z = z) =
pz I(f (z, θi ) = dk ).
z∈Ω

z∈Ω

Thus the generalized probability of exposure for unit i is also known exactly.
Each component probability, πi (dk ), is equal to the expected proportion of
treatment assignments that induce exposure dk for unit i.
Below, we will refer to joint exposure probabilities when discussing variance estimators. That is, we define πij (dk ) as the probability of the joint
event that both units i and j are subject to exposure dk , and we define
πij (dk , dl ) as the probability of the joint event that units i and j are subject
to exposures dk and dl , respectively. To compute both individual and joint
exposure probabilities from the experiment’s design, first define the N × |Ω|
matrix
Ik = [I(f (z, θi ) = dk )]

z∈Ω

=

i=1,...,N







I(f (z1 , θ1 ) = dk )
I(f (z1 , θ2 ) = dk )
..
.

I(f (z2 , θ1 ) = dk )
I(f (z2 , θ2 ) = dk )
..
.

I(f (z1 , θN ) = dk ) I(f (z2 , θN ) = dk )

...
...
..
.


I(f (z|Ω| , θ1 ) = dk )
I(f (z|Ω| , θ2 ) = dk ) 

,

I(f (z|Ω| , θN ) = dk )

which is a matrix of indicators for whether units are in exposure condition
k over possible assignment vectors. Define the |Ω| × |Ω| diagonal matrix
P = diag(pz1 , pz2 , ..., pz|Ω| ). Then



Ik PI0k = 


π1 (dk )
π21 (dk )
..
.

π12 (dk )
π2 (dk )
..
.

. . . π1N (dk )
. . . π2N (dk )
..
.

πN 1 (dk ) πN 2 (dk )

πN (dk )




,


is an N × N symmetric matrix with individual exposure probabilities, the
πi (dk )s, on the diagonal and joint exposure probabilities, the πij (dk )s, on

7

CAUSAL EFFECTS UNDER INTERFERENCE

the off-diagonals. The non-symmetric N × N

0
π12 (dk , dl )
 π21 (dk , dl )
0

Ik PI0l = 
..
..

.
.
πN 1 (dk , dl ) πN 2 (dk , dl )

matrix
. . . π1N (dk , dl )
. . . π2N (dk , dl )
..
.




,


0

yields all joint probabilities across exposure conditions k and l. The zeroes
on the diagonal are due to the fact that a unit cannot be subject to multiple
exposure conditions at once.
In practice, |Ω| may be so large that it is impractical to construct Ω
to compute the π i s and the joint probability matrices exactly. One may
nonetheless approximate the π i s and joint probabilities with arbitrary precision through simulation; that is, produce R random replicate zs based on
the randomization plan. From these R replicates, we can construct an N ×R
indicator matrix, bIk , for each of the k = 1, ..., K exposure conditions. Then,
an estimator for Ik PI0k that incorporates mild additive smoothing to ensure
non-zero marginal probability estimates is (Îk Î0k + ιN )/(R + 1), where ιN
is an N × N identity matrix. Similarly, an estimator for Ik PI0l , which does
not admit additive smoothing due to zero joint inclusion probabilities, is
(Îk Î0l )/R.
Proposition 3.1.

As R → ∞,
a.s.

a.s.

(Îk Î0k + ιN )/(R + 1) → Ik PI0k , and (Îk Î0l )/R → Ik PI0l .
All proofs appear in the appendix. Rates of convergence of bIkbI0k /R are discussed in Fattorini (2006) and Aronow (2013). Below we give guidance on
selecting a value of R based on a bound on the relative bias for an estimator
of a target quantity.
4. Average Potential Outcomes and Causal Effects. We develop
the case of estimating average unit-level causal effects of exposures. An average unit-level causal effect is defined in terms of a difference between the
average of units’ potential outcomes under one exposure versus the average under another exposure. The starting point is the estimation of average
potential outcomes under each of the exposure conditions. With that, the
analyst is in principle free to compute a variety of causal quantities of interest, not just average unit-level causal effects. For example, one could consider
effects that are defined as differences between the average of potential outcomes under one set of exposures versus the average under another set of

8

ARONOW AND SAMII

exposures. The direct, indirect and overall effects of Hudgens and Halloran (2008) are defined in this way using the construction of the “individual average potential outcome.” The hierarchical designs that they consider
are specifically tailored to ensure that estimators for such effects are nonparametrically identified. While our focus is on estimating exposure-specific
causal effects that are defined for arbitrary designs, such design-specific estimators can certainly be derived and analyzed using the framework developed
here. Our focus on the average of unit-level, exposure-specific causal effects
is due to its being the natural extension of the “average treatment effect”
that is the focus of much current causal inference and program evaluation
literature (e.g., Imbens and Wooldridge, 2009).
Suppose all units have non-zero probabilities of being subject to each of
the K exposures: 0 < πi (dk ) < 1 for all i and k. (When πi (dk ) = 0 for some
units, then design-based estimation of average potential outcomes and causal
effects must be restricted to the subset of units for which πi (dk ) > 0.) In the
most general terms, each z ∈ Ω can generate a potential outcome for unit i.
We label the randomization potential outcome of unit i associated with z as
yir (z). These randomization potential outcomes are fixed for all units in the
population and do not depend on the value of randomized treatment, Z. A
condition that we use in our analysis below is that the exposure mapping
fully characterizes interference:
Condition 1 (Properly specified exposure mapping). For all i ∈
{1, ..., N } and z, z0 ∈ Ω such that f (z, θi ) = f (z0 , θi ), yir (z) = yir (z0 ).
Given Condition 1, each unit i has |∆| = K potential outcomes, which we
can write in terms of the exposure conditions as (yi (d1 ), ..., yi (dK )), where
yi (dk ) = yir (z), ∀i ∈ {1, ..., N }, k ∈ {1, ..., K}, and z ∈ Ω such that f (z, θi ) =
dk .
Let Yi be the observed outcome for unit i. We assume the following consistency condition that relates the observed data to potential outcomes under
the exposure model (VanderWeele, 2009):
Condition 2 (Consistent potential outcomes).
Yi =

K
X

I(Di = dk )yi (dk ), ∀i ∈ {1, ..., N }.

k=1

Although SUTVA is violated at the level of treatment assignment (i.e., the
individual zi values), Conditions 1 and 2 restore a form of SUTVA with

CAUSAL EFFECTS UNDER INTERFERENCE

9

respect to exposures, in a manner that is conceptually similar to Hudgens
and Halloran (2008)’s stratified interference assumption. Below we examine
implications of violating Conditions 1 and 2—i.e., implications of misspecifying the exposure mapping. Throughout, unless otherwise noted, we will
be assuming Conditions 1 and 2.
P
1 T
We seek estimates for all k of µ(dk ) = N1 N
i=1 yi (dk ) = N y (dk ), where
T
y (dk ) is the total of the potential outcomes under dk . This allows us to
define an average causal effect of being in exposure condition dk as compared
to being in condition dl as
τ (dk , dl ) =

N
N
1 X
1 X
yi (dk ) −
yi (dl ).
N
N
i=1

i=1

The number of units in the population, N , is fixed, but we cannot estimate
y T (dk ) directly, as we only observe yi (dk ) for those with Di = dk . However,
by design, the collection of units for which we observe yi (dk ) is an unequalprobability without-replacement sample from (y1 (dk ), ..., yN (dk )), with the
sampling probabilities known exactly. By Horvitz and Thompson (1952), a
design-based estimator for y T (dk ) is the inverse probability weighted estimator
T
yd
HT (dk ) =

(1)

N
X

I(Di = dk )

i=1

Yi
πi (dk )

Below, we consider variance-reducing refinements to this estimator. We start
T
with an analysis of yd
HT (dk ) because it very clearly reveals first-order issues
for estimation under interference. Estimator 1 is unbiased, and its variance
is characterized in Lemma 4.1.
Lemma 4.1.
T
E [yd
HT (dk )] =

N
X

yi (dk )

i=1

T
Var [yd
HT (dk )] =

(2)

N
X



yi (dk )
πi (dk )[1 − πi (dk )]
πi (dk )

i=1
N X
X

2

[πij (dk ) − πi (dk )πj (dk )]

+

i=1 j6=i

yi (dk ) yj (dk )
.
πi (dk ) πj (dk )

10

ARONOW AND SAMII

Above we indicated that one can approximate Ik PI0k with (Îk Î0k +ιN )/(R+
1), which has diagonal elements,
πbi (dk ) =

Xi + 1
,
R+1

P
where Xi = R
r=1 Ir (f (zr , θi ) = dk ) and r = 1, ..., R indexes the replicates.
Define the Horvitz-Thompson estimator that uses the πbi (dk ) estimates:
T
y\
HT,R (dk ) =

N
X

I(Di = dk )

i=1

Yi
.
πbi (dk )

Following Fattorini (2006), the following proposition provides guidance on
T (d ).
choosing R in terms of a bound on the relative bias for estimating yc
k
Proposition 4.1.

T
The relative bias for y\
HT,R (dk ) is bounded as

T
T
E [y\
HT,R (dk )] − y (dk )

y T (dk )

≤ (1 − π0 (dk ))R+1 ,

where π0 (dk ) = mini {πi (dk )}.
For a relative bias target of b and given some approximation of π0 (dk ), the
bound implies selecting a number of replicates R ≥ log(b)/ log(1−π0 (dk ))−1.
Thus, for b = .005 and π0 (dk ) = .0005, this would imply at least 10,593
replicates. Given the apparent computational feasibility of producing enough
replicates so as to render relative biases negligible, from here on our analysis
assumes that we are working with Ik PI0k and Ik PI0l .
Given the estimator of the total of the N potential outcomes under expod
T
sure dk , a natural estimator for the mean is thus µd
HT (dk ) = (1/N )yHT (dk ),
T (d )]. This allows us to conwith variance Var (µd (d )) = (1/N 2 )Var [yd
HT

k

HT

k

struct the difference in estimated means
(3)

τd
d
d
HT (dk , dl ) = µ
HT (dk ) − µ
HT (dl ) =

i
1 hd
T (d ) − yd
T (d )
yHT
k
HT l
N

P
which is an estimator of τ (dk , dl ) = N1 N
i=1 [yi (dk ) − yi (dl )], the average
unit-level causal effect of exposure k versus exposure l.
Proposition 4.2.
(4)

N
N
1 X
1 X
yi (dk ) −
yi (dl )
E [τd
HT (dk , dl )] =
N
N
i=1

i=1

11

CAUSAL EFFECTS UNDER INTERFERENCE

Var (τd
HT (dk , dl )) =
(5)

1 n
d
T
T
Var [yd
HT (dk )] + Var [yHT (dl )]
N2
o
T (d ), yd
T (d )] ,
−2Cov [yd
HT

k

HT

l

where
d
T
T
Cov [yd
HT (dk ), yHT (dl )] =

N X
X
yi (dk ) yj (dl )
[πij (dk , dl ) − πi (dk )πj (dl )]
πi (dk ) πj (dl )
i=1 j6=i

−

(6)

N
X

yi (dk )yi (dl ).

i=1

Expressions (2) and (6) allow us to see the conditions under which exact
variances are identified. So long as all joint exposure probabilities are nonT
zero (that is, πij (dk ) > 0 for all i, j), unbiased estimators for Var [yd
HT (dk )]
are identified for the population U . Because we only observe one potential outcome for each unit, the last sum in (6) is always unidentified,
T (d ), yd
T (d )] is always unidentified. This is a familiar
and thus Cov [yd
k

HT

HT

l

problem in estimating the randomization variance for the average treatment effect—e.g., Neyman (1990 [1923]) or Freedman, Pisani and Purves
T (d )] is unidentified.
(1998, A32-A34). If π (d ) = 0 for some i, j, Var [yd
ij

k

HT

k

Similarly, if πij (dk , dl ) = 0 for some i, j, then additional components of
T (d )] are unidentified. Nonetheless, we can always identify
T (d ), yd
Cov [yd
HT

k

HT

l

d
d
T
T
T
estimators for Var [yd
HT (dk )] and Cov [yHT (dk ), yHT (dl )] that are guaranteed
to have nonnegative bias. Thus, we can always identify a conservative approximation to the exact variances. We discuss this in the next section.
5. Variance Estimators. We derive conservative estimators for both
T
Var [yd
HT (dk , dl )]. The formulations in this section follow
HT (dk )] and Var [τd
from Aronow and Samii (2013) which considers conservative variance estimation for generic sampling designs with some zero pairwise inclusion probabilities. Although not necessarily unbiased, the estimators we present here
are guaranteed to have a nonnegative bias relative to the randomization
distributions of the estimators.
Given πij (dk ) > 0 for all i, j, the Horvitz-Thompson estimator for

12

ARONOW AND SAMII

T
Var [yd
HT (dk )] is
T
d [yd
Var
HT (dk )] =

X


I(Di = dk )[1 − πi (dk )]

i∈U

+

X X

Yi
πi (dk )

2

I(Di = dk )I(Dj = dk )

i∈U j∈U \i

×

(7)

πij (dk ) − πi (dk )πj (dk ) Yi
Yj
.
πij (dk )
πi (dk ) πj (dk )

T
d [yd
If πij (dk ) > 0 for all i, j, then E [Var
HT (dk )]] =

Lemma 5.1.
T
Var [yd
HT (dk )].

Lemma 5.1 follows from unbiasedness of the Horvitz-Thomspon estimator for
measurable designs. Then an unbiased estimator for the variance of µd
HT (dk )
d
2
T
d [µd
d
is Var
HT (dk )] = (1/N )Var [yHT (dk )].
In the case where πij (dk ) = 0 for some i, j, the Horvitz-Thompson estiT (d )] is not unbiased, but its bias is readily characterized.
mator of Var [yd
HT

k

T
d [yd
Proposition 5.1. If πij (dk ) = 0 for some i, j, then E [Var
HT (dk )]] =
T (d )] + A, where
Var [yd
HT

k

A=

X

X

yi (dk )yj (dk ).

i∈U j∈{U \i:πij (dk )=0}

A proof for Proposition 5.1 follows from Aronow and Samii (2013, Proposition 1, reproduced in the appendix below).
d [µd
Note that Var
HT (dk )] is guaranteed to have nonnegative bias when
yi (dk )yj (dk ) ≥ 0 for all i, j with πij (dk ) = 0. The bias will be small when
the terms in the sum tend to offset each other, as when the relevant yi (dk )
and yj (dk ) values are centered on 0 and have low correlation with each other.
(This notation requires that we define 0/0 = 0.)
Another option is to use the following correction term (derived via Young’s
inequality),
#
"
2
2
X
X
I(D
=
d
)Y
I(D
=
d
)Y
j
k
j
i
k
i
c2 (dk ) =
+
,
A
2πi (dk )
2πj (dk )
i∈U j∈{U \i:πij (dk )=0}

c2 (dk ) = 0 if πij (dk ) > 0 for all i, j.
noting that A

CAUSAL EFFECTS UNDER INTERFERENCE

13

Proposition 5.2.
h
i
T (d )] + A
T
d [yd
c2 (dk ) ≥ Var [yd
E Var
k
HT
HT (dk )],
A proof for Propositon 5.2 follows directly from Aronow and Samii (2013,
\
Corollary
below). Then let Var
d
A [µ
HT (dk )] =
h 2, reproduced in the appendix
i
d
2
T
d [y (dk )] + A
c2 (dk ) . Var
\
(1/N ) Var
d
A [µ
HT (dk )] then provides a conservaHT

tive estimator for the variance of the estimated average of potential outcomes
under exposure dk .
d
T
T
As discussed above, Cov [yd
HT (dk ), yHT (dl )] is unidentified, which is to
say that there exist no unbiased or consistent estimators for this quantity.
However, we can compute an approximation that is guaranteed to have expectation less than or equal to the true covariance, providing a conservative
(here, nonnegatively biased) estimator for Var (τd
HT (dk , dl )). For the case
where πij (dk , dl ) > 0 for all i, j such that i 6= j, we propose the HorvitzThompson-type estimator for the covariance
"
X X I(Di = dk )I(Dj = dl ) Yi
Yj
d
d
T
T
d [y (dk ), y (dl )] =
Cov
HT
HT
πij (dk , dl )
πi (dk ) πj (dl )
i∈U j∈U \i
#
× [πij (dk , dl ) − πi (dk )πj (dl )]
(8)

X  I(Di = dk )Y 2 I(Di = dl )Y 2 
i
i
−
+
.
2πi (dk )
2πi (dl )
i∈U

Proposition 5.3. If πij (dk , dl ) > 0 for all i, j such that i 6= j,
i
h
d
d
d
T
T
T
T
d
E Cov [yHT (dk ), yHT (dl )] ≤ Cov [yd
HT (dk ), yHT (dl )],
A proof for Proposition 5.3 follows from noting that the term on the second
line in expression (8) has expected value less than or equal to the quantity in
the last line of expression (6), again via Young’s inequality. See Aronow and
Samii (2013, Proposition 2, reproduced in the appendix below) for greater
detail.
d
T
T
d [yd
Cov
HT (dk ), yHT (dl )] is exactly unbiased if, for all i ∈ U , yi (dl ) = yi (dk ),
implying no effect associated with condition l relative to condition k.
Proposition 5.4. If πij (dk , dl ) > 0 for all i, j such that i 6= j and for
all i ∈ U , yi (dl ) = yi (dk )
h
i
d
d
d
T
T
T
T
d [yd
E Cov
HT (dk ), yHT (dl )] = Cov [yHT (dk ), yHT (dl )],

14

ARONOW AND SAMII

A proof follows from Aronow and Samii (2013, Corollary 1, reproduced in
the appendix below).
For the case where πij (dk , dl ) = 0 for some i, j and k, l, we can refine the
expression for the covariance given in (6) to
"
X
X
yi (dk ) yj (dl )
d
d
T
T
Cov [yHT (dk ), yHT (dl )] =
πi (dk ) πj (dl )
i∈U j∈{U \i:πij (dk ,dl )>0}
#
× [πij (dk , dl ) − πi (dk )πj (dl )]
−

(9)

X

X

yi (dk )yj (dl ),

i∈U j∈{U :πij (dk ,dl )=0}

where the term on the last line subsumes the term on the last line in expression (6). This leads us to propose a more general estimator for the covariance
"
X
X
I(Di = dk )I(Dj = dl )
d
d
T
T
\
Cov
A [yHT (dk ), yHT (dl )] =
πij (dk , dl )
i∈U j∈{U \i:πij (dk ,dl )>0}

×

Yj
Yi
πi (dk ) πj (dl )
#

× [πij (dk , dl ) − πi (dk )πj (dl )]
−

(10)

I(Di = dk )Yi2
2πi (dk )
i∈U j∈{U :πij (dk ,dl )=0}
#
I(Dj = dl )Yj2
+
.
2πj (dl )

X

X



Proposition 5.5.
h
i
d
T (d ), yd
T (d )] ≤ Cov [yd
T (d ), yd
T (d )],
\
E Cov
[
y
A
k
l
k
l
HT

HT

HT

HT

A proof again follows from the fact the term in the last line in (10) has
expected value no greater than the term in the last line of (9) by Young’s
inequality.
Based on the variance expressions and correction terms defined above, we

15

CAUSAL EFFECTS UNDER INTERFERENCE

obtain a conservative variance estimator for Var (τd
HT (dk , dl )) as
n
1
T
T
d [τd
d [yd
c
d d
c
Var
Var
HT (dk , dl )] =
HT (dk )] + A2 (dk ) + Var [yHT (dl )] + A2 (dl )
N2
o
d
d
T
T
\
(11)
−2Cov
A [yHT (dk ), yHT (dl )] .
Proposition 5.6.
h
i
d [τd
E Var
HT (dk , dl )] ≥ Var [τd
HT (dk , dl )],
The result follows from Proposition 5.2, Proposition 5.5, and linearity of
expectations.
6. Asymptotics and Intervals. Consider a sequence of nested populations indexed by size N , (UN ). To define a notion of asymptotic growth, we
let N tend to infinity, allowing for the experimental design to be reapplied
anew to each UN , subject to the conditions defined below (Brewer, 1979;
Isaki and Fuller, 1982). Consistency and the asymptotic validity of Waldtype confidence intervals will then follow from restrictions on the growth
process of the design and exposure mapping.
6.1. Consistency. We first establish conditions for the estimator
τd
HT (dk , dl ) to converge to τ (dk , dl ) as N grows. We will show that, under
p
two regularity conditions, τd
HT (dk , dl ) − τ (dk , dl ) −→ 0 as N → ∞.
Condition 3 (Boundedness of potential outcomes and exposure probabilities). Potential outcomes and exposure probabilities are bounded, so
that, for all values i and dk , |yi (dk )| ≤ c1 < ∞ and |1/πi (dk )| ≤ c2 < ∞.
Condition 3 can be relaxed, though Condition 4 would likely need to be
strengthened accordingly.
We will also make an assumption about the amount of dependence in exposure conditions in the population. Define a pairwise dependency indicator
gij such that if gij = 0, then Di ⊥
⊥ Dj , else let gij = 1.
Condition 4 (Restrictions on pairwise dependence).
o(N 2 ).

PN PN
i=1

j=1 gij

=

Condition 4 entails that, as N grows, the amount of pairwise clustering in
exposure conditions induced by the design and exposure mapping is limited
in scope. As units are added to the sample, the number of new non-zero
entries in the expanding pairwise correlation matrix of exposures should be
limited by the order condition.

16

Proposition 6.1.
0 as N → ∞.

ARONOW AND SAMII
p

Given Conditions 3 and 4, τd
HT (dk , dl )−τ (dk , dl ) −→

6.2. Confidence Intervals. We now establish conditions for the asymptotic validity of Wald-type confidence intervals under stricter conditions
on the asymptotic growth process. Consistency for the variance estimators,
asymptotic normality, and therefore asymptotic validity of confidence intervals, follow straightforwardly when the amount of dependence across units
in the population is limited.
We shall assume that Condition 3 holds, but will strengthen Condition
4 to ensure that dependence across exposures is limited in scope. Unlike
Condition 4, we will exploit joint independence of observations rather than
pairwise independence. Define a binary dependency indicator hij over all
pairs (i, j) ∈ {1, 2, ..., N } × {1, 2, ..., N }, where hij satisfies the following: for
any pair of disjoint sets Γ1 and Γ2 ⊆ {1, ..., N } such that there exists no
pair (i, j) with hij = 1 and either (i) i ∈ Γ1 and j 6∈ Γ2 , (ii) j ∈ Γ1 and
i 6∈ Γ2 , (iii) i 6∈ Γ1 and j ∈ Γ2 , or (iv) j 6∈ Γ1 and i ∈ Γ2 , {Di , i ∈ Γ1 } and
{Di , i ∈ Γ2 } are independent.
Condition 5 (Local dependence).
There exists a finite constant m such
PN
that, for all UN , i ∈ 1, ..., N , j=1 hij ≤ m.
Condition 5 is equivalent to assuming that dependencies across exposures
can be represented by a dependency graph such that the maximal degree
of each unit tends to be limited relative to N . Condition 5 will allow us
to straightforwardly invoke a central limit theorem for random fields as
derived via Stein’s Method (Chen and Shao, 2004, Example 2.4.1). (The
authors thank Betsy Ogburn for the suggestion of the use of Stein’s Method
in this setting.) Finite m ensures that our variance estimators will converge
at
fast rate. Note that Condition 5 subsumes Condition 4, as
PN
PNa sufficiently
g
=
O(N ) when Condition 5 holds.
ij
j=1
i=1
It is illustrative to consider settings where Condition 5 holds. For
Bernoulli-randomized designs, Condition 5 would hold if interference were
characterized by first-order dependence on a graph connecting units and
network degrees were bounded above by some value m. Condition 5 also
generalizes the partial interference setting considered by, e.g., Sobel (2006)
and Liu and Hudgens (2014) given finite subpopulations across which interference is localized (in this case, m would be the size of the largest subpopulation). However, Condition 5 would be violated if changing the treatment
assigned to one unit would affect the exposure received by all N units. In
comparing Conditions 4 and 5, note that Condition 4 is a restriction on the

17

CAUSAL EFFECTS UNDER INTERFERENCE

order of growth of pairwise dependencies, while Condition 5 requires local
dependence. The latter condition is more restrictive as it imposes conditions
on all higher-order joint inclusion probabilities. It is possible that Condition
4 could hold, but Condition 5 would be violated, if, for example, there exists a single unit for which the number of associated pairwise dependencies
tended to infinity in N .
Condition 6 (Nonzero limiting variance.).
where c > 0.

p

N Var [τd
HT (dk , dl )] → c,

Convergence of N Var [τd
HT (dk , dl )] to a nonnegative constant is generally ensured by Conditions 3 and 5, sufficient for root-n consistency of τd
HT (dk , dl ).
Condition 6 is a mild regularity condition that ensures that this constant is
positive, and rules out degenerate cases (e.g., all outcomes are zero).
Proposition 6.2.
constructed as

Given Conditions 3, 5, and 6, Wald-type intervals

q
d HT (dk , dl )]
τd
HT (dk , dl ) ± z1−α/2 Var [τd
will tend to cover τHT (dk , dl ) at least 100(1 − α)% of the time for large N .
7. Refinements. The mean and difference-in-means estimators presented thus far are unbiased by sample theoretic arguments, and we have
derived conservative variance estimators. However, we may wish to improve
efficiency by incorporating auxiliary covariate information. In addition, by
analogy to results from the unequal probability sampling literature, ratio approximations of the Horvitz-Thompson estimator may significantly reduce
mean squared error with little cost in terms of bias (Särndal, Swensson and
Wretman, 1992, pp. 181-184). We discuss such refinements here.
7.1. Covariance Adjustment. Auxiliary covariate information may help
to improve efficiency. A first method of covariance adjustment is based on
the so-called “difference estimator” (Raj, 1965; Särndal, Swensson and Wretman, 1992, Ch. 6). Covariance adjustment of this variety can reduce the
randomization variance of the estimated exposure means and average causal
effects without compromising unbiasedness. In addition, the difference estimator addresses the problem of location non-invariance that afflicts HorvitzThompson-type estimators (Fuller, 2009, pp. 9-10). The estimator requires
prior knowledge of how outcomes relate to covariates, perhaps obtained from
analysis of auxiliary datasets.

18

ARONOW AND SAMII

Assume an auxiliary covariate vector xi is observed for each i. We have
some predefined function g (xi , ξi (dk )) → R, where ξi is a parameter vector.
Ideally g(·) is calibrated on auxiliary data to produce values that approximate yi (dk ). We assume Cov [g (xi , ξi (dk )) , Zi ] = 0 as a sufficient condition
for unbiasedness. Define
(12)
N
N
N
X
X
Yi
g (xi , ξi (dk )) X
c
T
yG (dk ) =
I(Di = dk )
−
+
I(Di = dk )
g (xi , ξi (dk )) ,
πi (dk )
πi (dk )
i=1

i=1

i=1

y T (dk )

which is unbiased for
by
" N
#
N
X
g (xi , ξi (dk )) X
+
E −
g (xi , ξi (dk )) = 0.
I(Di = dk )
πi (dk )
i=1

i=1

Define i (dk ) = Yi − g (xi , ξi (dk )) for cases with Di = dk . Then, by substitution,
(13)

T
yc
G (dk ) =

N
X
i=1

N

i (dk ) X
+
g (xi , ξi (dk )) .
I(Di = dk )
πi (dk )
i=1

c
T
T
Estimation proceeds as above using yc
G (dk ) in place of y (dk ) to estimate
T
y (dk ). Middleton and Aronow (2011) and Aronow and Middleton (2013)
T (d ) is location invariant. Variance estimation proceeds
demonstrate that yc
G

k

as in Section 5, using i (dk ) in place of yi (dk ) so long as g (xi , ξi (dk )) is fixed.
An approximation to the difference estimator is given by regression adjustment using the data at hand. Regression can be thought of as a way to
automate selection of the parameters in the difference estimator. In doing so,
unbiasedness is compromised although the regression estimator is typically
consistent (Särndal, Swensson and Wretman, 1992, pp. 225-239). We may
use weighted least squares to estimate a sensible parameter vector. For some
common experimental designs, the least squares criterion will be optimal
(Lin, 2013), and weighting by 1/πi (dk ) ensures that the regression proceeds
on a sample representative of the population of potential outcomes. With
additional details on Ik and g(.), it is possible to estimate optimal parameter vectors (Särndal, Swensson and Wretman, 1992, pp. 219-244), though
such values will typically be close to those produced by the weighted least
squares estimator (barring unusual and extreme forms of clustering).
Define an estimated parameter vector associated with exposure condition
dk
X
1
b k ) = arg min
ξ(d
[Yi − g (xi , ξ(dk ))]2 ,
πi (dk )
ξ(dk )
i:Di =dk

CAUSAL EFFECTS UNDER INTERFERENCE

19

where g(.) is the specification for the regression of Yi on I(Di = dk ) and xi .
Then the regression estimator for the total is


b k)
N
N


Yi − g xi , ξ(d
X
X
c
T
b k) .
(14)
yR (dk ) =
+
I(Di = dk )
g xi , ξ(d
πi (dk )
i=1

i=1

d
T
T
Estimation proceeds as above using yc
R (dk ) in place of yHT (dk ) to estimate
y T (dk ). Under weak regularity conditions on g(.), a variance estimator based
T (d ) is consistent (Särndal, Swensson and
on a Taylor linearization of yc
R

k

Wretman, 1992, pp. 236-237). The linearized variance estimator can be comb k )), for the yi (dk )
puted by substituting the residuals, ei = yi (dk ) − g(xi , ξ(d
terms in constructing the variance estimator given in expression (11).
7.2. Hajek Ratio Estimation Via Weighted Least Squares. The Hájek
(1971) ratio estimator is a refinement of the standard Horvitz-Thompson
estimator that often facilitates efficiency gains at the cost of some finite
N bias and complications in variance estimation. Let us first consider the
problem that the Hajek estimator is designed to resolve. The high variance
of µd
HT (dk ) is often driven by the fact that some randomizations may yield
units with exceptionally high values of the weights 1/πi (dk ). The Hajek refinement allows the denominator of the estimator to vary according to the
sum of the weights 1/πi (dk ), thus shrinking the magnitude of the estimator
when its value is large, and raising the magnitude of the estimator when its
value is small. The Hajek ratio estimator is
PN
(15)

µc
H (dk ) =

i=1 I(Di
PN
i=1 I(Di

= dk ) πiY(di k )
1
= dk ) πi (d
k)

.

P
1
Note that E [ N
i=1 I(Di = dk ) πi (dk ) ] = N , so that the Hajek estimator is
the ratio of two unbiased estimators. It is well known that the ratio of two
unbiased estimators is not an unbiased estimator of the ratio. However, the
bias will tend to be small relative to the estimator’s sampling variability,
and we may place bounds on its magnitude.
By Hartley and Ross (1954) and Särndal, Swensson and Wretman (1992,
p. 176),
v
!
u
N
u
X
1
1
tVar
|E [µc
I(Di = dk )
Var (µc
H (dk ))
H (dk )] − µ(dk )| ≤
N
πi (dk )
i=1

20

ARONOW AND SAMII

Under Conditions 3 and 4, both variances will converge to zero, and the
bias ratio will converge to zero. Practically speaking, the Hajek estimator
can be computed using weighted least squares, with covariance adjustment
through weighted least squares residualization. Variance estimation proceeds
via Taylor linearization (Särndal, Swensson and Wretman, 1992, pp. 172176). A linearized variance estimator can be computed by substituting the
residuals, ui = yi (dk ) − µc
H (dk ), for the yi (dk ) terms in constructing the
variance estimator given in expression (11).
8. Misspecification. Recall Condition 1, which states that the exposure mapping fully characterizes interference. Here we examine what happens when this assumption fails; e.g., there is interference between units that
is not fully characterized by the exposure mapping. By “misspecification”
of the exposure mapping, we refer to the situation in which the condition
Di = dk may be consistent with multiple potential outcomes for some i. As
in Section 4, we have randomization potential outcomes for unit i as yir (z)
for all z ∈ Ω.
Condition 7 (Misspecification). There exists some i ∈ {1, ..., N } and
0
0
r
r 0
z,
z
P ∈ Ω such rthat f (z, θi ) = f (z , θi ) and yi (z) 6= yi (z ). Then Yi =
z∈Ω I(Z = z)yi (z), ∀i ∈ {1, ..., N }.
The following proposition shows the implications of misspecification for
the potential outcome population total estimator given in expression (1).
T
Proposition 8.1. Define yd
HT (dk ) as above but suppose Condition 7
instead of Conditions 1 and 2. Then,

(16)

T
E [yd
HT (dk )] =

N
X

X

wi,z yir (z),

i=1 z:f (z,θi )=dk

where wi,z = pz /πi (dk ).
d
T
Under Condition 7, the estimator µd
HT (dk ) = (1/N )yHT (dk ) is unbiased
for the population mean of what Hudgens and Halloran (2008, p. 834) refer
to as the “individual average potential outcome” given Di = dk . The causal
effect estimate given in (3), which compares mean outcomes given exposures dk versus dl , is a difference in population means of individual average
randomization potential outcomes given different restrictions on the set of
treatments implied in constructing exposures dk and dl .

21

CAUSAL EFFECTS UNDER INTERFERENCE

Corollary 8.1.

Under Condition 7,

N
X
X
1

E [τd
wi,z yir (z) −
HT (dk , dl )] =
N
i=1

z:f (z,θi )=dk


X

wi,z0 yir (z0 ) .

z0 :f (z0 ,θi )=dl

Inference for such an effect would not follow immediately from the results
above. However, under partial interference, inference would follow from the
results of Liu and Hudgens (2014).
9. A naturalistic simulation with social network data. We use
a naturalistic simulation to illustrate how our framework may be applied
and also to study operating characteristics of the proposed estimators in a
realistic sample. We estimate direct and indirect effects of an experiment
with individuals linked in a complex, undirected social network. We use
friendship network data from American school classes collected through the
National Longitudinal Study of Adolescent Health (Add Health). The richness of these data makes Add Health a canonical dataset for methodological
research related to social networks, as with Bramoulle, Djebbari and Fortrin (2009), Chung, Lanza and Loken (2008), Goel and Salganik (2010),
Goodreau, Kitts and Morris (2009), Goodreau (2007), Handcock, Raftery
and Tantrum (2007), and Hunter, Goodreau and Handcock (2008). We simulate experiments in which a treatment, Z, is randomly assigned without
replacement and with uniform probability to 1/10 of individuals in a school
network. Indirect effects are transmitted only within a subject’s school. This
simulated experiment resembles various studies of network persuasion campaigns (Chen, Humphreys and Modi, 2010; Aral and Walker, 2011; Paluck,
2011), including the field experiment that we analyze below.
We define the exposure mapping as a function f (z, θi ) such that the parameter, θi , is a column vector equal to the transpose of subject i’s row in a
network adjacency matrix (modified such that we have zeroes on the diagonal). The inner product, z0 θi , counts the number of subject i’s peers assigned
to treatment. We use a simple exposure mapping that captures direct and
indirect effects of the treatment, with indirect effects being transmitted to
a subject’s immediate peers:

zi I(z0 θi > 0) = 1

 d11 (Direct + Indirect Exposure) :

d10 (Isolated Direct Exposure) :
zi I(z0 θi = 0) = 1
f (z, θi ) =
d (Indirect Exposure) :
(1 − zi )I(z0 θi > 0) = 1


 01
d00 (No Exposure) :
(1 − zi )I(z0 θi = 0) = 1
where each unit falls into exactly one of the four exposure conditions. This
exposure mapping was selected to mimic the one used in the application

22

ARONOW AND SAMII

studied in the next section. A contrast of mean outcomes under d10 versus
d00 isolates the effect of direct exposure in the absence of any interaction
with indirect exposure, whereas the d11 -d00 contrast yields an effect that
incorporates such interactive effects. The d01 -d00 contrast isolates the effect
of indirect exposure in the absence of any interaction with direct exposure.
This experiment is repeated independently across the 144 school classes
included in Add Health, with an average class size of 626 students. We
constructed the school network graphs as undirected graphs where a link
between two students was assigned if either student nominated the other as
a friend in the AddHealth survey. Students could nominate up to 5 male and
5 female friends. To ensure that our effect estimates all refer to the same
underlying population, we dropped subjects that reported zero friendship
ties. For the resulting sample, 42% of students have network degree in the
1 to 5 range, 40% in the 6 to 10 range, 18% in the 11 to 20 range, and 1%
greater than 20, with a maximum degree of 39. To give an idea of the range
of exposure probabilities, for the student with degree of 39, the probability
of isolated direct exposure was 0.00067. In the appendix, we display the
cumulative distribution functions for the four exposure probabilities. About
3% of students have an exposure probability of less than 0.01 for the direct
+ indirect exposure condition, 0.5% for isolated direct exposure, and then
there were no cases with probabilities less than 0.01 for either the indirector no-exposure conditions.
Figure 1 illustrates a treatment assignment and corresponding treatmentinduced exposures under this mapping. The figure illustrates two key issues
that our methods address. First is the connection between a unit’s underlying traits, in this case its network degree, and propensity to fall into one
or another exposure condition. The second is the irregular clustering that
occurs in exposure conditions. Such irregular clustering is precisely what
one must address in deriving variance estimates and intervals for estimated
effects.
We use as our outcome a variable in the dataset that records the number
of after-school activities in which each student participates. This variable
defines the yi (d00 ) values—that is, potential outcomes under the “control”
exposure. This makes our simulation naturalistic not only in the networks
that define the interference patterns, but also in the outcome data. The
variable exhibits a high degree of right skew, with mean 2.14, standard
deviation 2.64, and 0, .25, .5, .75, and 1 quantiles of 0, 1, 2, 3, and 33,
respectively. We consider a simple “dilated effects” scenario (Rosenbaum,
1999) where potential outcomes are such that yi (d11 ) = 2×yi (d00 ), yi (d10 ) =
1.5 × yi (d00 ), yi (d01 ) = 1.25 × yi (d00 ). We run 500 simulated replications of

CAUSAL EFFECTS UNDER INTERFERENCE
Treatment Assignment

● treated

● untreated

23

Exposure Conditions

● direct+indirect ● indirect
● isolated direct ● control

Fig 1. Illustration of a treatment assignment (left) and then treatment-induced exposures
(right) for one of the school classes in the study. Each dot is a student, and each line
represents an undirected friendship tie.

the experiment, applying five estimators in each scenario:
• The Horvitz-Thompson estimator for the causal effect given in expression (3), with the associated conservative variance estimator, given in
expression (11);
• The Hajek ratio estimator given in expression (15), with the associated
linearized variance estimator;
• The weighted least squares (WLS) estimator given in expression (14),
adjusting for network degree as the sole covariate, with the associated
linearized variance estimator;
• An ordinary least squares (OLS) estimator that regresses the outcome
on indicator variables for the exposure conditions, adjusting for network degree as a covariate, with MacKinnon and White (1985)’s finite
sample adjusted “HC2” heteroskedasticity consistent variance estimator;
• A simple difference in sample means (DSM) for the exposure conditions, also with the HC2 estimator.
With respect to point estimates, the Horvitz-Thompson estimator is unbiased but possibly unstable, while the Hajek and WLS estimators are consistent and expected to be more stable. The DSM estimator is expected to
be biased because it totally ignores relationships between exposure probabilities and outcomes. The OLS estimator controls for network degree, and

24

ARONOW AND SAMII

so this will remove bias due to correlation between exposure probabilities
and outcomes. However, OLS is known to be biased in its aggregation of
unit-level heterogeneity in causal effects (Angrist and Krueger, 1999). With
respect to standard error estimates and confidence intervals, the variance
estimators for the Horvitz-Thompson, Hajek, and WLS estimators are expected to be conservative though informative. The variance estimators for
OLS and DSM may be anti-conservative because they ignore the clustering
in exposure conditions.
Table 1 shows results of the simulation study, which conform to expectations. The Horvitz-Thompson, Hajek, and WLS estimators exhibit no perceivable bias. The Horvitz-Thompson estimator exhibits higher variability
than the Hajek and WLS estimators, although the differences are not very
pronounced, perhaps owing to the small number of cases with very small
exposure probabilities. The OLS estimator and DSM estimator are heavily
biased when considered relative to the variability of the effect estimates. The
bias in OLS is expected because unit-level causal effects, defined in terms
of differences, are heterogenous from unit to unit when underlying potential outcomes are based on dilated effects. Thus OLS will suffer from an
aggregation bias in addition to any biases due to inadequate conditioning
on network degree. The standard error estimates for the Horvitz-Thompson,
Hajek, and WLS estimators are informative but conservative, resulting in
empirical coverage rates that exceed nominal levels. The intervals for the
OLS and DSM variance estimators badly undercover, primarily due to the
bias in the point estimates rather than understatement of variability.
10. Analysis of a social network field experiment. In this section
we analyze a field experiment on the promotion of anti-conflict norms and
behavior among American middle school students. The experiment sought
to shed light on how such a program might affect attitudes and behaviors
of participant youth and also, crucially, to understand how these effects
transmit to participants’ social network peers. Full details and a richer analysis of the experiment are given in Paluck, Shepherd and Aronow (2016).
The experiment involved two levels of randomization. First, 28 of 56 schools
were randomly selected to host the anti-conflict program, via block randomization. Within all schools, a group of between 40 to 64 students were
nonrandomly selected as eligible to participate in the program. Within each
school hosting the program, half of the eligible students were then block
randomized to participate in the program, with blocking on gender, grade,
and a measure of network closure. Every two weeks over the course of the
school year, the program had participants attend meetings with program

25

CAUSAL EFFECTS UNDER INTERFERENCE
Table 1
Results from school friends’ network simulated experiment

Estimator
HT

Hajek

WLS

OLS

DSM

Estimand
τ (d01 , d00 )
τ (d10 , d00 )
τ (d11 , d00 )
τ (d01 , d00 )
τ (d10 , d00 )
τ (d11 , d00 )
τ (d01 , d00 )
τ (d10 , d00 )
τ (d11 , d00 )
τ (d01 , d00 )
τ (d10 , d00 )
τ (d11 , d00 )
τ (d01 , d00 )
τ (d10 , d00 )
τ (d11 , d00 )

Bias
0.00
0.00
0.00
0.00
0.00
0.00
0.00
0.00
0.00
-0.02
-0.08
0.12
0.42
-0.08
0.56

S.D.
0.04
0.10
0.13
0.03
0.07
0.12
0.03
0.07
0.11
0.03
0.06
0.09
0.02
0.06
0.09

RMSE
0.04
0.10
0.13
0.03
0.07
0.12
0.03
0.07
0.11
0.03
0.10
0.15
0.42
0.10
0.57

Mean
S.E.
0.05
0.19
0.28
0.03
0.13
0.25
0.03
0.12
0.25
0.02
0.07
0.09
0.02
0.07
0.09

95% CI
Coverage
0.960
0.986
0.990
0.968
0.992
0.986
0.970
0.992
0.988
0.842
0.706
0.660
0.000
0.726
0.000

90% CI
Coverage
0.924
0.970
0.970
0.916
0.970
0.970
0.928
0.968
0.950
0.768
0.576
0.530
0.000
0.614
0.000

HT = Horvitz-Thompson estimator with conservative variance estimator.
Hajek = Hajek estimator with linearized variance estimator.
WLS = Least squares weighted by exposure probabilities with covariate adjustment for network
degree and linearized variance estimator.
OLS = Ordinary least squares with covariate adjustment for network degree and
heteroskedasticity consistent variance estimator.
DSM = Simple difference in sample means with no covariate adjustment and heteroskedasticity
consistent variance estimator.
S.D. = Empirical standard deviation from simulation; RMSE = Root mean square error; S.E. =
standard error estimate; CI = Normal approximation confidence interval.

staff during which they discussed social conflicts and patterns of exclusion
at their school and formulated behavioral strategies to help friends and other
students. At the beginning of the school year, the research team measured
students’ social networks, asking students to nominate up to 10 students
in their school that they had chosen to spend time with, face to face or
online, in the last two weeks. These nominations were used to construct an
undirected adjacency graph, so that students were considered “peers” if either student nominated one another. In Figure 2, we present an illustrative
graph of one of these networks. As expected, students of the same grade
and gender are more likely to associate with one another. At the end of the
school year, the research team implemented a survey to measure behaviors
and attitudes that reflected conflict-related norms. In the current analysis,
we focus on one particular behavior: (self-reports of) wearing a wristband
issued to students through the program that was meant to reflect a student’s
public endorsement of anti-conflict norms.

26

ARONOW AND SAMII

Fig 2. Example of a proximity network for one school in the social network field experiment. Network edges were measured using student nomination data in the first survey.

Given this design, let zi = 0, 1 be an indicator for whether student i is
assigned to participate in the program and let z be the vector of student-level
assignments in the school. Let si = 0, 1 be an indicator for whether subject
i’s school hosts the program. Finally, as in the simulation study above, let
θi be a column vector equal to the transpose of student i’s row in the school
network adjacency matrix (with zeroes on the diagonal), in which case z0 θi ,
is again the number of subject i’s peers that are assigned to participate in
the program. Then, we define the exposure mapping as follows:

d111 (Direct + Indirect Exposure) :
zi I(z0 θi > 0)si = 1




zi I(z0 θi = 0)si = 1
 d101 (Isolated Direct Exposure) :
d011 (Indirect Exposure) :
(1 − zi )I(z0 θi > 0)si = 1
f (z, θi ) =


d
(School Exposure) :
(1 − zi )I(z0 θi = 0)si = 1


 001
d000 (No Exposure) :
(1 − si ) = 1
Three features emerge from examination of the exposure mapping. First,
the exposure mapping reflects four different sources of exposure to the pro-

CAUSAL EFFECTS UNDER INTERFERENCE

27

Table 2
Social network experiment results: effects of exposures on probability of wearing a
program wristband
Estimator
HT

Hajek

WLS

Estimand
τ (d001 , d000 )
τ (d011 , d000 )
τ (d101 , d000 )
τ (d111 , d000 )
τ (d001 , d000 )
τ (d011 , d000 )
τ (d101 , d000 )
τ (d111 , d000 )
τ (d001 , d000 )
τ (d011 , d000 )
τ (d101 , d000 )
τ (d111 , d000 )

Estimate
0.057
0.154
0.305
0.299
0.058
0.154
0.292
0.307
0.056
0.156
0.295
0.306

S.E.
0.062
0.029
0.141
0.020
0.064
0.037
0.123
0.049
0.066
0.037
0.124
0.049

95% CI
(-0.065, 0.179)
(0.097, 0.211)
(0.029, 0.581)
(0.260, 0.338)
(-0.067, 0.183)
(0.081, 0.227)
(0.051, 0.533)
(0.211, 0.403)
(-0.072, 0.186)
(0.083, 0.229)
(0.050, 0.536)
(0.212, 0.404)

HT = Horvitz-Thompson estimator with conservative variance estimator.
Hajek = Hajek estimator with linearized variance estimator.
WLS = Least squares weighted by exposure probabilities with covariate adjustment for network
degree and linearized variance estimator.
S.E. = Estimated standard error; CI = Normal approximation confidence interval.

gram: being in a school with the program (School), having a peer who was a
participation student (Indirect), and being a participating student (Direct).
Second, only students selected as “eligible” have a non-zero possibility of
being in all exposure conditions. Thus, we limit the present analysis to the
set of students with nonzero probabilities of exposure (N = 2, 050). (Paluck,
Shepherd and Aronow (2016) examine effects for members of the ineligible
subpopulation who nonetheless have non-zero probability of indirect exposure.) Third, the conditions for our asymptotic results hold in the number
of schools. This exposure model provides a parsimonious characterization
of first-order peer effects and school-wide climate effects, which were the
primary effects of interest for Paluck, Shepherd and Aronow (2016) when
designing the experiment. If other types of peer effects are present, analysis
under this exposure model estimates treatment-regime specific aggregates
that average over those other effects, as described in the section on misspecification above.
Table 2 presents Horvitz-Thompson (HT), Hajek, and weighted least
squares (WLS) estimates of the effects of different exposure conditions relative to the no exposure condition. The WLS estimates control for a subject’s
network degree (as in the simulation study above). The outcome of interest,
yi ∈ {0, 1} is a binary indicator for whether the subject wore a program
wristband. The effect estimates characterize, for eligible students, the aver-

28

ARONOW AND SAMII

age increase in the probability of wearing a wristband relative to the average
in the no exposure condition. (The average for eligible students in the no
exposure condition was essentially zero, at 0.000.)
The HT, Hajek, and WLS results all mostly agree. They suggest that
being in a program school but being a non-participant with no participant
peers (d001 ) has negligible effects for eligible students: our point estimate
suggests about a six percentage point increase in the probability of wearing
a wrist band, although the 95% confidence interval has a lower bound of
about -7 percentage points. However, effects for eligible students with either
indirect or direct exposure are substantially larger. The effect of indirect
exposure (d011 ) is about a 15 to 16 percentage point increase in the probability of wearing a wrist band (95% confidence interval between about 8
and 23 percentage points). The effect of direct exposure, whether or not it
is accompanied by indirect exposure (d101 or d111 ) is about a 30 percentage
point increase in the probability of wearing a wrist band (95% confidence
interval between about 5 and 50 percentage points for the d101 condition
and about 21 and 40 percentage points for the d111 condition).
Thus, the program is seen as having substantial direct but also indirect
effects on subject’s willingness to endorse anti-conflict norms by wearing a
program wristband. These indirect effects mean that one would drastically
underestimate the effect of the program if one performed a naive analysis
that simply compared participant and non-participant individuals in schools
hosting the program. Moreover, an analysis that failed to account for indirect
effects might understate the cost-effectiveness of the program: substantial
increases in school-level expressions of commitment to anti-conflict norms
would not require administering the program to everyone.
11. Conclusion. This paper proposes an analytical framework for
causal inference under interference and applies it to the analysis of experiments on social networks. As discussed in the introduction, the framework
can be applied to other settings where interference is considered to be important. The framework integrates (i) an experimental design that defines
the probability distribution for treatments assigned, (ii) an exposure mapping that relates treatments assigned to exposures received, and (iii) an
estimand chosen to make use of an experimental design to answer questions
of substantive interest. Using this framework, we develop methods for estimating average unit-level causal effects of exposures from a randomized
experiment. Our approach combines the known randomization process with
the analyst’s definition of treatment exposure, thus permitting inference under clear and defensible assumptions. Importantly, the union of the design of

CAUSAL EFFECTS UNDER INTERFERENCE

29

the experiment and the exposure mapping may imply unequal probabilities
of exposure and forms of dependence between units that may not be obvious
ex ante.
We develop estimators based on results from the literature on unequal
probability sampling rooted in the foundational insights of Horvitz and
Thompson (1952). The estimators are derived from the known sampling
distribution of the “direct” treatment, and they provide a basis for unbiased
effect estimation and conservative variance estimation. Wald-type intervals
based on a normal approximation provide a reasonable reflection of large N
behavior when clustering of exposure indicator values is limited. Nonetheless, it is well known that Horvitz-Thompson-type estimators may be volatile
in cases where selection probabilities vary greatly or exhibit strong inverse
correlation with outcome values (Basu, 1971). Thus, we provide refinements
that allow for variance control via covariance adjustment and Hajek estimation.
Our approach combines minimal assumptions about restrictions on potential outcomes with randomization-based estimators, and may be characterized as design-consistent. The estimands and methods presented here
may be useful in evaluating alternative experimental designs for estimating causal effects in the presence of interference (Airoldi, 2016; Baird et al.,
2016; Eckles, Karrer and Ugander, 2014; Toulis and Kao, 2013; Ugander
et al., 2013). Finally, the framework is readily applicable to deriving estimators for estimands other than the average unit-level effect of exposures.
APPENDIX A: PROOFS
A.1. Proof of Proposition 3.1. The replication procedure is equivalent to drawing a random sample without replacement from Ω, with probabilities of selection equal to those which are defined in the randomization
plan. The result follows from the Strong Law of Large Numbers.
A.2. Proof of Lemma 4.1. To show unbiasedness, by 2 we have
T
E [yd
HT (dk )] = E

N
hX

I(Di = dk )

i=1

=

N
X
i=1

Yi i
πi (dk )
N

E [I(Di = dk )]

yi (dk ) X
=
yi (dk ).
πi (dk )
i=1

T
The variance expression follows from the fact that yd
HT (dk ) is a sum of
correlated random variables.

30

ARONOW AND SAMII

A.3. Proof of Proposition 4.1. We have,
T
E [y\
HT,R (dk )] =

=

N
X
i=1
N
X



R+1
yi (dk )πi (dk )E
Xi + 1





yi (dk ) 1 − (1 − πi (dk ))R+1

i=1

= y T (dk ) −

N
X

yi (dk )(1 − πi (dk ))R+1 .

i=1

And so,
T
T
E [y\
HT,R (dk )] − y (dk ) =

N
X

yi (dk )(1 − πi (dk ))R+1

i=1
T

≤ |y (dk )|(1 − π0 (dk ))R+1 .
A.4. Proof of Proposition 4.2. Results (4) and (5) follow from
Lemma 4.1 and properties of the variance operator. For the covariance term
(6), first note that πii (dk , dl ) = 0. Then following Wood (2008),
d
T
T
Cov [yd
HT (dk ), yHT (dl )] =

N X
N
X

Cov [I(Di = dk ), I(Dj = dl )]

i=1 j=1

yi (dk ) yj (dl )
πi (dk ) πj (dl )

N X
N
X
yi (dk ) yj (dl )
=
[πij (dk , dl ) − πi (dk )πj (dl )] .
πi (dk ) πj (dl )
i=1 j=1

A.5. Key results for Propositions 5.1, 5.2, 5.3, and 5.4. We
reproduce key results from Aronow and Samii (2013) for the conservative variance corrections. We do so in the general case of the HorvitzThompson estimator for a population total. Suppose a population U indexed
by 1, ..., k, ..., N and a sampling design such that the probability of inclusion
in the sample for unit k is given by πk , and the joint inclusion probability
for units k and l is given by πkl .
The Horvitz-Thompson estimator of a population total is given by
X yk
X yk
=
Ik ,
t̂ =
πk
πk
k∈s

k∈U

where Ik ∈ {0, 1} is unit k’s inclusion indicator, the only stochastic component of the expression, with E(Ik ) = πk , the inclusion probability, and s and

31

CAUSAL EFFECTS UNDER INTERFERENCE

U refer to the sample and the population, respectively. Define E(Ik Il ) = πkl ,
the probability that both units k and l from U are included in the sample.
Since Ik Ik = Ik , E(Ik Ik ) = πkk = πk by construction. The variance of the
Horvitz-Thompson estimator for the total is given by
XX
yk yl
Var(t̂) =
Cov(Ik , Il )
πk πl
k∈U l∈U
 2 X X
X
yk yl
yk
Cov(Ik , Il )
+
=
Var(Ik )
.
πk
πk πl
k∈U l∈U \k

k∈U

Under a measurable design, two conditions obtain: (1) πk > 0 and πk is
known for all k ∈ U and (2) πkl > 0 and πkl is known for all k, l ∈ U .
Non-measurable designs include those for which either of the two conditions
for a measurable design do not hold. We label a sample from a measurable
design, sM , and an unbiased estimator for Var(t̂) on sM is given by,
d t̂) =
Var(

X X Cov(Ik , Il ) yk yl
XX
Cov(Ik , Il ) yk yl
=
Ik Il
,
πkl
πk πl
πkl
πk πl
M
M

k∈s

k∈U l∈U

l∈s

where the only stochastic part of the latter expression is Ik Il , and unbiasedness is due to E(Ik Il ) = πkl .
Suppose a non-measurable design for which πkl = 0 for some units k, l ∈
U . We label a sample from such a non-measurable design as s0 . Because Ik
is a Bernoulli random variable with probability πk , Cov(Ik , Il ) = πkl − πk πl
for k 6= l, and Cov(Ik , Ik ) = Var(Ik ) = πk (1 − πk ). Then, we can re-express
the variance above as,
Var(t̂) =

X


πk (1 − πk )

k∈U

=

X
k∈U


πk (1 − πk )

yk
πk

2

yk
πk

2

+

X X

(πkl − πk πl )

k∈U l∈U \k

+

X

X

k∈U l∈{U \k:πkl >0}

yk yl
πk πl

(πkl − πk πl )

X
yk yl X
−
yk yl .
πk πl
k∈U l∈{U \k:πkl =0}
|
{z
}
A

For k and l such that πkl = 0, the sampling design will never provide information on the component of the variance labeled as A above, since we will
never observe yk and yl together.
d t̂) is applied to s0 , the result is unbiased for Var(t̂) + A. We
When Var(
state this formally as follows:

32

ARONOW AND SAMII

Proposition A.1 (Aronow and Samii, 2013, Prop. 1). When s0 refers
to a sample from a design with some πkl = 0, we have,
h
i
X
X
d t̂) = Var(t̂) +
yk yl = Var(t̂) + A.
E Var(
k∈U l∈{U \k:πkl =0}

Proof. The result follows from,



X X Cov(Ik , Il ) yk yl
X
 = E
E
πkl
πk πl
0
0


Cov(Ik , Il ) yk yl 
Ik Il
πkl
πk πl
k∈U l∈{U :πkl >0}
 2 X
X
X
yk yl
yk
+
Cov(Ik , Il )
=
Var(Ik )
πk
πk πl
k∈U l∈{U \k:πkl >0}
k∈U
X
X
= Var(t̂) +
yk yl

k∈s l∈s

X

k∈U l∈{U \k:πkl =0}

= Var(t̂) + A.

Now, consider the following variance estimator:
[C (t̂) =
Var

X

X

k∈U l∈{U :πkl >0}

Cov(Ik , Il ) yk yl X
Ik Il
+
πkl
πk πl

X

k∈U l∈{U \k:πkl =0}

where akl , bkl are positive real numbers such that
k, l with πkl = 0.

1
akl

+

1
bkl



|yk |akl
|yl |bkl
Ik
+ Il
,
akl πk
bkl πl

= 1 for all pairs

Proposition A.2 (Aronow and Samii, 2013, Prop. 2).
h
i
[C (t̂) ≥ Var(t̂).
E Var
Proof. By Young’s inequality,
|yk |akl
|yl |bkl
+
≥ |yk ||yl |,
akl
bkl
if

1
akl

A∗ =

+

1
bkl

X

= 1. Define A∗ such that,
X

k∈U l∈{U \k:πkl =0}

X
|yk |akl |yl |bkl
+
≥
akl
bkl

X

k∈U l∈{U \k:πkl =0}

|yk ||yl | ≥

X

X

k∈U l∈{U \k:πkl =0}

yk yl = A

CAUSAL EFFECTS UNDER INTERFERENCE

33

and
A∗ ≥

X

X

|yk ||yl | ≥

k∈U l∈{U \k:πkl =0}

X

X

−yk yl = −A.

k∈U l∈{U \k:πkl =0}

Therefore
Var(t̂) + A + A∗ ≥ Var(t̂).
The associated Horvitz-Thompson estimator of A∗ would be


X
X
|yk |akl
|yl |bkl
∗
c
Ik
A =
+ Il
,
akl πk
bkl πl
k∈U l∈{U \k:πkl =0}

which is unbiased
h i by E(Ik ) = πk and E(Il ) = πl .
c∗ = A∗ , by Proposition A.1,
Since E A



X X Cov(Ik , Il ) yk yl
c∗  = Var(t̂) + A + A∗
E
+A
π
π
π
kl
k
l
0
0
k∈s l∈s

h
i
[C (t̂) ≥ Var(t̂).
E Var
Substituting terms,

E

X

X

k∈U l∈{U :πkl >0}

Ik Il

Cov(Ik , Il ) yk yl
+
πkl
πk πl

X

X

k∈U l∈{U \k:πkl =0}




b
a
kl
kl
|yl |
|yk |
 ≥ Var(t̂).
+ Il
Ik
akl πk
bkl πl


This estimator is unbiased under a special condition:
Corollary A.1 (Aronow and Samii, 2013, Cor. 1). If, for all pairs k, l
such that πkl = 0, (i) |yk |akl = |yl |bkl and (ii) −yk yl = |yk ||yl |,
h
i
[C (t̂) = Var(t̂).
E Var
Proof. By (i), (ii) and Young’s inequality,
|yk |akl
|yl |bkl
+
= |yk ||yl | = −yk yl .
akl
bkl

34

ARONOW AND SAMII

Therefore,
A∗ =

X

X

k∈U l∈{U \k:πkl =0}

=

X

X

|yk |akl
|yl |bkl
+
akl
bkl
|yk ||yl |

k∈U l∈{U \k:πkl =0}

=

X

X

−yk yl = −A

k∈U l∈{U \k:πkl =0}

It follows that
Var(t̂) + A + A∗ = Var(t̂)
and

h
i
[C (t̂) = Var(t̂).
E Var

In general, it would be difficult to assign optimal values of akl and bkl
for all pairs k, l such that πkl = 0. Instead, we examine one intuitive case,
assigning all akl = bkl = 2:


X
X
X
yk2
yl2
Cov(Ik , Il ) yk yl X
\
VarC2 (t̂) =
Ik Il
+
Ik
+ Il
.
πkl
πk πl
2πk
2πl
k∈U l∈{U :πkl >0}

k∈U l∈{U \k:πkl =0}

[C (t̂), Var
\
As a special case of Var
C2 (t̂) is also conservative:
Corollary A.2 (Aronow and Samii, 2013, Cor. 2).
h
i
\
E Var
C2 (t̂) ≥ Var(t̂).
Proof. For all pairs k, l such that πkl = 0,
Proposition A.2 therefore holds.

1
akl

+

1
bkl

=

1
2

+

1
2

= 1.

A.6. Proof of Proposition 6.1. We follow the logic of Robinson
(1982). µd
HT (dk ) is unbiased for µ(dk ), and thus we need only consider the
variance. Condition 3 implies that for all values i and dk , |yi (dk )|/πi (dk ) ≤
2
c3 P
< ∞.
Substituting from Equation (2), N 2 Var (µd
HT (dk )) ≤ c3 N +
P
N
N
c23 i=1 j=1 gij . Consistency of µd
HT (dk ) for µ(dk ) is therefore ensured
PN PN
p
2
when i=1 j=1 gij = o(N ), as this implies that µd
HT (dk )−µHT (dk ) −→ 0.
Consistency of τd
HT (dk , dl ) for τ (dk , dl ) follows by Slutsky’s Theorem.

CAUSAL EFFECTS UNDER INTERFERENCE

35

A.7. Proof of Proposition 6.2. We follow a proof technique similar to that of Aronow, Samii and Assenova (2015) to establish converd [τd
gence of N Var
HT (dk , dl )], though for a considerably more general setd [τd
ting. By Proposition 5.6, E [N Var
HT (dk , dl )]] ≥ N Var [τd
HT (dk , dl )]. Thus,
p
d
by Chebyshev’s Inequality, Var [N Var [τd
HT (dk , dl )]] → 0 is sufficient to
d
establish convergence of N Var [τd
HT (dk , dl )] to a value greater than or
equal to N Var [τd
(d
,
d
)],
which
is itself nonzero by Condition 6. Denote
HT k l
d [τd
aij (Di , Dj ) as the sum of the elements in Var
HT (dk , dl )] that incorporate
observations i and j. Note that all aij (Di , Dj ) are bounded above by some
finite constant by Condition 3.


N X
N
X
−2
d [τd
Var [N Var
Var 
hij aij (Di , Dj )
HT (dk , dl )]] ≤ N
i=1 j=1

= N −2

N X
N X
N X
N
X

Cov [hij aij (Di , Dj ), hkl akl (Dk , Dl )].

i=1 j=1 k=1 l=1

Note that Cov [hij aij (Di , Dj ), hkl akl (Dk , Dl )] 6= 0 if and only if hij = 1 and
hkl = 1 and either: hik = 1, hil = 1, hjk = 1, or hjl = 1. By Condition 5,
given m  N , each of these four conditions is satisfied by fewer than N m3
of the elements of the quadruple summation, and the number of elements
−2 ×
d [τd
in their union is at most 4N m3 . Thus, Var [N Var
HT (dk , dl )]] = O(N
−1
N ) = O(N ).
Define
τd
τd
HT (dk , dl ) − τHT (dk , dl )
HT (dk , dl ) − τHT (dk , dl )
q
p
=
t=
Var [τd
HT (dk , dl )]
d [τd
Var
(d
,
d
)]
HT k l

Var [τd
HT (dk , dl )]
d [τd
Var
HT (dk , dl )]

Under Conditions 3, 5, and 6, then, by Chen and Shao (2004,
HT
√(dk ,dl )−τHT (dk ,dl ) is asymptotically N (0, 1), while
Theorem 2.7), τd
Var [τd
HT (dk ,dl )]

d HT (dk , dl )])1/2 converges in probability to a quantity
(Var [τd
HT (dk , dl )]/Var [τd
in (0, 1]. By Slutsky’s Theorem, t is asymptotically normal
q and Wald-type
d HT (dk , dl )]
confidence intervals constructed as τd
HT (dk , dl ) ± z1−α/2 Var [τd
will tend to cover τHT (dk , dl ) at least 100(1 − α)% of the time as N → ∞.

!1/2
.

36

ARONOW AND SAMII

A.8. Proof of Proposition 8.1. The result follows from iterating expectations:
#
"N
X
Y
i
\
T
E [yHT
(dk )] = E
I(Di = dk )
πi (dk )
i=1


N
X
I(Di = dk )
=
E [Yi |Di = dk ]
E
πi (dk )
i=1

=

=

=

N
X
i=1
N
X

E [Yi |Di = dk ]
P

z:f (z,θi )=dk

P
i=1
N
X

pz yir (z)

z:f (z,θi )=dk

X

i=1 z:f (z,θi )=dk

pz

pz
y r (z).
πi (dk ) i

37

CAUSAL EFFECTS UNDER INTERFERENCE

APPENDIX B: SIMULATION STUDY EXPOSURE PROBABILITIES
d1 0
1.0
0.8
0.6

F(π(d1 0))

0.2
0.0
0.4

0.6

0.8

1.0

0.0

0.2

0.4

π(d1 1)

π(d1 0)

d0 1

d0 0

0.6

0.8

1.0

0.6

0.8

1.0

0.8
0.6

F(π(d0 0))

0.0

0.2

0.4
0.0

0.2

0.4

0.6

0.8

1.0

0.2

1.0

0.0

F(π(d0 1))

0.4

0.6
0.4
0.0

0.2

F(π(d1 1))

0.8

1.0

d1 1

0.0

0.2

0.4

0.6
π(d0 1)

0.8

1.0

0.0

0.2

0.4
π(d0 0)

Fig 3. Empirical CDFs of probabilities for the four types of exposure in the simulated
social network experiment.

38

ARONOW AND SAMII

REFERENCES
Airoldi, E. M. (2016). Estimating Causal Effects in the Presence of Interfering Units.
Paper Presented at YINS Distinguished Lecture Series, Yale University.
Angrist, J. D. and Krueger, A. B. (1999). Empirical Strategies in Labor Economics. In
Handbook of Labor Economics, (O. C. Ahsenfelter and D. Card, eds.) 3 North Holland,
Amsterdam.
Aral, S. and Walker, D. (2011). Creating Social Contagion Through Viral Product
Design: A Randomized Trial of Peer Influence in Networks. Management Science 57
1623-1639.
Aronow, P. M. (2013). Model Assisted Causal Inference PhD thesis, Department of
Political Science, Yale University, New Haven, CT.
Aronow, P. M. and Middleton, J. A. (2013). A Class of Unbiased Estimators of the
Average Treatment Effect in Randomized Experiments. Journal of Causal Inference 1
135-144. Manuscript, Yale University and New York University.
Aronow, P. M. and Samii, C. (2013). Conservative Variance Estimation for Sampling
Designs with Zero Pairwise Inclusion Probabilities. Survey Methodology 39 231-241.
Aronow, P. M., Samii, C. and Assenova, V. A. (2015). Cluster Robust Variance Estimation for Dyadic Data. Political Analysis 23 546-577.
Baird, S., Bohren, J. A., McIntosh, C. and Ozler, B. (2016). Designing Experiments
to Measure Spillover Effects. Unpublished manuscript, George Washington University,
University of Pennsylvania, University of California-San Diego, and World Bank.
Basu, D. (1971). An Essay on the Logical Foundations of Survey Sampling, Part I. In
Foundations of Statistical Inference (V. Godambe and D. A. Sprott, eds.) 203–242.
Holt, Rinehart, and Winston, Toronto.
Bowers, J., Fredrickson, M. M. and Panagopolous, C. (2013). Reasoning about
Interference Between Units: A General Framework. Political Analysis 21 97-124.
Bramoulle, Y., Djebbari, H. and Fortrin, B. (2009). Identification of Peer Effects
through Social Networks. Journal of Econometrics 150 41-55.
Brewer, K. (1979). A Class of Robust Sampling Designs for Large-Scale Surveys. Journal
of the American Statistical Association 74 911-915.
Chen, J., Humphreys, M. and Modi, V. (2010). Technology Diffusion and Social Networks: Evidence from a Field Experiment in Uganda. Manuscript, Columbia University.
Chen, L. H. and Shao, Q. (2004). Normal Approximation Under Local Dependence. The
Annals of Probability 32 1985-2028.
Chung, H., Lanza, S. T. and Loken, E. (2008). Latent Transition Analysis: Inference
and Estimation. Statistics in Medicine 27 1834-1854.
Cole, S. R. and Frangakis, C. E. (2009). The Consistency Statement in Causal Inference: A Definition or an Assumption? Epidemiology 20 3-5.
Cox, D. R. (1958). Planning of Experiments. Wiley.
Eckles, D., Karrer, B. and Ugander, J. (2014). Design and Analysis of Experiments
in Networks: Reducing Bias from Interference. arXiv:1404.7530 [stat.ME].
Fattorini, L. (2006). Applying the Horvitz-Thompson Criterion in Complex Designs: A
Computer-Intensive Perspective for Estimating Inclusion Probabilities. Biometrika 93
269-278.
Freedman, D., Pisani, R. and Purves, R. (1998). Statistics, 3rd Ed. W.W. Norton,
New York.
Fuller, W. A. (2009). Sampling Statistics. Wiley, Hoboken.
Goel, S. and Salganik, M. J. (2010). Assessing Respondent-Driven Sampling. Proceedings of the National Academy of Science 107 6743-6747.

CAUSAL EFFECTS UNDER INTERFERENCE

39

Goodreau, S. M. (2007). Advances in Exponential Random Graph (p-star) Models Applied to a Large Social Network. Social Networks 29 231-248.
Goodreau, S. M., Kitts, J. A. and Morris, M. (2009). Birds of a Feather, or Friend
of a Friend? Using Exponential Random Graph Models to Investigate Adolescents in
Social Networks. Demography 46 103-125.
Hájek, J. (1971). Comment on “An Essay on the Logical Foundations of Survey Sampling,
Part One,”. In Foundations of Statistical Inference (V. P. Godambe and D. A. Sprott,
eds.) Holt, Rinehart and Winston, Toronto.
Handcock, M. S., Raftery, A. E. and Tantrum, J. M. (2007). Model-based Clustering
for Social Networks. Journal of the Royal Statistical Society, Series A 170 301-354.
Hartley, H. O. and Ross, A. (1954). Unbiased Ratio Estimators. Nature 174 270–271.
Holland, P. W. (1986). Statistics and Causal Inference. Journal of the American Statistical Association 81 945-960.
Horvitz, D. G. and Thompson, D. J. (1952). A Generalization of Sampling Without
Replacement from a Finite Universe. Journal of the American Statistical Association
47 663-685.
Hudgens, M. G. and Halloran, M. E. (2008). Toward Causal Inference with Interference. Journal of the American Statistical Association 103 832-842.
Hunter, D. R., Goodreau, S. M. and Handcock, M. S. (2008). Goodness of Fit of
Social Network Models. Journal of the American Statistical Association 103 248-258.
Imbens, G. W. (2000). The Role of the Propensity Score in Estimating Dose-Response
Functions. Biometrika 87 706-710.
Imbens, G. W. and Rubin, D. B. (2015). Causal Inference for Statistics, Social and
Biomedical Sciences: An Introduction. Cambridge University Press, Cambridge.
Imbens, G. W. and Wooldridge, J. M. (2009). Recent Developments in the Econometrics of Program Evaluation. Journal of Economic Literature 47 5-86.
Isaki, C. T. and Fuller, W. A. (1982). Survey Design Under the Regression Superpopulation Model. Journal of the American Statistical Association 77 89-96.
Lin, W. (2013). Agnostic Notes on Regression Adjustments to Experimental Data: Reexamining Freedman’s Critique. Annals of Applied Statistics 7 295-318.
Liu, L. and Hudgens, M. G. (2014). Large Sample Randomization Inference of Causal
Effects in the Presence of Interference. Journal of the American Statistical Association
109 288-301.
MacKinnon, J. G. and White, H. (1985). Some Heteroskedasticity-Consistent Covariance Matrix Estimators with Improved Finite Sample Properties. Journal of Econometrics 29 305-325.
Manski, C. F. (1995). Identification Problems in the Social Sciences. Harvard University
Press, Cambridge, MA.
Manski, C. F. (2013). Identification of Treatment Response with Social Interactions. The
Econometrics Journal 16 S1-S23.
Middleton, J. A. and Aronow, P. M. (2011). Unbiased Estimation of the Average
Treatment Effect in Cluster-Randomized Experiments. Paper Presented at the Annual
Meeting of the Midwest Political Science Association, Chicago.
Neyman, J. S. (1990 [1923]). On the Application of Probability Theory to Agricultural
Experiments, Essay on Principles, Section 9 (reprint). Statistical Science 5 465-472.
Ogburn, E. L. and VanderWeele, T. J. (2014). Causal Diagrams for Interference.
Statistical Science 29 559-578.
Paluck, E. L. (2011). Peer Pressure Against Prejudice: A High School Field Experiment
Examining Social Network Change. Journal of Experimental Psychology 47 350-358.
Paluck, E. L., Shepherd, H. and Aronow, P. M. (2016). Changing Climates of Con-

40

ARONOW AND SAMII

flict: A Social Network Experiment in 56 Schools. Proceedings of the National Academy
of Science In press doi: 10.1073/pnas.1514483113.
Raj, D. (1965). On a Method of Using Multi-Auxiliary Information in Sample Surveys.
Journal of the American Statistical Association 60 270-277.
Robinson, P. M. (1982). On the Convergence of the Horvitz-Thompson Estimator. Australian Journal of Statistics 24 234-238.
Rosenbaum, P. R. (1999). Reduced Sensitivity to Hidden Bias at Upper Quantiles in
Observational Studies with Dilated Treatment Effects. Biometrics 55 560-564.
Rosenbaum, P. R. (2007). Interference Between Units in Randomized Experiments. Journal of the American Statistical Association 102 191-200.
Rubin, D. B. (1978). Bayesian Inference for Causal Effects: The Role of Randomization.
The Annals of Statistics 6 34-58.
Rubin, D. B. (1990). Formal Models of Statistical Inference for Causal Effects. Journal
of Statistical Planning and Inference 25 279-292.
Särndal, C.-E., Swensson, B. and Wretman, J. (1992). Model Assisted Survey Sampling. Springer, New York.
Sobel, M. E. (2006). What do Randomized Studies of Housing Mobility Demonstrate?:
Causal Inference in the Face of Interference. Journal of the American Statistical Association 101 1398-1407.
Tchetgen-Tchetgen, E. J. and VanderWeele, T. J. (2012). On Causal Inference in
the Presence of Interference. Statistical Methods in Medical Research 21 55-75.
Toulis, P. and Kao, E. (2013). Estimation of Causal Peer Influence Effects. Journal of
Machine Learning Research 28 1489–1497.
Ugander, J., Karrer, B., Backstrom, L. and Kleinberg, J. (2013). Graph Cluster
Randomization: Network Exposure to Multiple Universes. arXiv:1305.6979 [cs.SI].
VanderWeele, T. J. (2009). Concerning the Consistency Assumption in Causal Inference. Epidemiology 20 880-883.
Wood, J. (2008). On the Covariance Between Related Horvitz-Thompson Estimators.
Journal of Official Statistics 24 53-78.
Peter M. Aronow
Departments of Political Science and Biostatistics
Yale University
77 Prospect Street
New Haven, CT 06520
United States
E-mail: peter.aronow@yale.edu

Cyrus Samii
Politics Department
New York University
19 West 4th Street
New York, NY 10012
United States
E-mail: cds2083@nyu.edu

