Some asymptotic results for fiducial and confidence distributions
Piero Veronese 1 and Eugenio Melilli
Bocconi University, via Röntgen, 1, 20136, Milan, Italy

arXiv:1612.04288v2 [] 17 Oct 2017

piero.veronese@unibocconi.it, eugenio.melilli@unibocconi.it

Abstract. Under standard regularity assumptions, we provide simple approximations for specific classes of
fiducial and confidence distributions and discuss their connections with objective Bayesian posteriors. For a real
parameter the approximations are accurate at least to order O(n−1 ). For the mean parameter µ = (µ1 , . . . , µk )
of an exponential family, our fiducial distribution is asymptotically normal and invariant to the importance
ordering of the µi ’s.
Keywords: ancillary statistic, confidence curve, coverage probability, natural exponential family, matching
prior, reference prior.

1

Introduction

Confidence and fiducial distributions, often confused in the past, have recently received a renewed attention
by statisticians thanks to several contributions which clarify the concepts within a purely frequentist setting
and overcome the lack of rigor and completeness typical of the original formulations. For a wide and comprehensive presentation of the theory of confidence distributions and a rich bibliography we refer the reader
to the book by Schweder & Hjort (2016) and to the review paper by Xie & Singh (2013). This latter also
highlights the importance of this theory in meta-analysis, see also Liu et al. (2015). For what concerns fiducial
distributions Hannig and his coauthors, starting from the original idea of Fisher, have developed in several
papers a generalized fiducial inference which is suitable for a large range of situations; see Hannig et al. (2016)
for a complete review on the topic and updated references.
Given a random vector S (representing the observations or a sufficient statistic) with distribution indexed
by η = (θ, λ), where θ is the real parameter of interest, a confidence distribution (CD) for θ is a function C
of S and θ such that: i) C(s, ·) is a distribution function on R for any fixed realization s of S and ii) C(S, θ)
has a uniform distribution on (0, 1), whatever the true value of η. The second condition is crucial because it
implies that the coverage of the intervals derived from C is exact. If it is satisfied only for the sample size
tending to infinity, C is an asymptotic CD and the coverage is correct only approximately. Given a CD, it is
1 Corresponding

author

1

possible to define the confidence curve ccs (θ) = |1 − 2C(s, θ)|, which displays the confidence intervals induced
by C for all levels, see Schweder & Hjort (2016, Sec. 1.6).
A fiducial distribution (FD) for a parameter θ has been obtained by several authors starting from a datagenerating equation S = G(U, θ), with U a random vector with known distribution, which allows to transfer
randomness from S to θ. In particular Hannig (2009, 2016) derives an explicit expression for the density of a
FD which coincides with that originally proposed by Fisher (1930), namely hs (θ) = |∂Fθ (s)/∂θ|, when both θ
and S are real and G(U, θ) = Fθ−1 (U ), with Fθ distribution function of S and U uniform in (0, 1).
In this paper we consider the specific definition of FD given in Veronese & Melilli (2016), recalled in Section
3, which for a real parameter and a continuous S again simplifies to the Fisher’s formula. In particular, we
assume that the FD function is

Hs (θ) = 1 − Fθ (s) = Prθ (S > s)

(1)

with Fθ (s) decreasing and differentiable in θ and with limits 0 and 1 when θ tends to the boundaries of
its parameter space. This conditions are always true, for example, if Fθ belongs to a regular real natural
exponential family (NEF). This FD is also a CD (asymptotically in the discrete case). For the multi-parameter
case a peculiar aspect of our FD is its dependence on the inferential importance ordering of the parameters,
similarly to what happens for the objective Bayesian posterior obtained from a reference prior. The connections
between our definition and Hannig’s setup are discussed in Veronese & Melilli (2016).
In Section 2.1, extending a result proved in Veronese & Melilli (2015) for a NEF, we give a second order
asymptotic expansion of our FD/CD in the real parameter case based only on the maximum likelihood estimator (MLE). This expansion does not require any other regularity conditions than the standard ones usually
assumed in maximum likelihood asymptotic theory. Furthermore, we show that it coincides with the expansion of the Bayesian posterior induced by the Jeffrey prior. This fact establishes a connection with objective
Bayesian inference, whose aim is to produce posterior distributions free of any subjective prior information. In
Section 2.2, starting from the well known p∗ -formula of Barndorff-Nielsen (1980, 1983), we propose and discuss
a FD/CD which, using an ancillary statistic in addition to the MLE, has good asymptotic behavior. Higher
order asymptotics for generalized fiducial distributions have been discussed, at our knowledge, only in the
unpublished paper Pal Majumder & Hannig (2016). However, its focus is different being devoted to identify
data generating equation with desirable properties. In Section 3 we consider a NEF with a multidimensional
parameter and show that, without any further regularity conditions, the asymptotic FD of the mean parameter
is normal, it does no longer depend on the inferential ordering of the parameters and coincides with the cor-

2

responding asymptotic Bayesian posterior. Some examples illustrate the good properties and performances of
the various proposed FD/CD with emphases on coverage and expected length of confidence intervals. Finally,
the Appendix includes the proofs of all the theorems and propositions stated in the paper.

2

Asymptotics for fiducial and confidence distributions: the real
parameter case

2.1

An expansion with error of order O(n−1 )

In Veronese & Melilli (2015) an Edgeworth expansion with an error of order O(n−1 ) of the FD/CD for the
mean parameter of a real NEF was derived. Here we generalize this result to an arbitrary regular model.
Let X = (X1 , . . . , Xn ) be an i.i.d. sample of size n from a density (with respect to the Lebesgue measure)
parameterized by θ belonging to an open set Θ ⊆ R. Let θ̂ be the MLE of θ based on X and denote by
pθ (θ̂) its density. Let ℓ(θ) = n−1 log pθ (θ̂) and let ℓ′′ (θ̂) and ℓ′′′ (θ̂) be the second and the third derivative of
ℓ(θ) with respect to θ, evaluated in θ̂. Then the expected and observed Fisher information of θ̂ (per unit)
are I(θ) = −n−1 Eθ (∂ 2 log pθ (θ̂)/∂θ2 ) and −ℓ′′ (θ̂), respectively. Let b = b(θ̂) = −1/ℓ′′ (θ̂). Consider now
p
Z = n/b (θ − θ̂), which is an approximate standardized version of θ in the FD/CD-setup, and let Hn,θ̂ (z)

be its FD/CD derived from the sampling distribution of θ̂. If θ̂ is sufficient, Hn,θ̂ (z) is exact, otherwise

it is a natural approximation of the exact one, see e.g. Schweder & Hjort (2016). To prove our result we
resort to the expansion of the frequentist probability Prθ (Z ≤ z) provided in Datta & Ghosh (1995) or in
Mukerjee & Ghosh (1997). Thus we need the regularity assumptions used in these papers, see also Ghosh
(1994, Ch. 8) and Bickel & Ghosh (1990) for a precise statement. Notice that the conditions required for the
frequentist expansion of the distribution of the MLE are rarely reported in a rigorous way in books and papers.
However, what is important here is that, in order to prove our result, we do not need any further assumption
and this fact allows an immediate and fair comparison between MLE- and FD/CD-asymptotic theory.
Theorem 1 Let X be an i.i.d. sample of size n from a density pθ , θ ∈ Θ ⊆ R. Then, under the regularity
p
assumptions cited above, the distribution function Hn,θ̂ (z) of the FD/CD for Z = n/b(θ−θ̂) has the expansion

1 3/2 ′′′
2
Hn,θ̂ (z) = Φ(z) − φ(z) b ℓ (θ̂)(z − 1) n−1/2 + O(n−1 ).
6


(2)

If pθ also satisfies the conditions for the expansion of a Bayesian posterior, see e.g. Johnson (1970, Theorem
2.1, with K = 1), we have the following

3

Corollary 1 In the same setting of Theorem 1, let π J (θ) ∝ I(θ)1/2 be the Jeffreys prior for θ. If π J is
improper, assume that there exists an n0 ≥ 1 such that the posterior distribution π J (θ|z) of θ is proper for
n ≥ n0 , almost surely for all θ. Then the expansion of π J (θ|z) coincides with that of Hn,θ̂ (z) given in (2).
Theorem 1 and Corollary 1 confirm the idea that the Jeffreys posterior is really free of any subjective prior
information. Furthermore, they naturally establish a connection between FD/CD-theory and matching priors,
i.e. priors that ensure approximate frequentist validity of posterior credible sets. More precisely, a prior π for
which Prθ (θ ≤ q1−α (X, π)) = 1 − α + o(n−r/2 ), where q1−α (X, π) denotes the (1 − α)th posterior quantile of
θ, is called a matching prior of order r, see Datta & Mukerjee (2004) for a general review and references. For
a regular model indexed by a real parameter it is well known, see Datta & Mukerjee (2004, Theorem 2.5.1),
that the Jeffreys prior π J is the unique first order matching prior and is also a second order matching prior if
and only if the model satisfies the following condition:
I(θ)−3/2 Eθ [(∂ℓ(θ)/∂θ)3 ]

is a constant free of θ.

(3)

Veronese & Melilli (2015) study the existence of a prior (named fiducial prior ) which induces a Bayesian
posterior coinciding with the FD, extending a result given by Lindley (1958) for a continuous univariate
sufficient statistic, see also Taraldsen & Lindqvist (2015) for a generalization to multivariate group models.
Because a FD/CD realizes the exact matching, we immediately have the following
Corollary 2 If a fiducial prior π F exists, then it coincides with the Jeffreys prior π J . Furthermore, the
condition (3) is necessary for the existence of π F .
Notice that for a model belonging to a NEF, with mean parameter µ and variance function V (µ), condition
(3) becomes: “2V ′ (µ)V (µ)−1/2 is constant”. The solution of this differential equation is V (µ) = (c1 µ + c2 )2 ,
i.e. a fiducial prior for a parameter of a NEF may exist only if its variance function is quadratic. This result
was found for the first time in Veronese & Melilli (2015), using a totally different approach.
Example 1 (Exponential distribution). Let (X1 , . . . , Xn ) be an i.i.d. sample from an exponential distribution with mean µ. The MLE µ̂ of µ is the sample mean and b = b(µ̂) = µ̂2 . Then the expansions of the
√
√
FD/CD for Z = ( n/µ̂)(µ − µ̂) in (2) and that of the standardized MLE W = −Z = ( n/µ̂)(µ̂ − µ), see
(A.3), are respectively


Φ(z) − φ(z) 2(z 2 − 1)/3 n−1/2 + O(n−1 ),


Φ(w) − φ(w) −(2w2 + 1)/3 n−1/2 + O(n−1 ).
4

(4)
(5)

10
8
6
2

4

Expected length

12

0.92
0.90

Coverage

0.88

0

0.86

0

5

10

15

0

µ

5

10

15

µ

Figure 1: Coverages and expected lengths for the 90% intervals with n = 15 based on: exact
FD/CD (red), Normal approximation (green), expansions of FD/CD (black) and of MLE (blue).
It follows that the confidence intervals obtained from (4) and (5) are different, contrary to what happens for
those based only on the normal approximation. Their coverages and expected lengths are reported in Figure 1
for a sample of size n = 15 and confidence level 0.9. Notice that the coverage of the FD/CD-intervals is much
closer to the nominal level than that of the intervals based on the MLE, while the expected lengths are quite
similar. For the sake of comparison Figure 1 reports also the coverage and the expected length of the intervals
based on the exact FD/CD, which is an inverse-gamma(n, nµ̂), see Veronese & Melilli (2015, Tab.1). The
latter intervals are clearly exact, but wider. Finally, by Corollary 1, the expansion (4) coincides with that of
the Jeffreys posterior. It is easy to verify, according to Corollary 2, that the fiducial prior exists and coincides
with π J (µ) ∝ 1/µ.

⋄

Example 2 (Fisher’s gamma hyperbola-Nile problem). Fisher (1973, Sec.VI.9) considers a sample of size n
from a curved exponential family obtained by two independent gamma distributions with means constrained
on an hyperbole. Following Efron & Hinkley (1978), we directly start with the sufficient statistic S = (S1 , S2 ),
with S1 and S2 distributed according to ga(n, e−η ) and ga(n, eη ), respectively. Here ga(α, β) denotes a gamma
distribution with shape parameter α and mean α/β. It follows that the likelihood of the model is Lη (s) =
exp{−e−η s1 − eη s2 } and that the MLE of η is η̂ = (1/2) log(S1 /S2 ). Even if an exact inference on η cannot
be performed using only η̂, the minimal sufficient statistic S is indeed bivariate, an asymptotic FD/CD for η,
based on η̂, can be easily obtained from Theorem 1. Since ℓ′′′ (η̂) = 0, it follows from (2) that, in this case, the
√
normal distribution N(η̂, b/n), with b = −1/ℓ′′(η̂) = n/(2 s1 s2 ), is an approximate FD/CD of η with error
of order O(n−1 ). Figure 2 reports the plot of its density compared with the exact FD/CD based on S, which
will be derived in the next section. It shows the goodness of the approximation even for a very small sample
size (n = 5 in the plot). Finally, it is easy to check that Eη [(∂ℓ(η)/∂η)3 ] = 0, thus condition (3) holds and, by
Corollary 2, a fiducial prior might exist for this model. Indeed it exists and we will find it in the next section.⋄
Another criterium to define matching priors studied in the Bayesian literature is based directly on the
distribution functions, see Datta & Mukerjee (2004, Sec. 3.2). Because Hn,θ̂ (z) is stochastic in a frequentist setup, as it occurs for a posterior distribution, we can consider the matching between Eθ (Hn,θ̂ (z)) =
5

Figure 2: Approximate (black) and exact(red) fiducial densities for η in the Fisher’s gamma
hyperbola for a sample size n = 5, s1 = 17.321, s2 = 0.116.
np

np
o
o
n/b(θ − θ̂) ≤ z
n/b(θ − θ̂) ≤ z . Clearly quantiles and distribution functions are
Eθ Prθ̂
and Prθ

strongly connected and thus it is not surprising that the conditions for the existence of matching priors in the

two criteria are related. Indeed, the first order matching conditions are the same, while this is not true for
the second order ones. Notice that the matching in terms of quantiles is obtained using the quantity Z which
can be seen as an approximate pivotal quantity. This is meaningful in an asymptotic setting, but it is not
appropriate for small sample sizes. In this case, the FD/CD realizes an exact matching if we replace Z with
the pivotal quantity given by the distribution function of θ̂, namely Fθ (θ̂). Indeed, we have



Eθ Prθ̂ {Fθ (θ̂) ≤ z} = Eθ Prθ̂ {1 − Hθ̂ (θ) ≤ z} =


Eθ Prθ̂ {Hθ̂ (θ) ≥ 1 − z} = 1 − Eθ Prθ̂ {Hθ̂ (θ) ≤ 1 − z} = 1 − Eθ (1 − z) = z,
and because Prθ {Fθ (θ̂) ≤ z} = z, the exact matching for distribution functions holds. However, an exact
FD/CD does not always exist and thus it is natural to look for approximations which have nice asymptotic
properties. Furthermore, in a multiparameter case quantiles are not well defined and thus the study of the
frequentist properties of a multivariate FD/CD can be conducted along the lines developed for matching
distribution functions.

2.2

An approximation based on the Barndorff-Nielsen p∗ -formula

Consider a sample X whose distribution depends on a real parameter θ. In the previous section we have
obtained an approximate FD/CD for θ starting from the distribution of the MLE θ̂. However, if θ̂ is not
sufficient, the approximation of the FD/CD can be improved adding the remaining information included in
the sample. This can be done resorting to the “conditionality resolution” of the statistical model, i.e. the
construction of an ancillary statistic A and of an approximate conditional distribution of θ̂ given A = a. We
refer to Barndorff-Nielsen (1980, 1983) for a detailed discussion on the topic and recall here only some useful
facts. His well known approximate distribution of θ̂ given A = a is
p∗θ (θ̂|a) = c(a, θ)|j(θ̂)|1/2 L(θ; x)/L(θ̂; x),
6

(6)

where L(θ; x) is the likelihood function, j(θ̂) is the observed Fisher information and c(a, θ) is the normalizing
constant which does not depend on θ in many important cases. Formula (6) is quite simple, is generally
accurate to order O(n−1 ), or even O(n−3/2 ), and exact in specific cases. Here the term approximation refers
to one of the two following situations: i) there exists an ancillary statistic A, but it is not possible to construct
the exact conditional distribution of θ̂ given A = a; ii) an exact ancillary statistic does not exist and an
approximate one is used. It is worth to remark that formula p∗ is invariant to reparameterizations and is exact
for transformation models. Furthermore, under repeated sampling from a real NEF, where no conditioning is
involved, p∗ is often of order O(n−3/2 ) and is exact for normal (known variance), gamma (known shape) and
inverse-gaussian (known shape) distributions.
If Fθ∗ (θ̂|a) denotes the distribution function corresponding to p∗θ (θ̂|a) and satisfies the conditions reported
after (1), we can derive an approximate FD/CD for θ as h∗x (θ) = |∂Fθ∗ (θ̂|a)/∂θ|. This construction of a
FD/CD is not different in essence from the widespread procedure used to derive a Bayesian posterior starting
from an approximate (e.g. profile, pseudo or composite) likelihood. A similar approach based on approximate
likelihood is used also by Schweder & Hjort (2016) to construct a CD.
The next result concerning a real NEF is useful when the exact distribution of θ̂ is difficult to obtain.
Proposition 1 If θ̂ is the MLE of θ based on an i.i.d. sample from a real regular NEF, with density pθ (x) =
exp{θx − M (θ)}, then h∗θ̂ (θ) = |∂Fθ∗ (θ̂)/∂θ| is an exact FD/CD for θ based on p∗θ (θ̂). It is an approximate
FD/CD based on the whole sample and its order of approximation depends on that of p∗θ (θ̂).
The following examples, concerning curved exponential families, i.e. NEFs in which a constraint on the
natural parameter space is imposed, illustrate another typical case in which formula (6) can be fruitfully
applied to construct a FD/CD.
Example 2 ctd. As previously observed, the MLE η̂ is not sufficient and thus the exact FD/CD can be
√
obtained starting from the conditional distribution of η̂ given the ancillary statistic A = S1 S2 /n, proposed
by Fisher (1973, Sec. VI.10-11). After some calculations, one obtains

pη (η̂|a) = exp{−2na cosh(η̂ − η)}/(2K0 (2na)),
where K0 (w) =

R∞
0

(7)

exp{−w cosh(z)}dz is the modified Bessel function of the second order evaluated in (0, w).

As observed by Efron & Hinkley (1978), it is easy to see from (7) that this example involves a translation (and
thus a transformation) model, so that pη (η̂|a) = p∗η (η̂|a). Thus the exact FD for η is hη̂,a (η) = −∂Fη∗ (η̂|a)/∂η
and, because η is a location parameter, it equals the posterior obtained from the Jeffreys prior π J (η) ∝ 1, see
Veronese & Melilli (2016, Prop.8). The nature of the parameter η also implies that inferences based on MLE
7

Figure 3: Confidence curves for a sample size n = 15, generated from ρ = 0.3 with s1 = 19.248
and s2 = 4.827, r = 0.414 and ρ̂ = 0.209. Left graph: ccr (green), ccrstab (brown), cc0 (blue) and
cc1 (red). Right graph: cc1 (red), cc∗ (black), ccJ (orange). The horizontal line identifies the 95%
confidence intervals.
and hη̂,a (η) coincide.

⋄

Example 3 (Bivariate normal model). Consider an i.i.d. sample (Xi , Yi ), i = 1, . . . , n, from a bivariate
normal distribution with expectations 0, variances 1 and correlation coefficient ρ. This is a simple curved
Pn
Pn
exponential model with sufficient statistics S1 = i=1 (Xi2 + Yi2 )/2 and S2 = i=1 Xi Yi , but the inference

on ρ is a challenging problem as shown in Fosdick & Raftery (2012) and Fosdick & Perlman (2016). Both
Efron & Hinkley (1978) and Barndorff-Nielsen (1980) use this example to illustrate the construction of an
approximate ancillary statistic in a conditional inference setting. Their proposals essentially coincide and lead
p
to consider the “affine” ancillary A = (S1 − n)/ n(1 + ρ̂2 ), where ρ̂ is the MLE of ρ.
To discuss the performance of h∗ obtained starting from p∗ , we compare it with other possible asymptotic

FDs and with the Bayesian posterior obtained from the Jeffreys prior π J (ρ) ∝ (ρ2 + 1)1/2 /(1 − ρ2 ). In
particular, we consider the following FDs: hr and hrstab obtained from the sample correlation coefficient r
and its stabilizing transformation which, as well known, improves the inferential performance of r, see also
Schweder & Hjort (2016, pag. 209 and 224); h0 and h1 obtained considering the first one or the first two terms
of (2), respectively. We assume a sample size n = 15 because a larger value of n, e.g. 50, produces essentially
the same (good) results for all choices. The left graph of Figure 3 reports an example of the confidence curves
ccr , ccrstab , cc0 and cc1 corresponding to the previous FDs. The curves present different behaviors because
they are based on the two estimators r and ρ̂ of ρ, which assume quite different values in the sample. The
right graph compares cc1 with cc∗ and ccJ obtained from h∗ and Jeffreys posterior, respectively. As expected,
the last two curves, both based on the sufficient statistics S1 and S2 , are very similar and induce confidence
intervals narrower than those induced by cc1 . To better appreciate the good behavior of h∗ , we compare
the corresponding coverage and expected length with those of hr , hrstab and the Jeffreys posterior. Figure
4 confirms the very bad inferential performance of hr . The intervals corresponding to h∗ have the coverage
closest to the nominal one, while those obtained by hrstab present an over-coverage. However, these latter
intervals have a uniformly larger expected length. Finally, Bayesian intervals show an intermediate behavior
in terms of both coverage and expected length. The same example is discussed by Pal Majumder & Hannig
8

1.0
0.8
0.6

expected length

0.4
0.2

coverage

0.86 0.88 0.90 0.92 0.94 0.96 0.98
0.0

0.2

0.4

0.6

0.8

0.0

0.2

ρ

0.4

0.6

0.8

ρ

Figure 4: Coverages and expected lengths of the 95% intervals with n = 15 based on: h∗ (black),
hrstab (brown), π J (orange) and hr (green).

(2016), but they have a different aim and consider different FDs.

3

Asymptotics for fiducial distributions: the multidimensional parameter case

For a parameter θ in Rd , inspired by the step-by-step procedure proposed by Fisher (1973), Veronese & Melilli
(2016) give a simple and quite general definition of FD, which we summarize here. We refer to the latter
paper for details, examples, relationships with objective Bayesian analysis performed using reference priors
and a comparison with Hannig’s fiducial approach. Notice that for a multidimensional parameter there is not
a unique definition of CD, see Schweder & Hjort (2016, Ch.9), so that in the following we refer only to FDs.
Given a random vector S, representing the sample or a sufficient statistic, with dimension m ≥ d and density
pθ , consider the partition S = (S[d] , S−[d] ), where S[d] = (S1 , . . . , Sd ) and S−[d] = (Sd+1 , . . . , Sm ), and suppose
that S−[d] is ancillary for θ. Clearly, if d = m, S−[d] disappears. Thus, the density pθ of S can be written
as pθ (s[d] |s−[d] )p(s−[d] ) and the information on θ provided by the whole sample is included in the conditional
distribution of S[d] given S−[d] . Assume now that there exists a one-to-one smooth reparameterization from θ
to φ = (φ1 , . . . , φd ), with the φi ’s ordered with respect to their inferential importance, such that

pφ (s[d] |s−[d] ) =

d
Y

k=1

pφd−k+1 (sk |s[k−1] , s−[d] ; φ[d−k] ),

(8)

with obvious meaning for s[0] and φ[0] . If, for each k, the one-dimensional conditional distribution function
of Sk is monotone and differentiable in φk and has limits 0 and 1 when φk tends to the boundaries of its
parameter space (this is always true, for example, if this distribution belongs to a regular real NEF), it is
possible to define the joint fiducial density of φ as

hs (φ) =

d
Y

k=1

hs[k] ,s−[d] (φd−k+1 |φ[d−k] ),

9

(9)

where

hs[k] ,s−[d] (φd−k+1 |φ[d−k] ) =

∂
Fφ
(sk |s[k−1] , s−[d] ; φ[d−k] )
∂φd−k+1 d−k+1

(10)

is inspired by the definition of the FD for a real parameter. Some remarks useful in the sequel follow.
i) When m = d = 1, so that an ancillary statistic is not needed, formulas (9) and (10) reduce to hs (φ) =
|∂Fφ (s)/∂φ|, the original proposal of Fisher (1930).
ii) When d > 1 but the parameter of interest is φ1 only, it follows from (9) that its FD is simply given by

hs (φ1 ) =

∂
Fφ (sd |s[d−1] , s−[d] ) ,
∂φ1 1

which is based on the whole sample and is also a CD. A typical choice for Sd is given by the MLE φb1 of φ1

and thus, when φb1 is not sufficient, one has to consider the distribution of φb1 given the ancillary statistic s−[d]

as done in Section 2.2.

iii) The FD in (9) is generally not invariant under a reparameterization of the model unless the transformation
from φ to λ = (λ1 , . . . , λd ) say, maintains the same increasing order of importance in the components of the two
vectors and λk is a function of φ1 , . . . , φk , for each k = 1, . . . , d, i.e. φ(λ) is a lower triangular transformation.
In Veronese & Melilli (2015) it is shown that the univariate FD/CD for a real NEF is asymptotically
normal. Because the multivariate FD defined in (9) is a product of one-dimensional conditional FDs, it is
quite natural to expect that also the FD for a d-dimensional NEF is asymptotically normal.
Theorem 2 Let X = (X1 , . . . , Xn ) be an i.i.d. sample from a regular NEF on Rd with Xi having density
Pd
pθ (xi ) = exp{ k=1 θk xk − M (θ)}, mean vector µ = µ(θ) and variance function V(µ) = Varµ (Xi ). FurtherPn
more, let x̄ be the observed value of the sample mean X̄ = n−1 i=1 Xi . If Xi admits bounded density with

respect to the Lebesgue measure or is supported by a lattice, then the fiducial distribution of µ is asymptotically
order-invariant and asymptotically normal with mean x̄ and covariance matrix V(x̄)/n.

Since V(x̄) coincides with the reciprocal of both the observed and the estimated expected Fisher information
matrix, recalling standard results about asymptotic Bayesian posterior distributions, see e.g. Johnson & Ladalla
(1979), the following corollary immediately holds.
Corollary 3 Consider the statistical model specified in Theorem 2. If we assume a positive prior for µ
having continuous first partial derivatives, then the asymptotic Bayesian posterior for µ coincides with the
asymptotically normal fiducial distribution.

The asymptotic normality for multidimensional generalized fiducial distributions has been proved by Sonderegger & Hannig
10

(2014) under a set of regularity assumptions. We remark that the previous two results are specific for our
definition of FD and hold for NEFs without any extra regularity condition. Furthermore, the proof of Theorem
2, given in the Appendix, is completely different from the standard ones used to show asymptotic normality
in frequentist, Bayesian or generalized fiducial settings. It is based on the convergence of the conditional
distributions determined by the importance ordering of the parameters, it heavily relies on the properties of
the mixed parametrization of the NEF and consequently the result is given in terms of the mean parameter,
which is more interpretable than the natural one.
Consider now a parameter λ = g(µ), with g a one-to-one lower triangular continuously differentiable
function. From Veronese & Melilli (2016, Prop. 1), it follows that the FD for λ can be obtained from that for
µ by the standard change of variable technique and thus we can construct the asymptotic FD in the same way.
However, Theorem 2 states that the asymptotic FD for µ is order invariant and hence it could be interesting
to investigate if this is true also for an arbitrary parameter. This conjecture might be reasonable looking at
what happens in the Bayesian theory where the asymptotic (reference) posteriors do not depend on the order
of the parameter components. The following example illustrate this point.
Example 4. Consider a sample of size n from a multinomial experiment with outcome probability vector
P
P
p = (p1 , . . . , pd ), with dk=1 pk ≤ 1. Then, the vector of counts S = (S1 , . . . , Sd ), with dk=1 Sk ≤ n, is

distributed according to a multinomial distribution with parameters n and p. Using the step-by-step procedure
described above, Veronese & Melilli (2016, formula 25) have proved that the FD for p is a generalized Dirichlet
distribution which depends on the specific fixed ordering of the pi ’s. Assume now d = 2 and consider the
transformation φ1 = p1 /p2 and φ2 = p2 which is not lower triangular. The FD of φ = (φ1 , φ2 ) in this order is
s −1/2

hs (φ) ∝ φ11

s +s2 −1/2

(1 + φ1 )−1/2 φ21

(1 − (1 + φ1 )φ2 )n−s1 −s2 −1/2 .

(11)

This latter is different from the FD induced by that of p but coincides with the posterior distribution obtained
from the reference prior for φ, see Veronese & Melilli (2016, Sec. 5.4).
Consider now the asymptotic setting. From Theorem 2 it follows that the asymptotic FD of p = (p1 , . . . , pd )
is N(x̄, V(x̄)/n), with x̄ = s/n and where the elements of V(x̄) are vkk = x̄k (1 − x̄k ) and vkr = −x̄k x̄r ,
k 6= r. It is easy to verify that for d = 2 it induces on φ a normal distribution with means x̄1 /x̄2 , x̄2 ,
variances x̄1 (x̄1 + x̄2 )/(nx̄32 ), x̄2 (1− x̄2 ) and covariance −x̄1 /x̄2 . This distribution coincides with the asymptotic
distribution corresponding to (11) ( derived for example using standard results on Bayesian theory) and this
fact supports our conjecture that asymptotic FDs are invariant to the importance ordering of the parameters
and can always been derived through the standard delta method.

11

Appendix
Proof of Theorem 1. For the sake of clearness, in this proof we denote by Θ̂ the MLE of a parameter θ and by
θ̂ the corresponding estimate. If Fθ (θ̂) is the distribution function of Θ̂, assumed decreasing in θ, let 1 − Fθ (θ̂)
be the FD for θ. If Fθ (θ̂) is increasing the proof is similar with 1 − Fθ (θ̂) replaced by Fθ (θ̂). Then
Hn,θ̂ (z) = Prθ̂
where θn = z

np
o
n/b(θ − θ̂) ≤ z = Prθ̂ {θ ≤ θn } = 1 − Prθn {Θ̂∗n ≤ θ̂},

(12)

p
∗
b/n + θ̂ and Θ̂∗n is the MLE based on n i.i.d. random variables Xn,i
, i = 1, . . . , n, belonging

to the same family of distributions of Xi , but with parameter θn . Note that θn converges to θ for n → ∞,
because θ̂ converges to the “true” value θ for almost all sequences (x1 , x2 , . . .) and Θ is an open interval. Thus
θn belongs to Θ for n large enough and for each z ∈ R. Starting from (12), we can also write
p
p
Hn,θ̂ (z) = 1 − Prθn { n/b(Θ̂∗n − θn ) ≤ n/b(θ̂ − θn )}
p
p
= Prθn { n/b(Θ̂∗n − θn ) ≥ −z} = Prθn { n/b(θn − Θ̂∗n ) ≤ z}.
Thus, the asymptotic expansion of Hn,θ̂ (z) can be derived by expanding the frequentist distribution function of
p
∗
n/b(θn − Θ̂∗n ). This expansion can be directly obtained by standard results, even if {Xn,i
, i = 1, 2, . . . , n; n =
1, 2, . . . , } is a triangular array because we consider only random variables and a first order approximation, see
p
e.g. Garcı́a-Soidán (1998) and Petrov (1995, Theorem 5.22). The frequentist expansion of Z = n/b(θ − Θ̂)

has been provided in several papers about matching priors under a set of regularity assumptions. Using formula
(3.2.3) in Datta & Mukerjee (2004) with θ = θn and recalling that Θ̂∗n is the MLE of θn , we obtain
Prθn {





p
1 I ′ (θn )
1
ℓ′′′ (θn )
1
1
2
√ + O( ).
n/b(θn − Θ̂∗n ) ≤ z} = Φ(z) − φ(z)
E
(z
+
2)
+
θ
2 I(θn )3/2
6 n (−ℓ′′ (θn ))3/2
n
n

Now, because −ℓ′′ (θ̂)−I(θ̂) = Op (n−1/2 ) (see e.g. Severini, 2000, Sec. 3.5.3) and θn − θ̂ = z

(13)

p
b/n = Op (n−1/2 ),

we have I(θn ) = −ℓ′′ (θ̂) + Op (n−1/2 ) = 1/b + Op (n−1/2 ). Moreover, applying the delta method to the
expectation in (13), this expansion becomes


p
1
1
Prθn { n/b(θn − Θ̂∗n ) ≤ z} = Φ(z) − φ(z) − b3/2 ℓ′′′ (θ̂) + b3/2 ℓ′′′ (θ̂)(z 2 + 2) n−1/2 + O(n−1 )
2
6


1
= Φ(z) − φ(z) b3/2 ℓ′′′ (θ̂)(z 2 − 1) n−1/2 + O(n−1 ),
6
and the theorem is proved.

⋄
12

Proof of Corollary 1. The result follows immediately using the expansion of the posterior distribution provided
by Johnson (1970, Theorem 2.1 and formulae (2.25) and (2.26)), assuming π J (θ) ∝ I(θ)1/2 as prior. Notice
that under the stated conditions on the posterior, this result can be used even if the prior is improper, as
observed in Ghosh et al. (2006, pag. 106).

⋄

Proof of Proposition 1. Recalling that for a real NEF x̄ = M ′ (θ̂), we can write

p∗θ (θ̂) = exp{n(θM ′ (θ̂) − M ∗ (θ))},
R
where M ∗ (θ) = log( exp{n(θM ′ (θ̂))}dν(θ̂)), with ν(θ̂) denoting the dominating measure of the density of

θ̂. Thus p∗θ (θ̂) belongs to a regular real NEF and the result follows immediately by Veronese & Melilli (2015,
Theorem 1).

⋄

Proof of Theorem 2. Given a square d × d matrix A, we use Ak[r] to denote the vector of the first r elements of
the k-th row of A and A[k][k] to denote the matrix identified by the first k rows and columns of A. Moreover,
AT denotes the transpose of A.and
In order to determine the asymptotic FD of µ we apply the step-by-step procedure introduced in Section
3 to the conditional distribution of X̄k given X̄[k−1] = x̄[k−1] for each k. Clearly for k = 1, we have the
marginal distribution of X̄1 . Since the covariance matrix V(µ) of Xi is finite, by the central limit theorem
X̄ is asymptotically N(µ, n−1 V(x̄)) and thus the marginal distribution of X̄[k] is also asymptotically normal
with E(X̄[k] ) = µ[k] and V ar(X̄[k] ) = n−1 V(x̄)[k][k] . Let

−1
λk = µk + V(x̄)k[k−1] V(x̄)[k−1][k−1]
(x̄[k−1] − µ[k−1] )

(14)


−1 
T
qk = V(x̄)kk − V(x̄)k[k−1] V(x̄)[k−1][k−1]
V(x̄)k[k−1] .

(15)

and

Using known results about the convergence of conditional distributions, see Steck (1957, Theorem 2.4) or
Barndorff-Nielsen & Cox (1979, Sec.4), it follows that the conditional distribution of X̄k given X̄[k−1] = x̄[k−1]
is asymptotically N(λk , n−1 qk ).
Now recall that for a NEF it is always possible to consider the so called “mixed parameterization”
(µ[k] , θ−[k] ) which is one-to-one with the natural parameter θ, see e.g. Brown (1986, ch. 3). For θ−[k]
fixed, the distribution of X̄[k] belongs to a NEF with parameter θ [k] and thus the conditional distribution of
X̄k given X̄[k−1] = x̄[k−1] depends only on θk . The same must be true of course for the corresponding asymptotic distribution, so that its mean parameter λk depends only on θk and hence only on µk . Considering now
13

the alternative mixed parameter (µ[k−1] , θk , θ −[k] ), it follows that there exists a one-to-one correspondence
between θk and µk , for µ[k−1] and θ−[k] fixed. As a consequence µ[k−1] can be fixed arbitrarily in the mixed
parameterizations (µ[k−1] , θk , θ−[k] ) with no effect on the conditional distribution and we specifically assume
µ[k−1] = x̄[k−1] . Using the parameter (x̄[k−1] , µk , θ−[k] ), we have that λk coincides with µk , see (14). Summing
up, each of the three parameters λk , θk and µk represents a possible parameterization of the asymptotic conditional distribution of X̄k given X̄[k−1] = x̄[k−1] , for fixed x̄[k−1] and θ−[k] . Thus we can find the asymptotic
FD of λk . Consider now a random vector X̄∗ with distribution belonging to the same family of that of X̄,
√
with mixed parameter (x̄[k−1] , µ∗k , θ −[k] ), where µ∗k = x̄k + zk / n, with zk ∈ R, as in the proof of Theorem 1.
Notice that the marginal distributions of X̄∗[k−1] and of X̄[k−1] are equal. Such a µ∗k is well defined for large
n since (x̄[k−1] , x̄k , θ−[k] ) is a possible value for the mixed parameter in the distribution of the whole vector,
because the NEF is regular and thus the parameter space is open.
For n varying and fixed k, the sequence of marginal sample means X̄∗[k] derives from random vectors whose
mean parameter depends on n, so that it forms a triangular array. In order to determine the FD of λk , we can
√
consider the quantity n(λk − x̄k ), which is a sort of standardization of λk in our fiducial context. Using (1),
similarly to what done in (12), we can write

Prx̄k



zk
λk ≤ x̄k + √ X̄∗ [k−1] = x̄[k−1] , θ−[k]
n

∗
(16)
= 1 − Prλ∗k X̄k ≤ x̄k |X̄∗ [k−1] = x̄[k−1] , θ−[k] ,


√
n(λk − x̄k ) ≤ zk |X̄∗ [k−1] = x̄[k−1] , θ−[k] = Prx̄k

√
where λ∗k = x̄k + zk / n. Since V ar(X̄∗[k] ) is a continuous function of µ∗ = E(X̄∗ ), it converges to a positive
definite matrix for each k when µ∗ converges to the “true” value of µ, for n → ∞. Then, using the result on
the convergence of a conditional distribution presented at the beginning of the proof with µ replaced by µ∗ ,
we have that X̄k∗ given X̄[k−1] = x̄[k−1] is asymptotically N(λ∗k , qk /n). Notice that from the existence of the
second moment of each component of X̄∗[k−1] , it follows that the condition required by Steck (1957, Theorem
2.4, formula (28)), for the case of triangular arrays, is satisfied. Thus, the asymptotic normality of X̄k∗ given
X̄[k−1] = x̄[k−1] implies, for n → +∞,

sup Prλ∗k X̄k∗ ≤ x̄k |X̄[k−1] = x̄[k−1] , θ −[k] − Φ
zk

r


n
(x̄k − λ∗k ) → 0 a.s.
qk

Recalling the expression of λ∗k , we obtain

√
sup Prx̄k +zk /√n X̄k ≤ x̄k |X̄[k−1] = x̄[k−1] , θ−[k] − Φ (−zk / qk ) → 0
zk

14

a.s.

which, using (16), gives

sup Prx̄k
zk


√
√
n(λk − x̄k ) ≤ zk |X̄[k−1] = x̄[k−1] , θ−[k] − Φ (zk / qk ) → 0

a.s.

We can conclude that the conditional FD of λk given θ−[k] is asymptotically normal with mean x̄k and
variance n−1 qk , and thus it does not depend on θ−[k] . Recalling the one-to-one correspondence between θk
and λk , for fixed θ−[k] , and in particular that λd is a one-to-one function of θd , it follows that λ1 , λ2 , . . . , λd
are asymptotically independent, so that the full vector λ = (λ1 , λ2 , . . . , λd ) is asymptotically N(x̄, n−1 Q(x̄)),
where Q(x̄) is the diagonal matrix with k-th element qk .
To obtain the asymptotic FD of µ we consider the one-to-one lower-triangular transformation µ = g(λ),
with λ = g−1 (µ) given by (14) for k = 1, . . . , d. Consider now the lower d × d triangular matrix A = A(x̄)
whose k-th row is made up by the vector −V(x̄)k[k−1] [V(x̄)[k−1][k−1] ]−1 , in the first k−1 positions, 1 in the k-th
position and 0 elsewhere. Thus we can write λ = Aµ + (I − A)x̄ and µ = A−1 λ + (I − A−1 )x̄, with I denoting
the identity matrix of order d. By applying the Cramér delta method it follows that µ is asymptotically normal
with (asymptotic) mean and covariance matrix A−1 x̄ + (I − A−1 )x̄ = x̄ and n−1 A−1 Q(x̄)A−1 T , respectively.
We now show that A−1 Q(x̄)A−1 T = V(x̄) or, equivalently, Q(x̄) = AV(x̄)AT . By direct computation it is
easy to see that the (k, h)-th element of AV(x̄), k, h = 1, 2, . . . , d, is
T

V(x̄)kh − V(x̄)k[k−1] [V(x̄)[k−1][k−1] ]−1 V(x̄)h[k−1] .

(17)

Notice that (17) is 0 for k > h because the product of its last two factors gives a (k −1)-dimensional vector with
1 in the h-th position and 0 otherwise. The matrix AV(x̄)AT is of course symmetric, so that it is sufficient
to proceed only for k ≥ h. On its diagonal we have
T

V(x̄)kk − V(x̄)k[k−1] [V(x̄)[k−1][k−1] ]−1 V(x̄)k[k−1] ,

k = 1, . . . , d,

(18)

because the only nonzero element in the product of the k-th row of AV(x̄) and the k-th column of AT is the
product of (17), with h = k, and 1. For k > h, the (k, h)-th element of AV(x̄)AT is 0, because the first k − 1
components of the k-th row of AV(x̄) and the last d − h components of the h-th column of AT are zero. Thus
the matrix AV(x̄)AT coincides with Q(x̄) and this completes the proof of the theorem.

15

⋄

Acknowledgments
This research was supported by grants from Bocconi University.

References
Barndorff-Nielsen, O. (1980). Conditionality resolutions. Biometrika, 67, 293–310.
Barndorff-Nielsen, O. & Cox, D. R. (1979). Edgeworth and saddle-point approximation with statistical applications. J. R. Stat. Soc. Ser. B, 41, 279–312.
Bickel, P. J. & Ghosh, J. (1990). A decomposition for the likelihood ratio statistic and the bartlett correction–a
bayesian argument. The Annals of Statistics, (pp. 1070–1090).
Brown, L. D. (1986). Fundamentals of statistical exponential families with applications in statistical decision
theory. Lecture Notes-Monograph Series, 9, 1–279.
Datta, G. S. & Ghosh, J. K. (1995). On priors proving frequentist validity for Bayesian inference. Biometrika,
82, 37–45.
Datta, G. S. & Mukerjee, R. (2004). Probability matching priors: higher order asymptotics. Lecture Notes in
Statistics, 178, 1–126.
Efron, B. & Hinkley, D. V. (1978). Assessing the accuracy of the maximum likelihood estimator: Observed
versus expected Fisher information. Biometrika, 65, 457–482.
Fisher, R. A. (1930). Inverse probability. Proceedings of the Cambridge Philosophical Society, 26, 528–535.
Fisher, R. A. (1973). Statistical methods and scientific inference. Hafner Press: New York.
Fosdick, B. K. & Perlman, M. D. (2016). Variance-stabilizing and confidence-stabilizing transformations for
the normal correlation coefficient with known variances. Comm. Statist. Simulation Comput., 45, 1918–1935.
Fosdick, B. K. & Raftery, A. E. (2012). Estimating the correlation in bivariate normal datat with known
variances and small sample size. The American Statistician, 66, 34–41.
Garcı́a-Soidán, P. H. (1998). Edgeworth expansions for triangular arrays. Communications in Statistics-Theory
and Methods, 27(3), 705–722.
Ghosh, J. K. (1994). Higher order asymptotic. Institute of Mathematical Statistics and American Statistical
Association: Hayward, California.
16

Ghosh, J. K., Delampady, M., & Samanta, T. (2006). An introduction to Bayesian analysis. Springer: New
York.
Hannig, J. (2009). On generalized fiducial inference. Statist. Sinica, 19, 491–544.
Hannig, J., Iyer, H. K., Lai, R. C. S., & Lee, T. C. M. (2016). Generalized fiducial inference: A review and
new results. J. American Statist. Assoc., 44, 476–483.
Johnson, R. A. (1970). Asymptotic expansions associated with posterior distributions. Ann. Math. Statist,
41, 851–864.
Johnson, R. A. & Ladalla, J. N. (1979). The large sample behaviour of posterior distributions when sampling
from multiparameter exponential family models, and allied results. Sankhyā, Series B, 41, 196–215.
Lindley, D. V. (1958). Fiducial distributions and Bayes theorem. J. R. Stat. Soc. Ser. B, 20, 102–107.
Liu, D., Liu, R. Y., & Xie, M. (2015). Multivariate meta-analysis of heterogeneous studies using only summary
statistics: efficiency and robustness. J. Amer. Statist. Assoc., 110, 326–340.
Mukerjee, R. & Ghosh, M. (1997). Second-order probability matching priors. Biometrika, 84, 970–975.
Pal Majumder, A. & Hannig, J. (2016). Higher order asymptotics of Generalized Fiducial Distribution.
arXiv:1608.07186 [], (pp. 1–33).
Petrov, V. V. (1995). Limit theorems of probability theory. Clarendom Press: Oxford.
Schweder, T. & Hjort, N. L. (2016). Confidence, likelihood and probability. London: Cambridge University
Press.
Severini, T. A. (2000). Likelihood methods in statistics, volume 22. Oxford University Press, Oxford.
Sonderegger, D. L. & Hannig, J. (2014). Fiducial theory for free-knot splines. In Contemporary Developments
in Statistical Theory (pp. 155–189). Springer, New York.
Steck, G. P. (1957). Limit theorems for conditional distributions. Univ. California Publ. Statist, 2, 237–284.
Taraldsen, G. & Lindqvist, B. H. (2015). Fiducial and posterior sampling. Communications in Statistics Theory and Methods, 44, 3754–3767.
Veronese, P. & Melilli, E. (2015). Fiducial and confidence distributions for real exponential families. Scand.
J. Stat., 42, 471–484.

17

Veronese, P. & Melilli, E. (2016). Objective bayesian and fiducial inference: some results and comparisons. To
appear in Journal of Statistical Planning and Inference; arXiv:1612.01882 [], (pp. 1–37).
Xie, M. & Singh, K. (2013). Confidence distribution, the frequentist distribution estimator of a parameter: a
review. Internat. Stat. Rev, 81, 3–39.

18

