Fiducial and Confidence Distributions for Real Exponential Families

Transcription

1 Scandinavian Journal of Statistics doi: /sjos Published by Wiley Publishing Ltd. Fiducial and Confidence Distributions for Real Exponential Families PIERO VERONESE and EUGENIO MELILLI Department of Decision Sciences, Bocconi University, Milan, Italy ABSTRACT. We develop an easy and direct way to define and compute the fiducial distribution of a real parameter for both continuous and discrete exponential families. Furthermore, such a distribution satisfies the requirements to be considered a confidence distribution. Many examples are provided for models, which, although very simple, are widely used in applications. A characterization of the families for which the fiducial distribution coincides with a Bayesian posterior is given, and the strict connection with Jeffreys prior is shown. Asymptotic expansions of fiducial distributions are obtained without any further assumptions, and again, the relationship with the objective Bayesian analysis is pointed out. Finally, using the Edgeworth expansions, we compare the coverage of the fiducial intervals with that of other common intervals, proving the good behaviour of the former. Key words: conjugate prior, coverage probability, Edgeworth s expansion, Jeffreys prior, natural exponential family, posterior distribution, quadratic variance function 1. Introduction Fiducial inference was introduced by R.A. Fisher in the 1930s; see Fisher (1935, 1973) and, for a general review, Pedersen (1978). Roughly speaking, the underlying idea is to have a method to transfer randomness from the observed quantity T to the parameter indexing the statistical model. The aim is to build a probability distribution on, capturing all the information given by the data, without resorting to the Bayes theorem, which requires prior information. Originally, Fisher considered a continuous sufficient statistic T with cumulative distribution function F depending on a real parameter. He showed that, under suitable conditions, the quantiles of F can be used to identify the quantiles of the fiducial distribution. These are obtained by solving, with respect to, the inequality t<q./, where t is an observed value of T and q./ is the quantile of F. Equivalently, if q./ is an increasing function of, the cumulative fiducial distribution function of and the corresponding density can be defined respectively by H t./ D 1 F.t/ and h F.t/: (1) For example, let N X be the mean of a sample of size n from a normal population with mean and known variance 2, written as N.; 2 /, and denote by ˆ and the cumulative distribution function and the quantile of a N.0; 1/, respectively. In this case, for an observed Nx, solving Nx <q./ D C = p n, we obtain > Nx = p n. According to Fisher, Nx = p n defines the 1 quantile of the fiducial distribution of, which is therefore N Nx; 2 =n.in other words, we obtain H Nx./ D 1 F. Nx/ D 1 ˆ p n.nx / = D ˆ p n. Nx/=. Notwithstanding several efforts, Fisher was not able to provide a general theory applicable to complex models with several parameters or not admitting sufficient statistics. He also explicitly recognized the difficulty to address the discrete case and to handle the non-uniqueness inherent in the fiducial approach. Moreover, the construction of a distribution on depending on t, not based on the Bayes theorem, implies that a joint distribution of and T is not required, and this is a source of probability inconsistencies; see Zabell (1992) and Dawid & Stone (1982).

2 2 P. Veronese and E. Melilli Scand J Statist Various attempts to retain the fiducial idea avoiding its drawbacks have been made till now. Many of them start from a slight reformulation of the previous argument, using the notion of pivotal quantity, a function U of T and with completely known distribution. Assuming that this distribution does not change before and after T has been observed, direct probability statements on can be derived inverting the pivotal function. In particular, Fraser (1961, 1968) provided a rigorous framework for fiducial inference, associating a group structure with the statistical model and introducing the notion of structural equation to link T with and U. Studies on fiducial inference are still going on; see, for example, Hannig (2009), which also included an extensive review, Taraldsen & Lindqvist (2013) and Martin & Liu (2013). The generalized fiducial inference suggested by Hannig seems to us the proposal closest to the original idea of Fisher. It is quite general, and it includes also discrete models and seems to perform well in several context; see Hannig et al. (2007), Hannig & Iyer (2008) and Wandler & Hannig (2012). However, all general approaches to fiducial inference have potential sources of non-uniqueness, and this appears somewhat in contrast with the objectivity of the procedure originally advocated by Fisher. The concept of fiducial distribution is strictly related to that of confidence distribution. Starting from a one-sided level confidence interval. 1;.t// corresponding to an observation t, a confidence distribution was originally defined as the inverse function of.t/ with respect to, denoted by t./. For instance, for the mean in the normal example,.t/ D. Nx/ DNxC = p n, so that Nx./ D ˆ p n. Nx/=, that is, the confidence distribution for coincides with the fiducial distribution N Nx;= p n. Confidence distributions have recently received a new attention after the modern definition, which does not involve any controversial, is clean and unifies a wide range of inferences; see Schweder & Hjort (2002) and Singh et al. (2005). A confidence distribution is a purely frequentist concept, and it is currently viewed as an estimator for the parameter of interest, instead of an inherent distribution of the parameter (Xie & Singh, 2013, p. 9). We also refer to this article for an extended review on the topic. In our paper, we consider only models belonging to the real natural exponential family (NEF). Even if this setting is quite restricted, it includes many important commonly used distributions and allows a unified treatment of the continuous and discrete cases. We show that for a NEF, there exists a fiducial distribution according to (1), which is also a confidence distribution in its modern meaning. As it can be obtained without any prior information, the connection with the objective Bayesian analysis appears to be worthy of investigation. Indeed, we will show that the proposed fiducial/confidence distribution has many points of contact with the posterior distribution obtained via the Jeffreys prior for both small and large sample sizes. Thus, it inherits the good properties of the Jeffreys posterior, without requiring the adoption of the Bayes theorem. More specifically, in Section 2, we briefly review the real NEF. In Section 3.1, we state the existence of the fiducial/confidence distribution and provide the expressions for its cumulative distribution and its density. For discrete models, two possible distributions can be derived, and we address this non-uniqueness by proposing suitable averages. Many examples are given, and a relationship between the expectation of the fiducial distribution of the natural parameter and its maximum likelihood estimate (MLE) is stated. In Section 3.2, assuming a fixed sample size and starting from a result by Lindley (1958), we discuss and characterize the relationships among fiducial distributions, Bayesian posteriors, conjugate families and the Jeffreys priors. In Section 4, we show with no extra-assumptions that the proposed fiducial/confidence distribution is asymptotically normal, and we measure the accuracy of the approximation using the Edgeworth expansion. Moreover, we prove that it is asymptotically equivalent up to order O.n 1 / to the Bayesian posterior obtained by the Jeffreys prior. A sort of inverse problem was

3 Scand J Statist Fiducial and confidence distributions 3 faced by Welch & Peers (1963). They identified the Jeffreys prior as the unique distribution under which the posterior probabilities of one-sided intervals approximately coincide with their coverage probabilities. In other terms, they found a matching prior; see, for example, Datta & Mukerjee (2004). It is worth emphasizing that, contrary to Welch and Peers, we study the asymptotic form of a confidence distribution, which is not obtained via the Bayes theorem. In Section 5, following the approach suggested by Brown et al. (2002, 2003), expansions for the coverages and the expected lengths of the intervals based on the fiducial distributions are provided, and their good performances are shown through a comparison with other commonly used confidence intervals. Finally, in Section 6, we briefly discuss a possible extension to the multivariate case. All proofs are included in the Supporting Information, together with details on examples, a table and other graphs useful for further evaluations of the fiducial/confidence distributions. 2. Preliminary results on the natural exponential family Here, we provide a brief review of the real NEF. For a thorough treatment, see Barndorff-Nielsen (1978) and, with a view towards Bayesian applications, Gutiérrez-Peña & Smith (1997). Given a non-degenerate -finite measure on R, define M./ D ln R exp¹ xº.dx/, and let be the interior of N D¹ 2 R W M./ < 1º. The family F of probability measures on R whose densities, with respect to, are of the form p.x/ D exp¹ x M./º, 2, is called NEF with natural parameter, provided is not empty. In particular, the family is said to be regular if N is open, that is, N D. The function M./ is convex, and it is called the cumulant transform of the measure. In the sequel, F will always denote a regular NEF. The family F can be alternatively parameterized in the mean parameter D./ D M 0./ D dm./=d, as./ is a one-to-one differentiable map from onto D. /. The inverse function of./ is denoted by./, that is, D./. The variance of F is given by M 00./ D d 2 M./=d 2. The function V./ D M 00..// defines the variance function, which, together with the meandomain, characterizes the family. Morris (1982) showed that the most widely used families of distributions belong to the class of real NEFs with quadratic variance function (QVF), for which V./ D Q 2 C L C C > 0 with 2 and Q; L; C 2 R. More precisely, Morris proved that these families can be obtained by a non-singular affine transformation of one of the following six basic families: normal, N.; 2 /, with known variance 2 ; binomial, Bi.m; p/, with known m; Poisson, Po./; Gamma, Ga. ; ˇ/, with mean D =ˇ and known ; and negative-binomial, Ne-Bi(m; p), with known m and generalized hyperbolic-secant. Consider now n random variables X 1 ;:::;X n independent and identically distributed (i.i.d.) according to the density p of F. Then S n D P n id1 X i is the minimal sufficient statistics for the sample, and its density, with respect to the convolution measure n of, is p n;.s/ D exp¹ s nm./º; 2 : (2) Thus, S n is distributed according to a NEF with natural parameter, cumulant transform nm./ and cumulative distribution function denoted by F n;. 3. Fiducial and confidence distributions 3.1. Definitions and examples As remarked in Section 1, contrary to the fiducial distributions that have never been fully defined, the definition of confidence distribution is unique and clear. Denoting by x the realization of a random sample X of size n, Schweder & Hjort (2002), see also Singh et al. (2005), proposed the following.

4 4 P. Veronese and E. Melilli Scand J Statist Definition 1. A function H n;x./ is called a confidence distribution for a parameter 2 if: (i) for each x;h n;x./ is a cumulative distribution function on ; and (ii) at the true parameter value D 0 ;H n;x. 0 /, as a function of X, follows the uniform distribution. H n;x./ is called an asymptotic confidence distribution if assumption (ii) holds only asymptotically. However, in the context of real NEFs, it is possible to construct in a simple way a fiducial distribution that agrees with the original idea of Fisher recalled in (1) and is a confidence distribution at the same time. The starting point is the following result by Petrone & Veronese (2010). Theorem 1. Let F n;.s/ be the cumulative distribution function of a NEF F, with density (2) and support S n. Let a n D inf S n, b n D sup S n and define Sn D Œa n;b n / if n.a n />0; otherwise, Sn D.a n;b n /. Then, for s 2 Sn, the function 8 ˆ< 0 inf H n;s./ D 1 F n;.s/ inf <<sup (3) ˆ: 1 sup is an absolutely continuous cumulative distribution function with density, for 2, h H n;s./ F n;.s/ D.nM 0./ t/p n;.t/ d n.t/: (4). 1;s The previous theorem implies that H n;sn./ is continuous and monotonic in, and, as a function of a continuous S n, it follows a uniform distribution. As a consequence, H n;sn./ is a pivot. More generally, we can state the following. Proposition 1. Given an i.i.d. sample x 1 ;:::;x n from a NEF F, let s D P n id1 x i. Then H n;s./ in (3) is both a cumulative fiducial distribution function of the natural parameter satisfying (1) and a (possibly asymptotic) confidence distribution for according to definition 1. It follows that h n;s./ in (4) is both the fiducial and confidence densities for. The cumulative fiducial/confidence distribution function H n;s./ depends on the sufficient statistic S n, and thus, besides containing all the sample information, it has optimality properties in terms of precision; see, for example, Xie & Singh (2013, sec. 5). We remark that, for continuous NEFs, H n;s./ also satisfies the definition of the fiducial distribution given by Dawid & Stone (1982). More specifically, there exists a Fisherian simple functional model compatible with F n;, which generates a distribution for coinciding with H n;s, as, by theorem 1, F n; satisfies the sufficient regularity conditions stated in Dawid & Stone (1982, sec. 2). For brevity, in the sequel, we will often refer to the fiducial/confidence distribution H n;s simply as fiducial distribution. For discrete NEFs, F n;.s/ D Pr ¹S n sº and Pr ¹S n <sº do not coincide, and thus, besides H n;s in (3), one could define a left fiducial distribution as Hǹ;s./ D 1 Pr ¹S n <sºd1 Pr ¹S n s 1 ºDH n;s 1./; (5) where s 1 2 S n denotes the point before s in the support of S n. For convenience, H n;s will be sometimes called right fiducial distribution. As observed in Section 1, the non-uniqueness of fiducial distributions in discrete models is a common aspect of all approaches. Here, a natural way to overcome this problem might be considering the mixture H M n;s./ D.H n;s./c

5 Scand J Statist Fiducial and confidence distributions 5 Hǹ;s.//=2 D Pr ¹S n >sºc1=2 Pr ¹S n D sº. Remarkably, Hn;s M coincides with the approximate confidence distribution proposed for discrete data by Schweder & Hjort (2002); see also Hannig & Xie (2012). Theorem 1 holds for each s 2 Sn, but it fails for s D b n < 1, as H n;bn./ D 1 F n;.b n / D 0 for each 2. This drawback clearly arises only if n.b n />0. In this case, one could use Hǹ;s, which, however, cannot be defined for s D a n if n.a n />0, as it occurs in the binomial model. Thus, both H n;s and Hǹ;s, and hence their mixture, can be undefined in specific cases. A possible solution will be discussed in Sections 3.2 and 4. Fiducial distributions are invariant under monotone continuous reparameterizations of the model. More precisely, if D./ is an increasing differentiable function of, then the cumulative fiducial distribution function of is Hn;s./ D 1 F n;./.s/ D H n;s..//. The corresponding density h n;s./ coincides with that obtained directly via a change-of-variable technique starting from h n;s./. A similar result holds if is decreasing in. An important example is given by the mean parameterization D M 0./. Asd=d D M 00./ and recalling that M 00..// D V./, wehave h n;s./ D h 1 n;s..// V./ D 1 Z.n t/p n;./.t/ d.t/ : (6) V./. 1;s In the sequel, we will denote by H n;s./ and h n;s./ the cumulative distribution function and the density of an arbitrary parameter, omitting the superscripts. We now illustrate the immediacy of our approach with several examples. Example 1 (Normal distribution). Let F be the family of the normal distributions N.; 2 /, with known 2.AsXN n D S n =n is distributed as a N.; 2 /, it follows immediately from (3) that the fiducial distribution of isan Nx; 2 =n, as seen in Section 1. Example 2 (Exponential distribution). Let F be the family of the (negative) exponential distributions Ga.1; / with mean D 1=, >0. Here, D, D. 1;0/, M./ D log. / and S n D P n id1 X i is distributed according to a Ga.n; /. Thus, by (4), for s 2.0; C1/, Z s h n;s./ D h n i t e t. / n t n 1.n/ dt D sn.n/. /n 1 e s : 0 Thus, the fiducial distribution of isaga.n; s/ and that of is an inverse-gamma In-Ga.n; s/. Example 3 (Bernoulli distribution). Let F be the family of the Bernoulli distributions Bi(1; p/, with success probability (and mean) p. Then S n is distributed as a Bi.n; p/. Using (3), we have! nx Z n H n;s.p/ D1 F n;.p/.s/ D p t.1 p/ n t 1 p D t s.1 t/ n s 1 dt; s B.sC1; n s/ tdsc1 for 0 s<n, where the last equality derives from the standard result linking the binomial and beta distributions. Thus, the fiducial distribution of p is a Be(s C 1; n s). As this density does not exist for s D n, in this case, one could use the left fiducial distribution Hǹ;s.p/, which is a Be(s; n s C 1), by (5). In turn, Hǹ;s.p/ is not defined for s D 0. The fiducial distribution for the parameter p has a long history. The mixture H n;s M D Hn;s C Hǹ;s =2 has been explicitly considered by Efron (1998). Moreover, both Hn;s and Hǹ;s, with all their possible mixtures, can be seen as members inside the class of generalized fiducial distributions proposed by Hannig (2009, example 6). He compares the elements of the class and concludes by recommending Hn;s M or, as a valid alternative, a fiducial distribution formally coinciding with the Jeffreys posterior. We will return on this connection later on. 0

6 6 P. Veronese and E. Melilli Scand J Statist Example 4 (Poisson distribution). Let F be the family of the Poisson distributions Po./ with mean. Then V./ D, and S n is distributed according to a Po.n/. Using formula (6), the fiducial density of is, for s D 0;1;:::, h n;s./ D 1 sx td0 " n t sx tš Œn t t e n D e n " sc1 X D e n n t.t 1/Š t 1 td1 sx td1 td0 n tc1 n t.t 1/Š t 1 t tš # sx td1 # n t.t 1/Š t 1 D nsc1 e n s ; sš which is a Ga.s C 1; n/. From (5), it follows that Hǹ;s.p/ isaga.s; n/, provided s 0. Example 5 (Geometric distribution). Let F be the family of geometric distributions, which describe the number of failures before the first success, and let p 2.0; 1/ be the success probability. Here, D log.1 p/, D. 1;0/, M./ D log.1 e / and S n has a negative-binomial distribution, Ne-bi.n; p/. In this case, the fiducial density for p can be obtained from (4) via a change of variable technique; see Appendix S2 in the Supporting Information for algebraic calculus. Thus, we have, for s 2¹0;1;:::º, h n;s.p/ D sx kd0! k C n 1 n.1 p/ k.1 p/ k 1 p n.n C s/š D n 1 p.n 1/ŠsŠ pn 1.1 p/ s ; which is a Be.n; s C 1/. From (5), it follows that Hǹ;s.p/ isabe.n; s/, provided s D 0. Example 6 (Pareto distribution). Let F be the family of Pareto distributions, with density p.x/ D x0 x 1, x>x 0 >0, >0. Here, D, and S n D P n id1 log.x i =x 0 / has distribution Ga.n; /. From example 2, it follows that the fiducial density for isaga.n; s/. Example 7 (Weibull distribution). Let F be the family of Weibull distributions, with density p.x/ D cx c 1 exp. x c /, x>0, >0, c>0. Here, D, and S n D P n id1 X c has i distribution Ga.n; /. Then, as before, h n;s./ is a Ga(n; s). Example 8. Let be the density of the standard Laplace distribution, that is,.x/ D exp¹ jxjº=2. Then, the cumulant transform of is M./ D log.1 2 / and p.x/ D exp x C log.1 2 /, x 2 R, 2. 1; 1/. From (4), assuming n D 1, wehave h 1;x./ D 1.x CjxjC1/ exp¹x jxjº: (7) 2 The expression of h n;s Information. for an arbitrary n is given in Appendix S2 in the Supporting The regularity of the NEF F is a crucial aspect to have a continuous fiducial distribution as required in proposition 1. For example, consider the inverse-gaussian model with density p.x/ D.2/ 1=2 x 3=2 exp x C. 2/ 1=2.2x/ 1± ; x > 0; 0; which is not regular, as is closed. Assuming for simplicity n D 1,wehavelim!0 H 1;x./ D 1 lim!0 F 1;.x/ D 1 2ˆ.1= p x/<1, and thus, H 1;x./ is not a distribution function.

7 Scand J Statist Fiducial and confidence distributions 7 We conclude this section with the following. Proposition 2. If F is a continuous NEF-QVF with density (2), then denoting by E Hn;s./ the expected value of with respect to H n;s, we have E Hn;s./ D M 0 1.s=n/. The previous result is useful when the expression of the fiducial density is not available in closed form, as in the case of the generalized hyperbolic-secant family. Furthermore, it allows one to establish a relationship between the natural fiducial estimate E Hn;s./ of and the MLE. As the MLE of D M 0./ is O D s=n, it follows from the invariance property of these estimators that O D M 0 1.s=n/, and thus, the MLE of coincides with the fiducial estimate. This confirms for continuous NEF-QVFs a perfect agreement with the standard results of the frequentist approach. Notice that proposition 2 does not hold for discrete NEF-QVFs, as can be immediately seen from example 4, or for NEFs with non-qvf. For this latter case, consider example 8. Here, M 0./D2=.1 2 /, 1<<1, and thus, DM 0 1./D..1C 2 / 1=2 1/=. A direct computation of the expected value of the fiducial density (7) gives, for x > 0, E H1;x./ D.e 2x 2x 2 C 2x 1/=2x 2, which is different from M 0 1.x/. Table 1 provides the fiducial distributions with the corresponding expected values and variances for some parameters of interest for the five most important basic NEF-QVFs. The results for Hǹ;s in the discrete cases can be easily obtained by replacing s with s Connections with Bayesian posterior distributions In examples 1 5, it is immediate to recognize that the fiducial/confidence distribution H n;s belongs to the conjugate family of the model, and thus, it is easy to verify the existence of a prior generating a posterior coinciding with H n;s. We will call such a prior fiducial prior. This problem was firstly addressed by Lindley (1958). He showed that, among continuous NEFs, only normal and Gamma models admit a fiducial prior. The two previous models have a QVF, and thus, this is a necessary condition in the continuous case. We prove in Appendix S1 of the Supporting Information that this condition is also necessary in the discrete case and plays a crucial role in the characterization of the NEFs admitting a fiducial prior. Table 1. Fiducial distributions for some real exponential families with corresponding expectations and variances for various parameterizations Fiducial density Expected value Variance N.; 2 / = 2 N(s=n; 2 =n/ Q D s=n 2 =n ( 2 known) Ga( ; ) = Ga(n ; s) Q D n =s n =s 2 ( known) In-Ga(n; s) Q D s=.n 1/ Q 2 = Bi(m; p) log p 1 p mp p Be.s C 1; nm s/ Qp D sc1 nmc1 Qp.1 Qp/ nmc2 (m known) Q D Œ 0.s C 1/ Œ 1.s C 1/C 0.nm s/ 1.nm s/ Po() log Ga(s C 1; n) Q D.s C 1/=n.s C 1/=n 2 Ne-Bi(m; p) log.1 p/ m.1 p/ p p Be(nm;s C 1) Qp D nm nmcsc1 Qp 2.1 Qp/ nmc Qp (m known) Q D Œ 0.s C 1/ Œ 1.s C 1/C 0.nm C s C 1/ 1.nm C s C 1/ The fiducial expected value of an arbitrary parameter is denoted by Q; 0 denotes the di-gamma function and 1 the tri-gamma function.

8 8 P. Veronese and E. Melilli Scand J Statist Theorem 2. A fiducial prior exists if and only if the NEF F is an affine transformation of one of the following families: normal with known variance, Gamma with known shape parameter, binomial, Poisson and negative-binomial. Moreover, for the normal family, the fiducial prior of the natural parameter is constant, while for all other basic families, it is proportional to M 0./. We now better clarify the connections between fiducial and conjugate priors for a fixed sample size. A family C.F/ of measures on, whose densities with respect to the Lebesgue measure are of the form. j s 0 ;n 0 / / exp¹s 0 n 0 M./º, s 0 2 R, n 0 2 R, is called standard conjugate to the family F. It is well known that, given a sample of size n from a NEF, the prior. j s 0 ;n 0 / generates the posterior distribution. j s 00 ;n 00 /, where s 00 D s C s 0 and n 00 D n C n 0. Proposition 3. Given the NEF F with natural parameter, if the fiducial prior exists, then the fiducial distribution H n;s./ belongs to C.F/ with s 00 D s C L and n 00 D n Q, where Q and L are the coefficients appearing in the variance function V./ D Q 2 C L C C of F. The connection between conjugate families and fiducial distributions is useful because we can use for the latter the well-known properties of the former, as illustrated in the following. Proposition 4. Let F be one of the families characterized in theorem 2 and write.dx/ D c.x/ 0.dx/, where 0 is the Lebesgue or counting measure. Then, for n 00 D n Q > 0 and s 00 =n 00 2, with s 00 D s C L, we have the following moments computed under H n;s : E Hn;s./ D s00 D n n 00 n Q Nx C L n Q ; VarHn;s./ D V.s00 =n 00 / n 2Q ; E log.r.s00 ;n 00 //; Var Hn;s./ log.r.s 00 ;n 00 2 E Hn;s.M.// D log.r.s00 ;n 00 //; Var Hn;s.M.// D 2 log.r.s 00 ;n 00 //; 2 where r.u; m/ D mc m.u/v.u=m/ with c m.u/ being the m-convolution of c.u/. Gutiérrez-Peña & Smith (1997) proved that for a NEF-QVF, the Jeffreys prior on is a standard conjugate with sj 0 D L=2 and n0 J D Q. Thus, a fiducial prior has the same prior sample size n 0 of the Jeffreys prior, while it has a different s 0, namely L=2 instead of L. However, for continuous models, L D 0, and thus, fiducial distributions and the Jeffreys posteriors coincide. For discrete models, where L D 1, this is not true. However, if we consider Hǹ;s instead of H n;s, proposition 3 holds with s replaced by s 1, and thus, s 00 ` D s C L 1 D s and n 00 ` D n00 D n Q. This suggests to consider, as a compromise between H n;s./ and Hǹ;s./, the average fiducial distribution Hn;s A./ obtained as the conjugate with parameters given by the arithmetic means s 00 C s 00 ` =2 D s C 1=2 and n 00 C n` 00 =2 D n Q, so that the relationship with the Jeffreys posterior still holds. The distribution Hn;s A./, contrary to the mixture Hn;s M D H n;s C Hǹ;s =2, is well defined for all s, and then it can be a valid alternative to overcome the non-uniqueness of the fiducial distribution in the discrete case. Indeed, Hn;s M and Hn;s A have a similar behaviour, sharing the same expected value of given by.s C1=2/=.n Q/ and having variances of, which differ by ¹4.n 2Q/.n Q/º 1 D O.n 2 /, as can be proved using the results in proposition Asymptotic behaviour From Table 1, because XN n D P n id1 X i =n converges to the true value 0 of for n!1,it follows that E Hn;s./! 0 and Var Hn;s./! 0. As a consequence, H n;s is consistent at 0, that is, it converges almost surely to the distribution degenerate at 0. This fact suggests to

9 Scand J Statist Fiducial and confidence distributions 9 study in details the asymptotic behaviour of H n;s and Hǹ;s. We present the results in terms of the mean parameter, but they can be extended to any smooth parameterization using Cramer s delta method. First, we prove a Bernstein Von Mises type theorem for H n;s and Hǹ;s and study their rate of convergence to normality. Owing to the properties of the NEF, our results follow without any assumption. In a more general setting, Hannig (2009, theorem 2) shows that, typically, the generalized fiducial distributions are asymptotically normal, but he requires a set of non-trivial assumptions. Theorem 3. Let X 1 ;X 2 ;::: be a sequence of i.i.d. random variables from a NEF F with mean parameter 2 and variance function V./. Let 0 2 be the true value of the parameter, and, for each n 1, let Nx n D P n id1 x i =n D s=n be the realization of the sample mean P n id1 X i =n D S n =n. Then, if is distributed according to H n;s./ or Hǹ;s./,.V. 0 /=n/ 1=2. Nx n / d! N.0; 1/ (8) for almost all sequences.x 1 ;x 2 ;:::/. The result remains true if 0 is replaced in (8) by Nx n. The previous theorem is also relevant in establishing consistency properties of some natural point estimators based on H n;s. Indeed, it implies that the condition.a 0 / required in theorem 1 of Xie & Singh (2013) is satisfied. As a consequence, the median of H n;s is a consistent estimator of, and the same is true for the mean of H n;s if the second moment of H n;s is bounded in probability. Theorem 3 can be refined, and an expansion of the fiducial distribution up to an error of order O.n 1 / can be provided. First, we recall that a random variable is said to have a lattice distribution if, with probability one, it takes on values of the form a C kr.k D 0; 1; 2;:::/, where a is a constant and r>0is the maximal span of the lattice. Theorem 4. Let X 1 ;X 2 ;::: be a sequence of i.i.d. random variables as in theorem 3. Denote by H n;s the cumulative fiducial distribution function of Z D p n. Nx n / and by the density of a standard normal. Then if the random variables are non-lattice, H n;s can be expanded as H n;s. / D ˆ! p V.Nxn /! p V.Nxn / V 0. Nx n / 2 2 C V.Nx n / V.Nx n / 3=2 6 while for lattice random variables with maximal span r, we have n 1=2 CO.n 1 /; (9)! H n;s. / D ˆ p V.Nxn /!" # V 0. Nx n / 2 2 C V.Nx n / r p V.Nxn / V.Nx n / 3=2 6 2 p V.Nx n / n 1=2 C O.n 1 / (10) on an almost sure set of sequences.x 1 ;x 2 ;:::/. The expansion of the left cumulative fiducial distribution function Hǹ;s is given by (10) with r replaced by r. It is easy to verify that (9) coincides with the asymptotic expansion of the posterior distribution obtained by Johnson (1970) under an exponential model and the Jeffreys prior. Johnson s result also holds when the prior is improper if, as remarked by Ghosh et al. (2006, p. 106), there exists an n 0 such that the posterior distribution of given.x 1 ;:::;x n0 / is proper for almost

10 10 P. Veronese and E. Melilli Scand J Statist all.x 1 ;:::;x n0 /. Thus, theorem 4 ensures that, for continuous NEFs, the fiducial and Jeffreys posterior distributions are at least asymptotically equivalent up to the order O.n 1 /. For lattice distributions, it is immediate to see that the same result holds for the mixture Hn;s M of the two fiducial distributions Hǹ;s and H n;s. Summing up, for all NEFs, the Jeffreys posterior can be used as a good approximation of the fiducial distribution. In Figure S1 of the Supporting Information, we use the model presented in example 8 to show that the approximation is good even if the sample size is too small to have asymptotic normality. 5. Coverage and expected length of fiducial intervals 5.1. Coverage According to Fisher s point of view, it is important to ensure that a fiducial distribution leads to procedures that are correct under the frequentist approach, at least asymptotically. As, for a continuous NEF, H n;s defined in (3) is a confidence distribution, this requirement is automatically satisfied. In particular, from point (ii) of Definition 1, the interval 1;Hn;s 1.1 / is an exact one-sided confidence interval of level 1 for. In a Bayesian setting, a similar problem has led to the notion of matching prior. This prior ensures that the corresponding posterior has 1 level credible sets with approximate 1 frequentist coverages; see Datta & Mukerjee (2004) for a general discussion. Notice that for the normal and Gamma models, for which the confidence property is exact and there exists a fiducial prior coinciding with the Jeffreys prior, this turns out to be an exact matching prior, as already well known. For the other continuous NEFs, fiducial intervals become a good alternative to Bayesian or to other commonly used intervals owing to their exact confidence property. However, even in this case, the fiducial and Jeffreys posterior distributions are very close by theorem 4, and consequently, confidence intervals are similar. Table S1 in the Supporting Information provides a numerical comparison of the fiducial, Jeffreys and Wald intervals, the third being significantly different from the first two. For discrete NEFs, the confidence property holds only asymptotically so that we discuss in details the intervals for the mean of three important families: binomial, Poisson, and negativebinomial. We use the second-order Edgeworth expansions to compare the coverages of the fiducial intervals with those of the Wald and Jeffreys intervals. Notice that the last intervals coincide with those generated by Hn;s A. As observed by Brown et al. (2003), in the discrete case, the difference between the true and nominal coverages can be split into two groups of terms. The first group includes the oscillating terms of order n 1=2 and n 1 deriving from discreteness, while the second one includes only the non-oscillating terms of order n 1 owing to deviation from normality, evaluated through skewness and kurtosis. The intervals are compared on the basis of the non-oscillating terms, as rigorously motivated for the binomial family by Brown et al. (2002, theorem 6). In the sequel, denotes the quantile of order 1 =2 of the standard normal distribution. Proposition 5. Let P n; be the probability measure associated with the discrete NEF-QVF F D ¹F n;./ ; 2 º, and let I r.s/ and I`.s/ be the right and left fiducial intervals obtained after observing S n D s. Then the expansion of the coverage probability of I i.s/, i D r; `, is P n;. 2 I i.s// D.1 /CA i.; /n 1=2 C.B i.; /CC i.; /. // n 1 CO.n 3=2 /; where A i.; / and B i.; / are the oscillating terms, while C i.; /. / is the non-oscillating one. The expressions of C i.; / are given in the second and third columns of Table 2.

11 Scand J Statist Fiducial and confidence distributions 11 Table 2. Expressions of the non-oscillating term of order n 1 for fiducial (C r and C l ), Wald (C W ) and Jeffreys (C J )intervals Bi(1,p) / 2 3 C r C` C W C J Po() Ne-Bi(p) C/ C3 2 4C 2C / / 2 4C 2C C3 2 4C 2C C/ C/ For the binomial distribution, D p, and for the negative-binomial, D.1 p/=p. Fig. 1. Coverages of intervals for the Poisson mean with D 0:05 and n from 2 to 50. First graph: Wald (thin line) and Jeffreys (thick line). Second graph: fiducial right (thick line) and fiducial left (thin line). As shown in Brown et al. (2003), the coverage expansions of the Wald and Jeffreys intervals have the same structure provided in proposition 5 for the fiducial intervals, and the coefficients of their non-oscillating terms are given in the last two columns of Table 2. Thus, the comparison becomes straightforward. We have chosen these two intervals as a benchmark because, in some sense, they have respectively the worst and best performances among commonly used intervals. In particular, the Wald intervals are always highly biased having the lowest negative values of the coefficients C W. Note that, excluding the case of the right fiducial intervals for the Poisson and negative-binomial families, all terms C.; / in Table 2 are negative, denoting an under-coverage of the intervals. Comparisons of the fiducial and Jeffreys intervals give different results for the three families. More specifically, for the binomial family, we have the following rankings: for p < 0:125, C`.p; / < C J.p; / < C r.p; / < 0, while for p > 0:875, the ranking is reversed. For p 2.0:125; 0:5/, wehavec`.p; / < C r.p; / < C J.p; / < 0, and, finally, for p 2.0:5; 0:875/, the ranking becomes C r.p; /<C`.p; /<C J.p; /<0. Thus, we can conclude that the Jeffreys interval is the best for intermediate values of the parameter p, while the two fiducial intervals have better coverage performances for large or small values of p. For the Poisson family, we observe a very good behaviour of the right fiducial interval, as C r.; / D 0. The ranking among intervals is uniform in the mean, and we have C`.; / < C J.; / < C r.; / D 0, with significant differences for small n; see Figure 1. Finally, for the negative-binomial model, the ranking is C`.; / < C J.; /<0<C r.; / for each. Graphs of the coverages of the intervals for the binomial and negative-binomial models are included as Figure S2 in the Supporting Information Expected length The performances of the fiducial distributions should be evaluated also in terms of their spread, and thus, we now study the parsimony in length of the various confidence intervals.

12 12 P. Veronese and E. Melilli Scand J Statist Table 3. Expressions of the coefficients, D r, D` and D J appearing in the term of order n 3=2 in (11) for fiducial and Jeffreys intervals D r D` D J Bi(1,p) p C 13 p ¹13p.1 p/ 1ºC2¹17p.1 p/º Po() Ne-Bi(p) p p/ 2 11p p/ 2 2p 2 C p/ p/2 2 2.p 2 C11pC1/ 26p 36.1 p/ 2 For the binomial distribution, D p, and for the negative-binomial, D.1 p/=p. Proposition 6. Let F D¹F n;./ ; 2 º be a discrete NEF with QVF V./ and denote by L i.s n / the random length of the interval I i.s n /, i D r; `, defined in proposition 5. Then, we have E.L i.s n // D 2 V./ 1=2 n 1=2 C Œ.; / C D i./ V./ 1=2 n 3=2 C O.n 2 /: (11) The expressions of the.; / and D i./ are given in Table 3. The expansions of the expected lengths of the Wald and Jeffreys intervals, I W and I J say, have the same form (11) as proved in Brown et al. (2003). From their results, it is easy to see that for I W, in each family, we have W.; / C D W./ D 1=4, while for I J,wehave J.p; / D.p; /, with the specific expressions of D J./ s given in the fourth column of Table 3. It follows, by a direct computation, that for the Poisson, the negative-binomial and the binomial with p < 0:5 the ranking from the shortest to the longest is I W, I`, I J, I r, while for the binomial with p > 0:5, the ranking becomes I W, I r, I J, I`. The interval I W is the shortest, but it is the worst in terms of coverage. The interval I J always has an intermediate position between I r and I` because D J./ coincides with the arithmetic mean of D r./ and D`./. This is not surprising, recalling that the asymptotic relationship of the mixture H n;s M D Hn;s C Hǹ;s =2 with the Jeffreys posterior and observing that the endpoints of the fiducial interval corresponding to Hn;s M are the means of those of I r and I`. This confirms that Hn;s M and Hn;s A are good choices as fiducial distributions for the discrete case. 6. Discussion and future directions In this paper, we have constructed a fiducial/confidence distribution for continuous and discrete real NEFs, which has nice properties. It provides a distribution for a parameter of the NEF without resorting to the Bayesian paradigm, but, at the same time, it is strongly connected with the posterior obtained with the Jeffreys prior, which represents the standard objective Bayesian choice for this case. A generalization to a multidimensional parameter should be the next step. Unfortunately, a direct extension of theorem 1 to a NEF with a vector-valued is not possible because 1 F n;.s/ fails to be a multivariate cumulative distribution function for. However, a possible way to overcome this problem is sketched later, and it will be studied in details in a future paper. Consider the normal model with both and 2 unknown. The sufficient statistics XN and S 2 D P n id1 Xi XN 2 =n are independent, and thus, we have F NX;S 2 ; Nx;s 2 X D F N. Nx/F S 2.s 2 /. 2 ; 2 2 In this case, we can construct the fiducial density for ; 2 as the product of the fiducial density for given 2 and that for 2 obtained applying (3) to the two marginal distributions of XN and S 2. It follows that the fiducial density of.; 2 / is a normal-inverse Gamma, with given 2 and 2 distributed according to a N Nx; 2 =n and an In-Ga.n=2;.n 1/s 2 =2/, respectively. This is the distribution proposed by Fisher (1935) and coincides with the posterior obtained through the reference prior.; 2 / / 2. Notice that this prior differs from that obtained

13 Scand J Statist Fiducial and confidence distributions 13 by the Jeffreys rule based on the information matrix, which is known to be unsatisfactory in the multidimensional case. The normal example rests on the specific fact that XN and S 2 are independent. However, in a general context, one can replicate a similar situation writing a joint distribution as a product of univariate conditional distributions. Consonni & Veronese (2001) defined conditionally reducible NEFs and show that they can be written as a product of conditional univariate NEFs, each parameterized with a specific likelihood-independent parameter, i say. Thus, in principle, it is possible to construct a fiducial univariate distribution for each i, then to derive their joint distribution as a product and finally to obtain the fiducial distribution for an arbitrary parameter via the change-of-variable rule. We conjecture that the strong relationship existing between fiducial distributions and the Jeffreys posteriors proved in this paper also holds in the multivariate case, with the Jeffreys priors replaced by reference priors using the results in Consonni et al. (2004). Acknowledgements This research was supported by grants from Bocconi University. We greatly appreciate the constructive comments from the editor and the two reviewers, which significantly improved the article. We also thank Michele Guindani for a discussions on topics in Section 3. References Barndorff-Nielsen, O. (1978). Information and exponential families in statistical theory, Wiley, Chichester. Brown, L. D., Cai, T. T. & DasGupta, A. (2002). Confidence intervals for a binomial proportion and asymptotic expansions. Ann. Statist. 30, Brown, L. D., Cai, T. T. & DasGupta, A. (2003). Interval estimation in exponential family. Statist. Sinica 13, Consonni, G. & Veronese, P. (2001). Conditionally reducible natural exponential families and enriched conjugate priors. Scand. J. Statist. 28, Consonni, G., Veronese, P. & Gutiérrez-Peña, E. (2004). Reference priors for exponential families with simple quadratic variance function. J. Multivariate Anal. 88, Datta, G. S. & Mukerjee, R. (2004). Probability matching priors: higher order asymptotics (lecture notes in statistics), Springer, New York. Dawid, A. P. & Stone, M. (1982). The functional-model basis of fiducial inference. Ann. Statist. 10, Efron, B. (1998). R.A. Fisher in the 21st century. Statist. Sci. 13, Fisher, R. A. (1935). The fiducial argument in statistical inference. Ann. Eugenic. VI, Fisher, R. A. (1973). Statistical methods and scientific inference, Hafner Press, New York. Fraser, D. A. S. (1961). On fiducial inference. Ann. Math. Statist. 32, Fraser, D. A. S. (1968). The structure of inference, Wiley, New York. Ghosh, J. K., Delampady, M. & Samanta, T. (2006). An introduction to Bayesian analysis, Springer, New York. Gutiérrez-Peña, E. & Smith, A. F. M. (1997). Exponential and Bayesian conjugate families: review and extensions (with discussion). TEST 6, Hannig, J. (2009). On generalized fiducial inference. Statist. Sinica 19, Hannig, J. & Iyer, H. (2008). Fiducial intervals for variance components in an unbalanced two-component normal mixed linear model. J. Amer. Statist. Assoc. 103, Hannig, J., Iyer, H. K. & Wang, C. M. (2007). Fiducial approach to uncertainty assessment accounting for error due to instrument resolution. Metrologia 44, Hannig, J. & Xie, M. (2012). A note on Dempster Shafer recombinations of confidence distributions. Electron. J. Stat. 6, Johnson, R. A. (1970). Asymptotic expansions associated with posterior distributions. Ann. Math. Statist. 41, Lindley, D. V. (1958). Fiducial distributions and Bayes theorem. J. R. Stat. Soc. Ser. B 20,

14 14 P. Veronese and E. Melilli Scand J Statist Martin, R. & Liu, C. (2013). Inferential models: a framework for prior-free posterior probabilistic inference. J. Amer. Statist. Assoc. 108, Morris, C. N. (1982). Natural exponential families with quadratic variance function. Ann. Statist. 10, Pedersen, J. G. (1978). Fiducial inference. International Statistical Review 46, Petrone, S. & Veronese, P. (2010). Feller operators and mixture priors in Bayesian nonparametrics. Statist. Sinica 20, Schweder, T. & Hjort, N. L. (2002). Confidence and likelihood. Scand. J. Stat. 29, Singh, K., Xie, M. & Strawderman, M. (2005). Combining information through confidence distribution. Ann. Statist. 33, Taraldsen, G. & Lindqvist, B. H. (2013). Fiducial theory and optimal inference. Ann. Statist. 41, Wandler, D. & Hannig, J. (2012). A fiducial approach to multiple comparisons. J. Statist. Plann. Inference 142, Welch, B.L. & Peers, H. W. (1963). On formulae for confidence points based on integrals of weighted likelihoods. J. R. Stat. Soc. Ser. B 25, Xie, M. & Singh, K. (2013). Confidence distribution, the frequentist distribution estimator of a parameter: areview.int. Stat. Rev. 81, Zabell, S. L. (1992). R.A. Fisher and the fiducial argument. Statist. Sci. 7, Received November 2013, in final form July 2014 Piero Veronese, Department of Decision Sciences, Bocconi University, via Roentgen 1, Milan, Italy. piero.veronese@unibocconi.it Supporting information Additional supporting information may be found in the online version of this article at the publisher s website.