Lorenzo Camponovo, Taisuke Otsu On Bartlett correctability of empirical likelihood in generalized power divergence family

Size: px

Start display at page:

Download "Lorenzo Camponovo, Taisuke Otsu On Bartlett correctability of empirical likelihood in generalized power divergence family"

Ira Strickland
5 years ago
Views:

1 Lorezo Campoovo, Taisuke Otsu O Bartlett correctability of empirical likelihood i geeralized power divergece family Article Accepted versio) Refereed) Origial citatio: Campoovo, Lorezo ad Otsu, Taisuke 2014) O Bartlett correctability of empirical likelihood i geeralized power divergece family. Statistics & Probability Letters, 86 1). pp ISSN DOI: /j.spl Reuse of this item is permitted through licesig uder the Creative Commos: 2013 Elsevier B.V. CC BY-NC-ND 4.0 This versio available at: Available i LSE Research Olie: April 2016 LSE has developed LSE Research Olie so that users may access research output of the School. Copyright ad Moral Rights for the papers o this site are retaied by the idividual authors ad/or other copyright owers. You may freely distribute the URL of the LSE Research Olie website.

2 O Bartlett correctability of empirical likelihood i geeralized power divergece family Lorezo Campoovo Uiversity of St.Galle Taisuke Otsu Lodo School of Ecoomics December 2013 Abstract Baggerly 1998) showed that empirical likelihood is the oly member i the Cressie-Read power divergece family to be Bartlett correctable. This paper stregthes Baggerly s result by showig that i a geeralized class of the power divergece family, which icludes the Cressie-Read family ad other oparametric likelihood such as Scheach s 2005, 2007) expoetially tilted empirical likelihood, empirical likelihood is still the oly member to be Bartlett correctable. 1 Itroductio Sice Owe 1988), empirical likelihood has bee used as a device to costruct oparametric likelihood for umerous statistical problems ad models as surveyed by Owe 2001). I spite of its oparametric costructio based o observed data poits, empirical likelihood shares similar properties to parametric likelihood. For example, the empirical likelihood ratio statistic obeys the chi-squared limitig distributio, so-called Wilks pheomeo. Aother distiguishig feature of empirical likelihood is that it admits Bartlett correctio, a secod-order refiemet based o a mea adjustmet. This poit was first made by DiCiccio, Hall ad Romao 1991) ad exteded to other cotexts, such as quatiles Che ad Hall, 1993), time series models Kitamura, 1997; Moti, 1997), local liear smoothers Che ad Qi, 2001), amog others. Also Bartlett correctability has bee studied for other costructios of oparametric likelihood. Jig ad Wood 1996) showed that expoetial tiltig or empirical etropy) likelihood is ot Bartlett correctable. Corcora 1998) costructed some Bartlett correctable oparametric likelihood based o a Taylor expasio of empirical likelihood. Baggerly 1998) stregtheed Jig ad Wood s 1996) result by showig that empirical likelihood is the oly member i the Cressie ad Read 1984) power divergece family to be Bartlett correctable. The authors would like to thak Elvezio Rochetti for helpful commets. lorezo.campoovo@uisg.ch. Address: Bodastrasse 6, 9000 St.Galle, Switzerlad. t.otsu@lse.ac.uk. Address: Houghto Street, Lodo WC2A 2AE, UK. 1

3 The Cressie-Read type oparametric likelihood is computed by choosig a tuig costat to defie both the shape of the criterio fuctio ad the form of weights allocated to data poits. Scheach 2005, 2007) suggested to choose differet tuig costats for the shape of the criterio ad the form of weights, ad proposed a more geeral class of oparametric likelihood. I particular, Scheach 2005) showed that expoetially tilted empirical likelihood where the criterio is log-likelihood but the weights are computed by expoetial tiltig) ca emerge as a valid likelihood fuctio for Bayesia iferece by a limitig argumet. Also Scheach 2007) argued that whe geeralized estimatig equatios are misspecified, the poit estimator based expoetially tilted empirical likelihood shows some robustess compared to the oe based o empirical likelihood. Give this backgroud, it is of iterest to exted Baggerly s 1998) aalysis to accommodate such ew likelihood costructios ad to study their Bartlett correctability. I this paper, we cofirm that i a geeralized class of the power divergece family cotaiig two tuig costats, empirical likelihood is still the oly member to be Bartlett correctable. This result ot oly icludes Baggerly s 1998) result as a special case, but also implies that Scheach s 2005, 2007) expoetially tilted empirical likelihood is ot Bartlett correctable. Techically we follow a covetioal approach based o the Edgeworth expasio DiCiccio, Hall ad Romao, 1991). We focus o characterizig the third ad fourth order joit cumulats of the siged root of the test statistic based o the geeralized power divergece family, ad show that those cumulats vaish at sufficietly fast rates oly whe we employ the empirical likelihood statistic. We illustrate the theoretical fidigs by a small simulatio study, which idicates that the empirical likelihood statistic with Bartlett correctio has better coverage properties tha other statistics. 2 Geeralized power divergece family We begi by itroducig the geeralized power divergece statistic. Cosider a scalar radom variable X draw from a ukow distributio F 0 with mea µ 0. Followig Owe 1988), the log-empirical likelihood ratio statistic for the mea is writte as l EL µ 0 ) = 2 max p 1,...,p log p i ), subject to p i = 1, p i X i = µ 0. It is kow that uder suitable regularity coditios the statistic l EL µ 0 ) coverges i distributio to the χ 2 1 distributio Owe, 1988) ad admits Bartlett correctio, which yields a cofidece iterval with coverage error of order O 2) DiCiccio, Hall ad Romao, 1991). Baggerly 1998) adapted the Cressie ad Read 1984) power divergece family for goodess-of-fit to the preset cotext ad cosidered the test statistic i the form of l γ µ 0 ) = 2 mi p 1,...,p γ γ + 1) { } p i ) γ+1 1, subject to p i = 1, p i X i = µ 0, 1) 2

4 if γ 1, 0, otherwise l 1 µ 0 ) = mi p1,...,p 2 log p i) ad l 0 µ 0 ) = mi p1,...,p 2 p i log p i ). Here γ R is a user-specified tuig costat. The empirical likelihood ratio statistic l EL µ 0 ) correspods to the case of γ = 1. The case of γ = 0 is ofte called the expoetial tiltig or empirical etropy statistic. Other popular choices for γ iclude the Neyma s modified χ 2 γ = 1), Helliger or Freema-Tukey γ = 1 2 ), ad Pearso s χ2 γ = 2). Baggerly 1998) showed that the power divergece statistic l γ µ 0 ) coverges i distributio to the χ 2 1 distributio for ay γ, ad that l γ µ 0 ) is Bartlett correctable oly for the case of γ = 1, the empirical likelihood ratio statistic. As Baggerly 1998) argued, a key isight of lack of) Bartlett correctability is that the third ad fourth order cumulats of the siged root of l γ µ 0 ) do ot vaish at sufficietly fast rates whe γ 1. From differet perspectives, Scheach 2005, 2007) itroduced the expoetially tilted empirical likelihood statistic l ET EL µ 0 ) = 2 log p ET,i ), i.e., the criterio fuctio is defied by l γ µ 0 ) with γ = 1, where p ET,1,... p ET, solve the miimizatio problem of l γ µ 0 ) with γ = 0, mi p 1,...,p p i log p i ), subject to p i = 1, p i X i = µ 0. Scheach 2007) cosidered geeralized estimatig equatios ad studied asymptotic properties of a poit estimator based o this statistic. Also Scheach 2005) argued that the fuctio l ET EL µ) ca be iterpreted as a valid likelihood fuctio for Bayesia iferece. It should be oted that the statistic l ET EL µ 0 ) does ot belog to the power divergece family 1). Therefore, Bartlett correctability of the statistic l ET EL µ 0 ) is a ope questio. I order to address this issue, we geeralize the power divergece statistic as follows l µ 0 ) = 2 γ γ + 1) if γ 1, 0, otherwise l 1,φ µ 0 ) = mi p1,...,p 2 log p φ,i) ad l 0,φ µ 0 ) = mi p1,...,p 2 p φ,i log p φ,i ), where p φ,1,... p φ, solve { } p φ,i ) γ+1 1, 2) p i = 1, p i X i = µ 0. 3) mi l φ µ 0 ), p 1,...,p subject to Here γ, φ R are user-specified tuig costats. Note that the shape of the criterio fuctio i 2) is give by l γ µ 0 ) but the probability weights {p φ,1,..., p φ, } are computed by l φ µ 0 ). If γ = φ, this statistic reduces to the power divergece statistic l γ µ 0 ). Also this statistic covers the expoetially tilted empirical likelihood statistic l ET EL µ 0 ) whe γ = 1 ad φ = 0. By adaptig the argumet i Baggerly 1998) ad Scheach 2005), we ca see that the statistic l µ 0 ) coverges i distributio to the χ 2 1 distributio uder the same coditios i Baggerly 1998, Theorem 1). The goal of this paper is to study Bartlett correctability of the geeralized statistic l µ 0 ). 3

5 3 Bartlett correctability To ivestigate Bartlett correctability of the geeralized power divergece statistic l µ 0 ), we follow the covetioal recipe of DiCiccio, Hall ad Romao 1991) ad Baggerly 1998), amog others. I particular, we first derive the siged root of the statistic l µ 0 ) based o a dual problem of the miimizatio i 3), ad the evaluate the third ad fourth order cumulats of the siged root. Based o these cumulat expressios, we verify for which values of γ ad φ these cumulats vaish at sufficietly fast rates to admit Bartlett correctio. To simplify the presetatio, we focus o the case where X i is scalar ad stadardized as µ 0 = E [X i ] = 0 ad V ar X i ) = 1. Although the presetatio ad techical argumet become more complicated, a similar result holds for the case where X i is a vector ad the parameter of iterest is a smooth fuctio of the mea of X i. Hereafter we oly preset the mai result. Techical details for the derivatios are available from the authors upo request. We use the followig defiitios α j = E [ ] X j i, A j = 1 X j i α j, for j = 1, 2,.... Note that α 1 = E [X i ] = 0, α 2 = V ar X i ) = 1, ad A j = O p 1/2 ) for ay j as far as sufficietly higher order momets exist. First, we fid the siged root of l µ 0 ) with µ 0 = 0. By applyig the Lagrage multiplier method, the solutio of 3) is p φ,i = s + tx i) 1 φ, for φ 0 ad p φ,i = 1 s exp tx i) for φ = 0, where s ad t solve s + tx i ) 1 1 φ = 1, 1 + s + tx i ) 1 φ X i = 0, 4) for φ 0 ad solve 1 s exp tx i) = 1 ad 1 s exp tx i) X i = 0 for φ = 0. From Baggerly 1998), we ca see that s = O p 1 ) ad t = O p 1/2 ). Expasios of 4) ad repeated substitutios yield expasios of s ad t as follows s = 1 2 φ 1 + φ) A φ 1 + φ) 1 φ) α 3A φ 1 + φ) A2 1A 2 ad φ 1 + φ) A2 1A φ 1 φ) 1 + φ) α 3A 3 1A φ 1 φ) 1 + φ) A3 1A {1 8 φ + φ) φ) φ) α3 2 1 } 3 1 φ) 1 + φ) 1 2φ) α 4 A 41 + O p 5/2). t = φa φ 1 φ) α 3A φa 1 A 2 φa 1 A φ 1 φ) α 3A 2 1A φ 1 φ) A2 1A 3 1 {φ 2 φ 1 + φ) + 1 φ) 2 α3 2 1 } 3 1 φ) 1 2φ) α 4 A O p 2 ). 4

6 By isertig these expressios to a expasio of l µ 0 ), 1 l µ 0 ) = A γ) α 3A 3 1 A 2 1A 2 + A 2 1A γ) α 3 A 3 1A γ) A3 1A 3 { γ 2 + γφ ) 2 φ γ 2 + γφ ) 2 φ2 α O p 5/2). Therefore, the siged root of the statistic 1 l µ 0 ) is obtaied as where 1 l µ 0 ) = R 1) = A 1, R 2) = 1 2 A 1A γ) α 3A 2 1, R 3) = 3 8 A 1A γ) α 3A 2 1A γ) A2 1A 3 { γ 4 + γφ ) 4 φ γ 4 + γ2 12 γφ 2 + φ2 4 ) R 1) 2 + R2) + R3) + Op 5/2), 1 9 2γ 9 γ γφ 4 φ2 8 ) α 4 } A 4 1 ) α3 2 γ8 γ γφ ) } 4 + φ2 α 4 A Note that R j) = O p j/2 ) for j = 1, 2, 3. We ca cofirm that for the empirical likelihood case i.e., φ = 1 ad γ = 1), this siged root expressio coicides with the oe i DiCiccio, Hall ad Romao 1991). Next, to ivestigate Bartlett correctability of the geeralized power divergece statistic, we evaluate the third ad fourth order cumulats deoted by κ 3 γ, φ) ad κ 4 γ, φ), respectively) of the siged root R = R 1) + R2) + R3). I particular, if the cumulats satisfy κ 3 γ, φ) = O 3), κ 4 γ, φ) = O 4), 5) the the covetioal argumet based o the Edgeworth expasio guaratees Bartlett correctability of the statistic 1 l µ 0 ). After legthy calculatios, the third order cumulat is obtaied as [ ] [ ) ] [ κ 3 γ, φ) = E R 1) + 3E R 1) 2 ) ] 2) R 3E R 1) 2 [ ] E R 2) + O 3) = 1 { γ) α 3 E [ A 4 ] [ ]) } 1 E A O 3). It is iterestig to ote that the third order cumulat κ 3 γ, φ) does ot deped o the tuig costat φ. From this expressio, we ca see that the first requiremet i 5) is satisfied oly whe γ = 1. Therefore, to evaluate the fourth order cumulat, we focus o the case of γ = 1. To this ed, it is useful to ote that for γ = 1, it holds R 1) 1,φ = R1) 1, 1, R2) 1,φ = R2) 1, 1, R3) 1,φ = R3) 1, φ + 1)2 1 + α 2 3 α 4 ) A

7 By utilizig these relatioships, the fourth order cumulat κ 4 γ, φ) with γ = 1 is obtaied as [ ) ] [ κ 4 1, φ) = 4E R 1) 3 ) ] 3) 1, 1 R 1,φ 12E R 1) 2 [ ] 1, 1 E R 1) 1, 1 R3) 1,φ +terms ivolvig momets of R 1) 1, 1ad R2) 1, 1 oly) + O 4) = φ)2 1 + α 2 3 α 4 ) 4E [ A 6 1 ] 12E [ A 2 1 ] E [ A 4 1 ]) + O 4 ). From this expressio, we ca see that the secod requiremet i 5) is satisfied oly whe φ = 1. Therefore, i the class of the geeralized power divergece statistic l µ 0 ) cosidered i this paper, oly empirical likelihood i.e., the case of φ = 1 ad γ = 1) is Bartlett correctable. This result also implies that the expoetially tilted empirical likelihood statistic i.e., the case of γ = 1 ad φ = 0) cosidered by Scheach 2005, 2007) is ot Bartlett correctable. 4 Discussios It is iterestig to ivestigate other properties of the geeralized statistic l µ 0 ) for differet values of γ ad φ. For example, Baggerly 1998) argued that the weight p φ,i give by 3) ca be egative whe φ > 0, ad Scheach 2007, p. 641) cojectured that lack of -cosistecy of the poit estimator uder misspecified geeralized estimatig equatios ca occur for ay φ 0. Also, if some weight p φ,i takes zero, the statistic l µ 0 ) diverges whe γ 1 but is fiite whe γ 0. From a practical poit of view, it should be oted that the miimizatio i 3) has a explicit solutio whe φ = 1. This paper shows that i the class of geeralized power divergece family, oly empirical likelihood φ = γ = 1) admits Bartlett correctio, which yields a cofidece iterval with coverage error of order O 2). This result also cofirms the fidig of Jig ad Wood 1996), expoetial tiltig likelihood φ = γ = 0) is ot Bartlett correctable. However, recet research has show a attractive multiplicative feature of the coverage error of the expoetial tiltig-based cofidece iterval. particular, Ma ad Rochetti 2011) show that i measuremet error models, the coverage error of the expoetial tiltig-based cofidece iterval takes the form of { 1 F χ 2 ) } O p 1 ), where F χ 2 ) is a distributio fuctio of some χ 2 distributio. As poited out i Ma ad Rochetti 2011), sice the term { 1 F χ 2 ) } is ofte very small i hypothesis testig, the relative error { 1 F χ 2 ) } O p 1 ) may be potetially more meaigful tha the absolute error. It is iterestig to exted this study to the geeralized power divergece family cosidered i this paper, ad determie the values of γ ad φ to admit the multiplicative form of the coverage error. I 5 Simulatios I this sectio, we study through Mote Carlo simulatios the accuracy of geeralized power divergece statistics. We adopt the same settigs aalyzed i DiCiccio, Hall ad Romao 1991) ad cosider testig for mea E [X] where X is geerated from a) stadard ormal, b) chi-squared with oe degree 6

8 of freedom, ad c) t-distributio with five degree of freedom. To illustrate our theoretical fidigs, we compare six test statistics: i) empirical likelihood EL), γ = φ = 1, ii) empirical likelihood with expoetial tiltig probability weights EL-ET), γ = 1 ad φ = 0, iii) expoetial tiltig ET), γ = φ = 0, iv) expoetial tiltig with empirical likelihood probability weights, ET-EL) γ = 0 ad φ = 1, v) empirical likelihood with theoretical Bartlett correctio correctio factor is computed by assumig kowledge of the populatio momets) EL-B), ad vi) empirical likelihood with empirical Bartlett correctio correctio factor is estimated by sample momets) EL-EB). Usig these statistics, we costruct cofidece itervals by employig chi-squared critical values. Table 1 reports empirical coverages of these cofidece itervals. The omial coverage probabilities are 90%, 95%, ad 99%. I Table 1, we observe that overall the geeralized power divergece statistics provide valid iferece for the mea. Ideed, the empirical coverages of all the statistics uder ivestigatio are reasoably close to the omial coverage probabilities. However, it is iterestig to ote that the test statistics with γ φ i.e., EL-ET ad ET-EL) typically do ot outperform power divergece statistics with γ = φ i.e., EL ad ET). Fially, as expected, we observe that both Bartlett corrected statistics i.e., EL-B ad EL-EB) are more accurate tha other statistics. I particular, the empirical coverages of the theoretically Bartlett corrected statistic are always very close to the omial coverage probabilities. I sum, geeralized power divergece statistics provide valid iferece for mea. However, they do ot outperform the Bartlett corrected empirical likelihood statistic. 7

9 Case a) N 0, 1) = 10 90% 95% 99% = 20 90% 95% 99% EL EL EL-ET EL-ET ET ET ET-EL ET-EL EL-B EL-B EL-EB EL-EB Case b) χ 2 1 = 20 90% 95% 99% = 40 90% 95% 99% EL EL EL-ET EL-ET ET ET ET-EL ET-EL EL-B EL-B EL-EB EL-EB Case c) t 5 = 15 90% 95% 99% = 30 90% 95% 99% EL EL EL-ET EL-ET ET ET ET-EL ET-EL EL-B EL-B EL-EB EL-EB Table 1: Empirical Coverages. We report empirical coverages for i) empirical likelihood EL), ii) empirical likelihood with expoetial tiltig probability weights EL-ET), iii) expoetial tiltig ET), iv) expoetial tiltig with empirical likelihood probability weights ET-EL), v) empirical likelihood with theoretical Bartlett correctio EL-B), ad vi) empirical likelihood with empirical Bartlett correctio EL-EB). The omial coverage probabilities are 90%, 95%, ad 99%. I the first pael, data are draw from the stadard ormal with sample sizes = 10 ad 20. I the secod pael, data are draw from the chi-squared distributio with oe degree of freedom with sample sizes = 20 ad 40. I the third pael, data are draw from the t-distributio with sample sizes = 15 ad 30. The umber of Mote Carlo replicatios is 10,000. 8

10 Refereces [1] Baggerly, K. A. 1998) Empirical likelihood as a goodess-of-fit measure, Biometrika, 85, [2] Che, S. X. ad P. Hall 1993) Smoothed empirical likelihood cofidece itervals for quatiles, Aals of Statistics, 21, [3] Che, S. X. ad Y. S. Qi 2001) Empirical likelihood cofidece itervals for local liear smoothers, Biometrika, 87, [4] Corcora, S. A. 1998) Bartlett adjustmet of empirical discrepacy statistics, Biometrika, 85, [5] Cressie, N. ad T. R. C. Read 1984) Multiomial goodess-of-fit tests, Joural of the Royal Statistical Society, B, 46, [6] DiCiccio, T., Hall, P. ad J. Romao 1991) Empirical likelihood is Bartlett-correctable, Aals of Statistics, 19, [7] Jig, B. Y. ad A. T. A. Wood 1996) Expoetial empirical likelihood is ot Bartlett correctable, Aals of Statistics, 24, [8] Kitamura, Y. 1997) Empirical likelihood methods with weakly depedet process, Aals of Statistics, 25, [9] Ma, Y. ad E. Rochetti 2011) Saddlepoit test i measuremet error models, Joural of the America Statistical Associatio, 106, [10] Moti, A. C. 1997) Empirical likelihood cofidece regios i time series models, Biometrika, 84, [11] Owe, A. B. 1988) Empirical likelihood ratio cofidece itervals for a sigle fuctioal, Biometrika, 75, [12] Owe, A. B. 2001) Empirical Likelihood, Chapma Hall/CRC. [13] Scheach, S. M. 2005) Bayesia expoetially tilted empirical likelihood, Biometrika, 92, [14] Scheach, S. M. 2007) Poit estimatio with expoetially tilted empirical likelihood, Aals of Statistics, 35,

ON BARTLETT CORRECTABILITY OF EMPIRICAL LIKELIHOOD IN GENERALIZED POWER DIVERGENCE FAMILY. Lorenzo Camponovo and Taisuke Otsu.

ON BARTLETT CORRECTABILITY OF EMPIRICAL LIKELIHOOD IN GENERALIZED POWER DIVERGENCE FAMILY By Lorezo Campoovo ad Taisuke Otsu October 011 COWLES FOUNDATION DISCUSSION PAPER NO. 185 COWLES FOUNDATION FOR