Joural of Mathematcs ad Statstcs (: 5-9, 4 ISSN: 549-3644 4 Scece Publcatos do:.3844/jmssp.4.5.9 Publshed Ole ( 4 (http://www.thescpub.com/jmss.toc THE EFFICIENCY OF EMPIRICAL LIKELIHOOD WITH NUISANCE PARAMETERS Xg Wag Departmet of Ecoomcs ad Face, Durham Uversty Busess School, Mll Hll Lae, Durham DH 3LB, UK Receved 4--3; Revsed 4--4; Accepted 4--7 ABSTRACT I ths study we check the asymptotc effcecy of emprcal lkelhood the presece of usace parameters combed wth augmeted momet codtos. We show that the presece of usace parameters, the asymptotc effcecy of the emprcal lkelhood estmator of the parameter of terest wll crease by addg more momet codtos, the sese of the postve semdefteess of the dfferece of formato matrces. As a by product, we pot out a ecessary codto for the asymptotc effcecy to be creased whe more momet codto are added. Keywords: Emprcal Lkelhood, Nusace Parameters Scece Publcatos. INTRODUCTION Lkelhood ferece may have some drawbacks whe estmatg a parameter of terest the presece of usace parameters. For example, (Neyma ad Scott, 948 cosdered ths problem ad foud that the maxmum lkelhood estmato could be ether cosstet or effcet wth may usace parameters. Ths study deal wth Emprcal Lkelhood (EL, whch s a oparametrc aalogue of maxmum lkelhood, the presece of usace parameters combed wth selecto of momet codtos. We show that wth the exstece of usace parameters, the asymptotc effcecy of the emprcal lkelhood estmator of the parameter of terest ca be crease by addg more momet codtos, the sese of the postve semdefteess of the dfferece of formato matrces. Partcularly, we focus o a specal case, where usace parameters oly appear some of the momet codtos. Ths case leads to a mportat result that the asymptotc effcecy ca crease wth added momet codto oly f t s ot orthogoal wth the orgal momet codtos.. MOMENT CONDITION WITH NUISANCE PARAMETERS Cosder a sequece of..d. realzatos { x } of a radom varable x from a ukow dstrbuto F, wth 5 beg the sample sze. Let θ be a p-dmesoal vector p of parameters a compact parameter space Θ R assocated wth F. Suppose that for a true value of θ whch s deoted as, { x } θ satsfes the followg momet codto Equato (: θ ( E g(x ; where, g s a m vector of real fuctos ad the expectato s take wth respect to F. We cosder the over-detfed case where m p. Ulke (Q ad Lawless, 994, we do t assume that the m fuctos of g are depedet, sce correlato betwee these fuctos plays a mportat role the aspect of asymptotc effcecy, whch we wll dscuss the followg secto. Now suppose the parameter θ ca be decomposed as θ ( β', '' wth correspodg θ ( β', ' ' where q pq β Β R, Φ R ad Θ Β Φ. If we are oly terest β but ot, the s a usace parameter the model ad we wrte the correspodg momet codto as Equato (: ( E g(x ;, For the true value β of β. The emprcal lkelhood rato statstc for ths model s Equato (3:
Xg Wag / Joural of Mathematcs ad Statstcs (: 5-9, 4 Scece Publcatos ( R( β, log + λ'g(x ; β, (3 where, λ s a m vector of Lagraga multplers, whch s a cotuous dfferetable fucto of ( β', '' (see, e.g., (Q ad Lawless, 994 ad s determed by Equato (4: ( ( g x ;, (4 + λ'g x ; β, To smplfy otatos, let: ( ( ɵ θ θ ( θ ( θ ( x ; θ Ω ( θ ( θ g x ; g,g g G E, E g x; g x; ' θ Lke ordary parametrc lkelhood, emprcal lkelhood deals wth usace parameter by proflg out ø (see, e.g., secto 3.5 of (Owe,. Let ɶ ɶ ( β be the mmzer of R(β,wth respect to. The profle log-emprcal lkelhood rato for β s Equato (5: ( m (, R β R (5 Φ Ad EL estmator for β s Equato (6: ( β arg m R β (6 β Β Assumpto θ (β, solves E[g(x; θ] uquely, or equvaletly, both β ad are strogly detfed. Remark Ths codto combed wth m p makes the parameter well detfed. I the study of (Stock ad Wrght,, they cosdered the problem of weak detfcato of the parameter, by assumg that the subvector β of θ s completely detfed, but s ot, the sese that the populato momet fucto s steep β aroud β but s early flat α. Ths dea provdes us a framework to aalyss problems mxed wth usace parameters, weak detfcato ad partal detfcato (Phllps, 989. Assumpto (a θ t (Θ. (b Ω s postve defte ad osgular (c. g(x, θ s twce cotuously 6 dfferetable a eghborhood of θ ad G s of full rak p (d. (x,θ 3, (x,θ/ θ ad g(x,θ/ θ θ are all bouded from above. We derve the propertes of the EL estmator of β the ext theorem. Theorem Uder assumpto -: Proof ( βɶ β d ( N,V β ɶ ɶ Vβ E + ɶ Ω E +, ɶ ( ( ɶ ɶ Ω Eg, β g, β The proof s smlar to (Q ad Lawless, 994. Dfferetate R(β wth respect to β ad λ respectvely gves Equato (7 ad 8: ( ( ɶ ɶ R( β ɶ β, β, + λ ( ɶ ɶ + λ g β, (7 ɶ R( β g ( β, (8 λ ɶ + λ g β, ( Deote the rght had sde of (7 ad (8 as Q (β,λ ad Q (β, λ respectvely. Sce ˆβ ad ˆλ maxmze ( ( R β,q β, λ ad frst order Taylor expaso aroud (β, gves: ( Q β, λ ( ( ( Q β, Q β, Q ( β, + βɶ β ˆ + λ + op δ λ ( Q β, λ ( ( ( ( Q β, Q β, Q ( β, + βɶ β + λ + op δ λ where, δ βɶ β + λˆ So β ɶ ad ˆλ ca be solved as: (
Xg Wag / Joural of Mathematcs ad Statstcs (: 5-9, 4 Scece Publcatos ( ( λˆ Q β, + o δ p S βɶ β op( δ I S E + S.E + S Q ( β, + op ( E + SQ ( β, + op ( Q Q u S S S Q S u ( β, E( gg E + β E + β From lemma of (Q ad Lawless, 994 uder Assumpto we / Q ( β, ( / g( x, θ Op ( ad δ O p / ( Remark. So we obta: ( βɶ β S.E + ( β + ( ( β S Q, o N,V d p The structure of the asymptotc varace-covarace matrx V β s dfferet from those (Stock ad Wrght,, whch they decompose E gˆ ( θ as E gˆ ( θ -/ m (θ+m (β where m (θ volves both of the two parameters ad m (β volves β ad the true value of. Lazar ad Myklad (999 cosder hgher order propertes of ˆβ through Edgeworth expaso of R(β, They fd that ˆβ may ot acheve hgher order accuracy whch ca be obtaed by ordary lkelhood the presece of usace parameters, also they show that the 7 emprcal lkelhood rato statstc does ot admt Bartlett correcto, ulke the case wthout usace parameters. 3. MORE MOMENT CONDITIONS Now we focus o the asymptotc effcecy of ˆβ whe there are more momet codto beg added. Suppose based o model (, we have the followg ew model by addg oe more momet dcator f (. Equato (9: ( ( g x ;, Eh( x ;, E f x ;, For more otatos we defe: ( ( h x; θ f x; θ G E,G E θ θ Ω Ω Ω E h (, h (, Ω Ω (9 I ths model, followg the setup the prevous secto, the parameter vector θ ( ca be detfed by ( aloe ad ow we are terested whether the covarace matrx V β ca be mproved wth extra, formato gve by f. Let the estmator of β based o both g ad f deoted as β ɶ ad the correspodg covarace matrx as V β : I geeral, well establshed results have show that at least usg f wll ot be harmful,.e., t wll ot crease the asymptotc varace of θ ɶ. Ad, or wll droppg f wll decrease the asymptotc varace of the estmator, relatve to that of the estmator based o both g ad f. See, corollary of (Q ad Lawless, 994. Remark 3 A smlar ad relevat stuato may be worth meto, whch s descrbed (Newey ad Wdmejer, 5; Ha ad Phlps, 6, for stace. They assume that the umber of momet codtos s creased wth the sample sze. Thus ths case extra formato are provded by both extra data ad extra momet codtos, whle our case oly by the latter oe wth fxed sample sze. They also allow the momet codtos are weak, whle we assume both g ad f are strog as dcated assumpto. Estmato uder may weak momet codtos s also dscussed by (Adrews ad Stock, 5.
Xg Wag / Joural of Mathematcs ad Statstcs (: 5-9, 4 Proposto The asymptotc effcecy of EL estmator of β ca be creased by addg more momet codtos. Proof Sce we ca always block the compoet of the vector of the momet fucto, for smplcty ad wthout loss of geeralty, we assume that both g ad f are of dmesoal oe. For coveece let: g g f f E E,E + + E Scece Publcatos The verse of V β, or the formato matrx of β wth both g ad f s Equato (: ( [ ] E E hh E Ω Ω E E E Ω Ω E Sce wthout f, the formato o β s: We have: ( E E gg E E S E ( [ ] ( [ ] E E hh E E E gg E ' E S S Ω S S E + E Ω S S E + E S S Ω E + E Ω E ( E SS E ( E SS E Ω ( Whch s postve semdefte, provdg E(gg s p.d as Assumpto dcates. Example Suppose we have a sequece of..d observatos of uvarate radom varable x,,x. Let E(x µ ad var (x σ Thus we have the followg two momet codtos Equato ( ad : ( β ( ( E g x; E x u 8 ( ( ( σ ( E f x;, E x u Ad ow we are oly terested the estmato of µ. The emprcal lkelhood estmator of µ s Equato (3: Ad: x µ µ ˆ arg m log + t µ ( x µ σˆ 3 ( E( x µ (( x µ σ g f E( gg E( gf Var( µ ˆ Ω E ( fg E( ff f σ σ E Notce that wthout g, Var(u ˆ equals σ. (3 f I the above example, we otce that E ad ths feature smplfes the calculato dramatcally. So we cosder the followg more specal model, where g does ot have usace parameter, but f has a usace parameter oly, although t brgs some formato from the data Equato (4: ( β ( g x; Eh( x;, E f x, The gradet vector of h (4 s: The formato o β s: ( E E hh E h θ f f Ω Ω E E Ω Ω (4 f E I Ω + Ω ( Ω ΩΩΩ ΩΩ E
Xg Wag / Joural of Mathematcs ad Statstcs (: 5-9, 4 where, I s the correspodg detty matrx. Now we have Equato (5 ad 6: f f E VE Scece Publcatos ( V Ω Ω Ω Ω Ω Ω Ω Ω (5 (6 By assumpto E(gg s postve semdefte, so ( Ω Ω Ω Ω s also p.s.d ad so s V. Thus we see that f provde extra formato for β. However, f (, E( gf Ω,V,so I I. So we have the followg proposto. Proposto Addtoal momet codtos whch cotas oly usace parameters wll provde extra formato o the parameter of terest oly f they are correlated to the orgal momet codtos. Remark 4 Whether g ad f are correlated s a testable codto. Sce E[g(x,β ] E[f(x, ], to test the correlato of g ad f t s equvalet to test the followg addtoal momet codto Equato (7: ( ( ( E x;, Eg x, f x, ρ β β (7 Ad ths ca be doe by stadard EL test procedure. 4. CONCLUSION I ths study we have dscussed the effcecy of the EL estmator the presece of usace parameters, va stadard asymptotc method. We are partcularly terested whether the asymptotc effcecy of the parameter of terest ca be mproved by addg more momet codtos. We foud that a ecessary codto for augmeted momet codto to be useful to mprove the asymptotc effcecy s that t s correlated to the orgal momet codto. It s worth metog that here we corporate more momet codtos wth sample sze beg fxed, whle researchers lke (Newey ad Wdmejer, 5; Ha ad Phlps, 9 6 cosder creasg the umber of momet codtos brought by creasg sample sze. For future research, t would be worth checkg the effcecy of the EL test wth usace parameters, because t wll be a exteso to the results foud (Wag, 3, where the large devato effcecy of the EL test wth weakly depedet data s establshed. 5. REFERENCES Adrews, D. W. K. ad J. H.Stock, 5. Iferece wth Weak Istrumets. NBER Research. Ha, C. ad P.C.B. Phlps, 6. GMM wth may momet codtos. Ecoometrca, 74: 47-9. DOI:./j.468-6.6.65.x Lazar, N.A. ad P.A. Myklad, 999. Emprcal lkelhood the presece of usace parameters. Bometrka, 86: 3-. do:.93/bomet/86..3 Newey, W.K. ad F. Wdmejer, 5. GMM wth May Weak Momet Codtos. Cetre for Mcrodata Methods. Neyma, J. ad E.L. Scott, 948. Cosstet estmates based o partally cosstet. Observ. Eco., 6: - 3. Owe, A.B.,. Emprcal Lkelhood. st Ed., CRC Press, Lodo, ISBN-: 43657, pp: 34. Phllps, P.C.B., 989. Partally detfed ecoometrc models. Ecoometr. Theory, 5: 8-4. DOI:.7/S66466648 Q, J ad J. Lawless, 994. Emprcal lkelhood ad geeral estmatg equatos. A. Statst., : 3-35. DOI:.4/aos/763537 Stock, J.H. ad J.H. Wrght,. GMM wth weak detfcato. Ecoometrca, 68: 55.96. Wag, X., 3. The rate fucto of s-mxg process ad ts applcato evaluatg emprcal lkelhood tests. J. Math. Stat., 9: 334-338. DOI:.3844/jmssp.3.334.338