Citation Statistica Sinica, 2000, v. 10 n. 2, p

Transcription

1 Titl Nonprmtri onfin intrvls s on xtrm ootstrp prntils uthor(s) L, SMS ittion Sttisti Sini, 2000, v. 10 n. 2, p Issu Dt 2000 URL Rights This work is lins unr rtiv ommons ttriution- Nonommril-NoDrivtivs 4.0 Intrntionl Lins.

2 Sttisti Sini 10(2000), NONPRMTRI ONFIDN INTRVLS SD ON XTRM OOTSTRP PRNTILS Stphn M. S. L Th Univrsity of Hong Kong strt: Mont rlo pproximtion of stnr ootstrp onfin intrvls rlis on th rwing of lrg numr, sy, of ootstrp rsmpls. onvntionl hoi of is oftn m on th orr of 1,000. Whil this hoi my prov to mor thn suffiint for som ss, it my fr from qut for othrs. nw pproh is suggst to onstrut onfin intrvls s on xtrm ootstrp prntils n n ptiv hoi of. It onomizs on th omputtionl ffort in prolm-spifi fshion, yiling stl onfin intrvls of stisftory ovrg ury. Ky wors n phrss: ootstrp, onfin limit, ovrg, gworth xpnsion, qui-til, xtrm prntil, Mont rlo, nonovrg, smooth funtion mol. 1. Introution Th ootstrp mtho hs n stui xtnsivly in th ontxt of onstruting nonprmtri onfin intrvls for rl prmtr. In prti, th onstrution rquirs Mont rlo simultions of lrg numr of ootstrp rsmpls. Most ommon ootstrp mthos of th prntil kin riv th onfin limits from intrmit orr sttistis s on ths rsmpls. xmpls inlu th kwrs n hyri prntil mthos, th ootstrp-t mtho, is-orrt () n lrt is-orrt ( ) mthos, n th itrt ootstrp mtho. n ovrviw of th ov mthos n foun in, for xmpl, Sho n Tu (1995, h. 4). Thir prtil implmnttion is invrily sujt to Mont rlo rror u to th vilility of only finit numr of ootstrp rsmpls. On trivil ut omputtionlly intnsiv rmy for th ov limittion is to rw mor ootstrp rsmpls to ttr pproximt th tils of th ootstrp istriution. Huristi rgumnts suggst tht hoi of on th orr of 1,000 oftn suffis for ommon situtions, s fron (1987). It is thrfor importnt to intify thos situtions whih fvour suh onvntionl hoi n thos whih o not. It will shown tht muh lrgr might nssry for ss whr th smpling istriution is highly

3 476 STPHN M. S. L skw, whr th smpl siz is smll, or whr th onfin lvl is vry los to on. On th othr hn, lthough n infinit orrspons to th xt thortil ootstrp intrvl, inrsing infinitly my not lwys fftiv in yiling n urt intrvl, to sy nothing of omputtionl ost. Thr r ss whr finit n us to just th Mont rlo vrition in onstrutiv wy to ountrln th intrinsi ovrg rror of th thortil ootstrp intrvl. Motivt y th n for n onomil yt snsil hoi of, wpropos nw pproh to onstruting ootstrp onfin intrvls s on n ptiv trmintion of. It minimizs th Mont rlo ffort in prolm-spifi fshion. Th rsulting onfin intrvls hv sirl ovrg ury, stl lngth n n points. Whil stnr α lvl ootstrp onfin limit is typilly otin from th αth or (1 α)th ootstrp prntil, pproximt using fix numr of ootstrp rsmpls, our mtho lwys rivs th onfin limit from th xtrm prntils ut justs ptivly to giv th orrt ovrg. s onsqun, our pproh minimizs th numr of rsmpls nssry for hiving th sir onfin lvl. This numr is not only usully muh smllr thn th onvntionl hoi in ss whr th lttr works prftly, ut lso provis sfty msur ginst lss ni ss whr th onvntionl hoi provs to inqut. From nothr prsptiv, our mtho trts s lirtion prmtr to yil th orrt nominl ovrg rthr thn to pproximt th thortil ootstrp ssoit with n infinit. Th notion of Mont rlo simultion rror is thrfor irrlvnt in our s, whr = in ft yils 100% ovrg lmost surly. Stion 2 givs th symptoti ovrg proilitis of th xtrm ootstrp n ootstrp-t prntils. Ths proilitis srv s uppr ouns for th ovrgs of th stnr kwrs prntil mtho, whih w shll now ll th prntil mtho for rvity, n ootstrp-t intrvls. Thir sizs init th limittions of ths intrvls whn pproximt using fix numr of ootstrp rsmpls. Numril xmpls r givn to illustrt suh limittions for vrity of unrlying istriutions n nominl ovrg lvls. W mk us of th thortil rsults in Stion 2 to vlop n xtrm ootstrp mtho of onstruting onfin limits. Stion 3 tils th lgorithm of th mtho for oth on-si n two-si intrvls. In this ppr w fous for onvnin on pplitions of th mtho to th prntil n ootstrp-t onstrutions. Th mtho n, howvr, ppli to othr kins of ootstrp intrvls. Stion 4 prsnts simultion stuy whih omprs our mtho with som stnr ootstrp mthos. Stion 5 summrizs our finings n xplors possil gnrliztions of our mtho. ll thnil tils r givn in th ppnix.

4 XTRM OOTSTRP PRNTILS 477 Hll (1986) hints t th orr of mgnitu of nssry for mintining th thortil ovrg ury njoy y th ootstrp-t intrvl. ooth n Hll (1994) otin n optiml rltionship twn th numrs of innr n outr lvl ootstrp rsmpls in onstrution of th itrt ootstrp onfin intrvl s on ovrg lirtion. orrsponing rsults for itrt ootstrp intrvls s on n point lirtion r givn in ooth n Prsnll (1998). L n Young (1999) xmin th joint fft of Mont rlo n smpling rrors on ovrg ury n vlop n itrt ootstrp mtho s on n ptiv hoi of th numr of innr lvl rsmpls. 2. Thory 2.1. Nottion Lt X =(X 1,...,X n ) rnom smpl rwn from n unknown -vrit istriution F. onsir th prolm of onstruting nonprmtri onfin intrvls for rl prmtr θ. W ssum htthry n Ghosh s (1978) smooth funtion mol, unr whih θ = g(µ) for smooth funtion g n µ = (X 1 ). This mol ovrs prmtrs whih n xprss s smooth funtions of momnts of F, inluing suh ommon xmpls s mn, vrin, rtio of mns n orrltion offiint. nturl stimtor of θ is ˆθ = g( X), whr X = n 1 n i=1 X i. W first introu som nottions. Dfin, for r =1, 2,... n i j =1,...,, ( g i1 i r (x) = r /( x (i 1) x (ir) ) g(x) n κ i 1,...,i r =um X (i1),...,x (ir)), whr x (i) nots th ith omponnt of th vtor x, X is gnri rnom vtor istriut unr F,num( ) nots th umulnt. W writ g i1 i r for g i1 i r (µ), ĝ i1 i r for g i1 i r ( X), n not y ˆκ i 1,...,i r th smpl umulnts s on X. Th symptoti vrin of n 1/2 ˆθ is thn σ 2 = i,j=1 g i g j κ i,j n plug-in stimtor of σ 2 is ˆσ 2 = i,j=1 ĝ i ĝ j ˆκ i,j. W my stnriz n stuntiz ˆθ to otin, rsptivly, S = n 1/2 (ˆθ θ)/σ n T = n 1/2 (ˆθ θ)/ˆσ. Lt X =(X1,...,X n) gnri ootstrp rsmpl, otin y rnom smpling from X with rplmnt. stnr Mont rlo pproximtion to ootstrp onfin intrvls rlis on th rwing of lrg numr, sy, of ootstrp rsmpls X1,...,X. Dfin ˆθ n ˆσ to th rsptiv vlus of ˆθ n ˆσ lult from th rsmpl X inst of from X. Similrly, fin = n1/2 (ˆθ ˆθ)/ˆσ. Dnot y ˆθ (1) ˆθ () n T (1) T () th T orr sttistis of th ˆθ n th T rsptivly. Stnr ootstrp mthos mk us of suh orr sttistis to fin uppr n lowr onfin limits s wll s two-si intrvls.

5 478 STPHN M. S. L 2.2. xtrm ovrg W fous on two ommon ootstrp mthos known rsptivly s th prntil (fron (1979)) n th ootstrp-t (fron (1981)) mthos. Dnot y [ ] th intgr prt funtion. Th prntil mtho spifis th uppr n lowr α lvl onfin limits to ˆθ ([(+1)α]) n ˆθ ([(+1)(1 α)]) rsptivly. Th two-si α lvl prntil mtho onfin intrvl is thn [ ˆθ ([(+1)(1 α)/2]), ˆθ ([(+1)(1+α)/2]) ]. Th orrsponing ootstrp-t spifitions r ˆθ n 1/2ˆσT ([(+1)(1 α)]), ˆθ n 1/2ˆσT ([(+1)α]) n [ ˆθ n 1/2ˆσT ([(+1)(1+α)/2]), ˆθ n 1/2ˆσT ([(+1)(1 α)/2]) ] rsptivly. It is lr tht, givn fix, th ovrg proilitis of ths intrvls n nvr x thir ountrprts otin y sustituting xtrm prntils of th ˆθ n T.Morspifilly,whv pr ( θ ˆθ ([(+1)α]) ) pr ( θ ˆθ () ) n pr ( θ ˆθ n 1/2ˆσT ([(+1)(1 α)]) ) pr ( θ ˆθ n 1/2ˆσT (1) ), orrsponing to th uppr limits. Similr inqulitis hol for th lowr limits n two-si intrvls. Th ov uppr ouns, fin s th xtrm ovrgs n ing fr of α, init th highst possil ovrg lvls ttinl y th orrsponing ootstrp onfin limits. Thy rvl in sns th limittions of stnr ootstrp intrvls, spilly whn th sir ovrg lvl is hosn so los to on tht it xs ths xtrm ovrgs. vn th vry powrful itrt ootstrp mtho s on ovrg lirtion suffrs from th sm limittions. S rn (1987) for th itrt ootstrp pproh. Dfin 1 = g i g j g k κ i,j,k, 2 = g i g j g kl κ i,k κ j,l, 3 = g i g j g k g l κ i,j,k,l, 4 = g i g j g k g lm κ i,l κ j,k,m, 5 = g i g j g kl g mp κ i,k κ j,m κ l,p, 6 = g i g j g k g lmp κ i,l κ j,m κ k,p, whr th summtion is ovr i, j, k, l, m, p =1,...,. Lt φ th stnr norml nsity funtion. W otin symptoti xpnsions for xtrm ovrgs ssoit with th prntil mtho n ootstrp-t onfin intrvls. W ssum vliity of gworth xpnsions for th ootstrp istriutions of S n T. Hll (1992, 5.2 givs til ount of suffiint onitions for this ssumption. In short, thy rquir tht g suffiintly smooth, tht momnts up to high orr xist, n tht rmér s onition hols.

6 XTRM OOTSTRP PRNTILS 479 Proposition 1. Unr Hll s (1992, 5.2 smooth funtion mol n ssuming tht n δ n for ny >δ>0, w hv n pr (θ ˆθ () )=1 ( +1) 1 (1/6)n 1/2 1 3 σ 3 1 { 1+O( 2 ) }, (1) pr (θ ˆθ (1) )=1 ( +1) 1 +(1/6)n 1/2 1 3 σ 3 1 { 1+O( 2 ) } (2) pr (ˆθ (1) θ ˆθ () )=1 2( +1) 1 (1/36)n σ 6 2 1{ 1+O( 2 ) }, (3) whr is th positiv solution to φ( 1 )=. xtrm ovrgs givn in Proposition 1 prsri uppr ouns for th ovrgs of ootstrp prntil mtho intrvls of ny nominl ovrg lvl. In prtiulr, (1) n (2) orrspon rsptivly to ouns for th uppr n lowr onfin limits, whrs (3) ouns th ovrgs of two-si intrvls. Similr rsults r lso otin for th ootstrp-t mtho, s givn in th following proposition. Proposition 2. Unr th onitions of Proposition 1, w hv n pr (θ ˆθ n 1/2ˆσT (1) )=1 ( +1) 1 + n { 1+O( 2 ) }, (4) pr (θ ˆθ n 1/2ˆσT () )=1 ( +1) 1 + n { 1+O( 2 ) } (5) pr (ˆθ n 1/2ˆσT () θ ˆθ n 1/2ˆσT (1) )=1 2(+1) 1 +2n {1+O( 2 )}, (6) whr = σ 6 ( )( )/4 σ 4 ( )/6 n is th positiv solution to φ( 1 )=. Proofs of Propositions 1 n 2 r outlin in th ppnix. It is lr from th propositions tht highr nominl lvl α lls for iggr. Not tht th ftor 1 ounts for th skwnss of th smpling istriution of ˆθ. Itthus follows from (1) tht iggr is typilly rquir y th prntil mtho to yil urt uppr onfin limits if th smpling istriution of ˆθ hs high positiv skwnss. similr rquirmnt is lso not, oring to (2), for th lowr limits if th istriution hs high ngtiv skwnss. For th two-si prntil mtho intrvl, ig is gnrlly nssry whnvr th smpling istriution of ˆθ is highly skw, s n sn from (3). Similr rmrks my lso m for th ootstrp-t mtho in th light of Proposition 2, lthough th ftor now rfrs to mor sutl proprtis of th smpling istriution of ˆθ. Morovr, th smpl siz n my fft th nssry siz of in wy whih pns on th signs of 1 n.

7 480 STPHN M. S. L Th rsults ontin in Propositions 1 n 2 n lso viw s supplmnts to th lrg vition proprtis of ootstrp istriutions with unspifi til rs of intrst. Hr th Mont rlo ffort shoul ttr trt s lirtion prmtr of strtgi vlu, rthr thn sour of unsirl simultion rror. Our mphsis is on its onntion with th xtrm ovrg n on wys to just it in orr to yil onstrutiv pplitions. Prntil mtho, N(0, 1) ootstrp-t, N(0, 1) xtrm ovrg xtrm ovrg xtrm ovrg xtrm ovrg Prntil mtho, oul xp ootstrp-t, oul xp Uppr Lowr 2-si Uppr Lowr 2-si Uppr/lowr 2-si Uppr Lowr 2-si (thortil) (mpiril) (thortil) (mpiril) Figur 1. symptoti xtrm ovrg proilitis of prntil mtho n ootstrp-t limits for th stnr norml n oul xponntil vrins Numril xmpl W illustrt th symptoti rsults with n xmpl whr θ is tkn to th stnr norml n oul xponntil vrins rsptivly. Figur 1 plots th xtrm ovrgs ginst for n = 20. Th ovrgs r omput from xprssions (1) to (6) with th omission of th O( 2 )trms.th xis is givn on th sl, whr stisfis = φ( 1 ). Th lft pnl shows th rsults for th prntil mtho. In gnrl, it tks mor ootstrp rsmpls to hiv highr ovrgs. For oth istriutions, mor rsmpls r rquir

8 XTRM OOTSTRP PRNTILS 481 to yil th sm xtrm ovrg for th uppr limits thn for th lowr ons. Th two-si limits rquir vn mor rsmpls to o so. Rll tht th ovrgs isply hr provi uppr ouns on thos of th stnr ootstrp prntil mtho intrvls n thir itrt vrsions. Tht th uppr limits yil ovrg ouns uniformly smllr thn th lowr limits signifis th vry poor on-si ovrg ury of th stnr ootstrp prntil mtho onfin intrvl. Rsults for th ootstrp-t mtho r givn on th right pnl of Figur 1. It follows from (4) n (5) tht oth uppr n lowr limits hv th sm xtrm ovrgs to orr O(n ). This nls th ootstrp-t intrvls to hiv ttr on-si ovrg ury ompr to th prntil mtho. Not lso tht th sm yils two-si nonovrg oul tht of its on-si ountrprts. From slightly iffrnt prsptiv, th plots in Figur 1 init th miniml sizs of rquir y th prntil n ootstrp-t mthos to prou givn nominl ovrg lvl. For instn, two-si ootstrp-t intrvl typilly rquirs in th rng of 10 to 100 in orr to giv 90% onfin lvl for norml smpls, whrs muh iggr (> 1000) is n to rh th sm lvl for oul xponntil smpls. mpiril figurs, stimt from 1,600 rnom smpls, r lso plott for nominl ovrgs twn 0.8 n 1. Thy mth th symptoti rsults quit wll in gnrl, xpt tht th simult two-si figurs for th prntil mtho r onsistntly iggr in th oul xponntil s. Rsults for othr hois of n r similr, with th urvs ing mor losly pk togthr s n inrss. Th ov finingsxmplify th rstritions of finit n th importn of snsil hoi of whih is ptl to vrity of situtions. On th othr hn, siz of sustntilly smllr thn th onvntionl prsription my oftn suffiint for prouing rsonly urt onfin limits, spilly for light-til smpling istriutions n mort onfin lvls. 3. Nw pproh 3.1. Gnrl finition W propos in this stion n pproh to onstruting ootstrp onfin limits s on miniml numr of ootstrp rsmpls. It is losly rlt to stnr ootstrp intrvls, ut lwys rivs th onfin limits from th xtrm ootstrp prntils. Th numr of ootstrp rsmpls,, is just nlytilly n ptivly to prou just th right nominl ovrg lvl. Th following isussion is ntr on th prntil mtho n ootstrp-t onstrutions, lthough our pproh fins xtnsions to othr typs of ootstrp intrvls.

9 482 STPHN M. S. L Mor spifilly, our pproh trts s lirtion prmtr n pproximts its vlu from ovrg xprssions suh s (1) to (6). If, for xmpl, n α lvl onfin intrvl is rquir, w qut ths xprssions, gin with th omission of th rror trms, to α n solv for. Thsolutionmy otin ithr numrilly or grphilly using lirtion plots lik thos xhiit in Figur 1. With trmin, th onfin intrvl is thn otin from th orrsponing xtrm ootstrp prntils. Our prour thus gnrts numr of nw ootstrp onfin intrvls, th forms of whih pn on th prtiulr ovrg xprssions us for trmining. Ths intrvls r, rsptivly, I P,L =[ˆθ (1), ), I P,U =(, ˆθ () ], I P,2 =[ˆθ (1), ˆθ () ], I T,L =[ˆθ n 1/2ˆσT (), ),I T,U =(, ˆθ n 1/2ˆσT (1) ], I T,2 =[ˆθ n 1/2ˆσT (), ˆθ n 1/2ˆσT (1) ]. Not tht I P,L n I P,U giv, rsptivly, th xtrm lowr n uppr onfin limits s on th prntil mtho, with otin from (2) n (1) rsptivly, whrs I P,2 is two-si xtrm prntil mtho intrvl with givn y (3). On th othr hn, xprssions (4), (5) n (6) giv ris to xtrm intrvls s on th ootstrp-t mtho, fin oringly s I T,U, I T,L n I T,2. For two-si qui-til α lvl intrvl, on shoul solv for sprtly y quting (1) n (2) to (1 + α)/2. Dnot th two solutions y 1 n 2 rsptivly. Th intrvl thn oms [ ] I P,2 = min {ˆθ i : i =1,..., 2 }, mx {ˆθ i : i =1,..., 1 }. Th ootstrp-t mtho hs no suh nlogu, sin th solutions for otin from quting (4) n (5) to (1 + α)/2 r th sm s tht givn y quting (6) to α. This is u to th highr-orr on-si ovrg ury of th ootstrp-t ompr to th prntil mtho stimtion of 1 n Not tht 1 n gnrlly pn on unknown popultion momnts n shoul thrfor stimt in prti. Unr th smooth funtion mol, w my stimt g i1 i r n κ i 1,...,i r y thir smpl vrsions ĝ i1 i r n ˆκ i 1,...,i r rsptivly. L n Young (1995) introu quik, xt n utomti prour for omputing prtil rivtivs of g up to high orr. Thir lgorithm rquirs th usr to spify only th funtion g n no mor nlyti input is nssry. Thr xist jkknif ltrntivs to stimting 1 n. Lt X i1,...,i r not th smpl mn of th ru smpl X\{X i1,...,x ir }. Dfin, for i, j, k =1,...,nn istint, Ĵi = g( X i ) ˆθ, Ĵij = g( X i,j ) ˆθ, Ĵijk = g( X i,j,k ) ˆθ n J ij =(n 2)Ĵij (n 1)(Ĵi + Ĵj). Thn th smpl vrsions Âi of i n

10 XTRM OOTSTRP PRNTILS 483 xprss ntirly in trms of th jkknif psuo-vlus Ĵi, Ĵij, Ĵijk, oring to th following rltions tht hol symptotilly up to O p (n 1 ): n i Ĵ i 2 =ˆσ 2, n 2 i Ĵ i 3 = Â1, 2n i<j ĴiĴj J ij = Â2, n 3 i Ĵ i 4 = Â3 +3ˆσ 4, n 2 i<j ĴiĴj(Ĵi + Ĵj) J ij = Â4, n i,j k i,j ĴiĴj J ik Jjk = Â5, n 1 (n 1) 3 i<j<k ĴiĴjĴk{ (n 3)Ĵijk (n 1)(Ĵi + Ĵj + Ĵk) } = Â4 + Â6/6+nÂ2( i Ĵi)/2. (7) Hr th summtions r ovr i, j, k =1,...,n, sujt to th spifi inqulity onstrints. Th rltions (7) r in ft pplil to situtions mor gnrl thn th smooth funtion mol whr g is rpl y gnrl sttistil funtionl. For tils of jkknif stimtion thniqus s Hinkly n Wi (1984) n Tu (1992). W rmrk tht 1 is losly rlt to th lrtion onstnt â rquir y fron s (1987) mtho, oring to th xpnsion â = n 1/2 σ 3 1 /6+ O p (n 1 ). S Hll (1992, 3.1 for th lttr rsult n fron (1987) for wys to lult â. It n lso shown tht ˆp 1 ( 1 )+ˆq 1 ( 1 )=(1/6)σ {1+O( 2 )}, whr ˆp 1 n ˆq 1 r polynomils in th two-trm gworth xpnsions for th ootstrp istriutions of S n T rsptivly: s Hll (1992, 3.3. Polnsky n Shuny (1997) n Polnsky (1997) suggst vrious mthos to stimt ˆp 1 +ˆq lgorithm W giv th lgorithm for onstruting I P,2 of nominl ovrg lvl α. lgorithms for th othr intrvls follow ftr ovious moifitions. Lt Â1 onsistnt stimtor of 1. Stp 1. Solv ( +1) 1 + n 1/2 1 3ˆσ 3 Â 1 /6=(1 α)/2 ( +1) 1 n 1/2 1 3ˆσ 3 Â 1 /6=(1 α)/2, for n not th solutions y 1 n 2 rsptivly. Lt (1) =min{ 1, 2 } n (2) =mx{ 1, 2 }. Stp 2. Drw (2) ootstrp rsmpls, X 1,...,X (2),fromX. Stp 3. lult ˆθ for h X, =1,..., (2). Stp 4. Dfin th intrvl to [ ] min {ˆθ : =1,..., 2}, mx {ˆθ : =1,..., 1}.

11 484 STPHN M. S. L Stp 1 n onvnintly rri out y numril mthos. To mk full us of th (2) rsmpls, Stp 4 my moifi to Stp 4.Orrthˆθ s ˆθ (1) ˆθ ( (2) ). If 1 = (2), fin th intrvl to 1! 1 1 2!( 1 2 )! (j 1)!{( 2 1)! (j 2 )!} 1 ˆθ (1 j+1), ˆθ (1 ) ; j= 2 if 2 = (2),finthintrvlto ˆθ (1), 2! 1 2 1!( 2 1 )! (j 1)!{( 1 1)! (j 1 )!} 1 ˆθ (j). j= 1 Stp 4 onstruts th n points y vrging ovr ll possil susts of siz i, i =1, 2, mong th (2) rsmpls, thry ruing th vriility of th n points. In prti, w rstrit th siz of to within rtin rng suh s Ifsolutionfor flls outsi this rng, w hoos tht vlu of in th rng whih stisfis th qution most losly Rmrks Our prour mploys xtrm ootstrp prntils inst of intrmit prntils in fining onfin limits. Th siz of thus trmin is optiml in th sns tht it rquirs th miniml omputtionl ffort to hiv rtin onfin lvl for stnr ootstrp intrvls. Not tht (1 + α)(1 α) 1 symptotilly, n is usully onsirly smllr thn wht hs n rommn for stnr ootstrp mthos. On th othr hn, it lso gurs ginst ss whr onvntionl hoi of fils to qutly ptur th til hviour of th ootstrp smpling istriution. Tl 1 lists th optiml sizs of for n = 20 n for th stnr norml, fol norml n oul xponntil istriutions. Thortil vlus of 1 n r us for riving hr. Th onfin lvl α rfrs to th ovrg of two-si intrvl, so tht th figurs for I P,L n I P,U orrspon to sizs of rquir to hiv onsi lvl of (1 + α)/2 h, or quivlntly, two-si lvl of α for I P,2.This llows irt omprison mong th two-si intrvls onsir hr. point shoul m out th sustntil isrpny twn th sizs of rquir for th uppr n lowr limits of I P,2 : s th figurs orrsponing to I P,U n I P,L. For th istriutions unr stuy, prvious symptoti n mpiril finings show tht th stnr ootstrp prntil mtho intrvl suffrs from

12 XTRM OOTSTRP PRNTILS 485 srious on-si ovrg rror, n thrfor fils to giv n urt quitil intrvl. Our prour for omputing I P,2 rivs its uppr limit from onsirly mor ootstrp rsmpls thn th lowr limit. This hlps shift th intrvl towrs th uppr til n givs ttr ln twn th two tils. This proprty is shr in gnrl y othr prmtrs in th smooth funtion mol stting. For, unr this mol, on-si ovrg ury is ontroll ssntilly y th ftor 1, t lst symptotilly, n our prour for omputing tks it into ount ptivly to rstor th ln. In this sns I P 2 njoys th itionl vntg ovr th stnr prntil mtho intrvl y hving mor urt on-si ovrgs. Tl 1. Optiml siz of for n = 20. Figurs for I P,U n I P,L r sujt to on-si ovrg lvls (1 + α)/2, n th rmining figurs to two-si ovrg lvls α. Norml t, N(0, 1) onfin lvl α I P,U Prntil mtho I P,L I P, ootstrp-t, I T, Fol norml t, N(0, 1) onfin lvl α I P,U Prntil mtho I P,L I P, ootstrp-t, I T, Doul xponntil t, xp ( x )/2 onfin lvl α I P,U Prntil mtho I P,L I P, ootstrp-t, I T, Th n for stimting 1 n nlytilly my t first sight rwk, spilly whn this involvs omputtion of high-orr smpl momnts whih oul turn out to highly unstl. Howvr, unlik othr nlytilly orrt onfin intrvls whih woul suffr from th sm prolm, our mtho onfins ll nlyti lultions xlusivly to rivtion of. It hs n g ovr th stnr ootstrp with its strtgi us of Mont rlo vrition to orrt for th thortil ovrg rror. Thr is mpiril vin tht n xtrmly stl ontrol ovr th hoi of is not s vitl s tht ovr th

13 486 STPHN M. S. L nlyti justmnts rquir to fix th n points of othr nlytilly orrt intrvls. Morovr, th xtrm intrvls s on th prntil mtho njoy th sm vntgs s mny stnr ootstrp mthos suh s th prntil mtho,, n thir itrt ountrprts. Ths vntgs inlu proprtis lik rng-prsrving, trnsformtion-rspting n monotoniity in α. stl vrin stimt is, unlik th ootstrp-t mtho, not rquir for h ootstrp rsmpl ovrg rror Th following proposition stts th orr of th ovrg rror of our intrvls in ss whr th nominl ovrg lvl is symptotilly los to on. Th proof is outlin in th ppnix, whr w ssum hs th thortil optiml vlu otin from our prour s on th tru vlus of 1 n. Th ovrg rsults rmin intt if 1 n r rpl y thir onsistnt stimts, sin th sustitution only ntils n rror of smllr orr. Proposition 3. ssum th onitions of Proposition 1 n tht th nominl ovrg lvl α stisfis n 1 1 α n δ 1, for ny δ 1, 1 (δ, ) with δ 1 < 1.Thn,withoptimlly hosn oring to our prour, (i) I P,L, I P,U n I P,2 hv ovrg rrors of orr O{n 1/2 1 (log n) 1/2 },n I P,2 of orr O{n 1 1 (log n) 2 };n (ii) I T,L, I T,U n I T,2 hv ovrg rrors of orr O(n 1 1 log n). Th ov ovrg rror, whih is givn in n solut sns, n intrprt s rltiv rror if th ftor 1 is ropp from th orr trm. It is thrfor of smllr orr thn th iffrn twn th nominl ovrg lvl n on. W not lso tht fftivnss of th xtrm prntil mtho hings on th rquirmnt tht α, n hn, inrssn inrss Numril xmpl W onlu with simpl xmpl rwn from Diiio n fron (1996). Th tst onsists of 4 ounts of 20 HIV-positiv sujts msur t slin n ftr 1 yr of ntivirl trtmnt. (S Tl 1 in Diiio n fron (1996) for th omplt tst.) W pply our mtho to gnrt nonprmtri two-si onfin intrvls for th orrltion offiint θ. Thsmpl stimtor ˆθ is lult to Figur 2 shows th n points n th sizs of for svn hois of th onfin lvl α. Th xis is givn on th logrithmi sl. Th stnr ootstrp prntil mtho n ootstrp-t intrvls r lso omput for omprison, h ing pproximt from 1,000 ootstrp rsmpls. rltivly smll numr of ootstrp rsmpls, rnging from 8 to 337, r rquir y our prour. For omputtion of I P,2,mor

14 XTRM OOTSTRP PRNTILS 487 rsmpls r n to yil th uppr thn th lowr limit, suggsting tht th stnr prntil mtho intrvl hs iggr symptoti nonovrg t th uppr til thn t th lowr, spilly whn α pprohs on. Our intrvl I P,2 is mor inlin to inlu th ˆθ ov ˆθ thn thos low, thus orrting for th unln lowr n uppr nonovrg proilitis. Th intrvl I T,2 xhiits similr tnny. On th ontrry, oth stnr ootstrp pprohs pl th intrvls mor to th lft of ˆθ. Th lttr hs n lim y Diiio n fron (1996) to sirl ftur on th sis of n xt intrvl otin unr th ivrit normlity ssumption. W fin thir lim qustionl in viw of our simultion rsults. In ft, tht th stnr ootstrp-t intrvl, whih hs ni qui-til proprtis, lis furthr to th right thn th stnr prntil mtho intrvl provis lu out th orrt shp of th intrvl. Our rsults ln furthr vin to this. (Lowr) Intrvls (Uppr) D 0.85 D 0.90 D D 0.95 D D nots stimtor D 10 onfin lvl onfin lvl / xtrm qul-til prntil (low/up) / xtrm prntil (low/up) / xtrm ootstrp-t (low/up) D/ Stnr prntil (low/up) / Stnr ootstrp-t (low/up) / xtrm qul-til prntil (low/up) xtrm prntil xtrm ootstrp-t Figur 2. xmpl: orrltion offiint of 4 t (Diiio n fron (1996)). Lft pnl: onfin limits of I P,2, I P,2, I T,2 n th stnr prntil mtho n ootstrp-t intrvls. Right pnl: numr of ootstrp rsmpls () rwn for onstruting I P,2, I P,2 n I T,2.

15 488 STPHN M. S. L 4. Simultion Stuy simultion stuy ws rri out to xmin th prformn of th xtrm onfin intrvls mpirilly. Two-si intrvls wr onstrut for θ, th vrin of on of thr unrlying istriutions: th stnr norml N(0, 1), th fol stnr norml N(0, 1) n th oul xponntil with nsity funtion xp ( x )/2. Rsults wr otin for iffrnt omintions of smpl sizs (n = 20 n 100) n nominl ovrg lvls (α =0.80, 0.85, 0.90, 0.925, 0.95, n 0.99). Optiml sizs of for th xtrm onfin intrvls wr stimt oring to th prour sri in Stion 3, whr 1 n wr stimt y thir strightforwr smpl stimts. Th stnr two-si prntil mtho n ootstrp-t intrvls wr lso omput s on 1,000 ootstrp rsmpls for omprison. Figurs 3, 4 n 5 summriz our finings otin from 1,600 rnom smpls rwn from h of th ov unrlying istriutions. Th top pnl shows th rltiv rrors in nonovrg proilitis t th lowr n uppr tils, fin rsptivly to n 2(1 α) 1 {(# intrvls ov θ)/1600 (1 α)/2} 2(1 α) 1 {(# intrvls low θ)/1600 (1 α)/2}, s wll s th ovrll rltiv rror in nonovrg proility, fin to (1 α) 1 {(# intrvls missing θ)/1600 (1 α)}. Th mil pnl givs tils out th vrg positions of th uppr n lowr limits, whr th rrows init th positions plus n minus thir stnr rrors. Th ottom pnl givs th vrg sizs of us in onstrution of th xtrm intrvls. Th sizs r plott on th logrithmi sl, with rrows initing th sizs plus n minus stnr rrors. Th simultion rsults roly gr with th symptoti thory vlop rlir for th vrious intrvls. W osrv tht in gnrl th stnr ootstrp-t intrvl is th most urt in trms of oth on-si n two-si nonovrg proilitis. Th stnr prntil mtho intrvl is inurt t ithr til, with xptionlly svr unrovrg t th uppr til. Th intrvl I P,2 orrts for suh on-si ovrg rrors vry fftivly, spilly t th uppr til. In most ss, it vn outprforms th stnr ootstrp-t mtho in trms of hving vry smll lowr til ovrg rror. Its ovrll two-si ovrg rror is smllr thn th stnr prntil mtho y onsirl mrgin ut is iggr thn th stnr ootstrp-t. Th intrvl I P,2 is gnrlly lss urt thn I P,2 in trms of on-si ovrg. Its two-

16 XTRM OOTSTRP PRNTILS 489 N(0, 1), n =20 N(0, 1), n = 100 onfin limits ovrll uppr lowr D D D D D D D onfin limits ovrll uppr lowr D D D D D D D onfin lvl onfin lvl Figur 3. xmpl: vrin of N(0, 1) for n = 20 n 100. Top pnl: rltiv rror in lowr, uppr n ovrll (two-si) nonovrg proilitis of, in sning orr of shing nsity, I P,2 (soli shing), I P,2, I T,2, th stnr prntil mtho intrvl n th stnr ootstrpt intrvl. Mil pnl: mn positions of lowr/uppr onfin limits of I P,2 (/), I P,2 (/), I T,2 (/), th prntil mtho intrvl (D/) n th ootstrp-t intrvl (/), with rrows initing thir stnr rrors. ottom pnl: mn numr of ootstrp rsmpls rwn for I P,2 (: lowr limit, : uppr limit), I P,2 () n I T,2 (), with rrows initing thir stnr rrors. si ovrg ury is, howvr, osionlly ttr sin ovrovrg t on til my hppn to ompnst for unrovrg t th othr. Th prformn of I T,2 pns lot on th smpl siz, suggsting its snsitivity to th hoi of. For smll smpl siz, it hs vry inurt lowr nonovrg proility n rsonly urt uppr on, rsulting in two-si ovrg rror quit similr to th stnr prntil mtho. n inrs in smpl siz,

17 490 STPHN M. S. L whih givs ris to mor stl hoi of, grtly improvs its prformn in trms of oth on-si n two-si ovrg uris. In ft, simultions of I T,2 hv n rrun using th thortil vlu of otin from (6), rsulting in th most urt ovrg proilitis otin thus fr, for oth smpl sizs. This suggsts n ppling potntil of I T,2 to giv high ovrg ury, lthough urt stimtion of rmins prtil iffiulty. Suh iffiulty, howvr, poss lss srious prolm for I P,2 n I P,2 u to thir rltiv insnsitivity to th hoi of. N(0, 1), n =20 N(0, 1), n = 100 onfin limits ovrll uppr lowr D D D D D D D lowr uppr ovrll onfin limits D D D D D D onfin lvl onfin lvl Figur 4. xmpl: vrin of N(0, 1) for n = 20 n 100. Top pnl: rltiv rror in lowr, uppr n ovrll (two-si) nonovrg proilitis of, in sning orr of shing nsity, I P,2 (soli shing), I P,2, I T,2, th stnr prntil mtho intrvl n th stnr ootstrpt intrvl. Mil pnl: mn positions of lowr/uppr onfin limits of I P,2 (/), I P,2 (/), I T,2 (/), th prntil mtho intrvl (D/) n th ootstrp-t intrvl (/), with rrows initing thir stnr rrors. ottom pnl: mn numr of ootstrp rsmpls rwn for I P,2 (: lowr limit, : uppr limit), I P,2 () n I T,2 (), with rrows initing thir stnr rrors. D

18 XTRM OOTSTRP PRNTILS 491 Doul xponntil, n = 20 Doul xponntil, n = 100 onfin limits ovrll uppr lowr D D D D D D D onfin limits ovrll uppr lowr D D D D D D D onfin lvl onfin lvl Figur 5. xmpl: vrin of oul xponntil for n = 20 n 100. Top pnl: rltiv rror in lowr, uppr n ovrll (two-si) nonovrg proilitis of, in sning orr of shing nsity, I P,2 (soli shing), I P,2, I T,2, th stnr prntil mtho intrvl n th stnr ootstrp-t intrvl. Mil pnl: mn positions of lowr/uppr onfin limits of I P,2 (/), I P,2 (/), I T,2 (/), th prntil mtho intrvl (D/) n th ootstrp-t intrvl (/), with rrows initing thir stnr rrors. ottom pnl: mn numr of ootstrp rsmpls rwn for I P,2 (: lowr limit, : uppr limit), I P,2 () n I T,2 (), with rrows initing thir stnr rrors. W s from th mil pnls of Figurs 3 to 5 tht th stnr ootstrpt intrvl, spit its wll-stlish ury, hs vry unstl n points n n ovrstrth lngth. This is minly u to its n for, usully unstl, vrin stimt for ˆθ. Proprtis suh s trnsformtion-rspting n rngprsrving r lso lost s rsult. On th othr hn, suh rwks pply to nithr th stnr nor th xtrm prntil mtho intrvls. Thus th lttr

19 492 STPHN M. S. L stns s n spilly strong omptitor mong xisting stnr ootstrp pprohs. Th optiml stimt for th xtrm intrvls is typilly smll ompr to th onvntionl hoi of, sy, = 1000, for th rng of onfin lvls stui in th xmpls. In prtiulr, th optiml slom gos yon 100 if th onfin lvl rmins low Th intrvl I P,2 tks mor ootstrp rsmpls to fin th uppr thn th lowr limit, whih hlps shift th intrvl furthr to th right of ˆθ to ompnst for th svr unrovrg suffr y th stnr prntil mtho t th uppr til. stimts of th optiml r sujt to vrious grs of vrition for th xtrm intrvls. roly spking, I P,2 is ssoit with th most flututing stimt of, follow nxt y I T,2, whil I P,2 givs th most stl stimt. It shoul not tht th xtrm intrvls I P,2 n I P,2 possss rthr stl lngths n n points omprl to th stnr prntil mtho intrvl, spit th us of muh smllr for onfin lvls low Th ngr of n ovrly long intrvl u to smll, s hs n isuss in Hll (1986), os not sm to xist hr. 5. Disussion To summriz, xtrm ootstrp onfin intrvls otin from n ptivly hosn numr of ootstrp rsmpls improv upon th stnr prntil mtho whih hs notoriously inurt ovrg. lthough thir ovrg ury my not ttr thn th stnr ootstrp-t mtho, th xtrm intrvls inst njoy vntgs suh s hving stl lngths n n points. Morovr, th us of xtrm prntils rstrits to rsonl siz whih just suffis for yiling th sir ovrg lvl. This optiml siz is oftn muh smllr thn th onvntionl hoi, provi α is not too los to on, in whih s th xtrm intrvl is omputtionlly vry sirl. Thr r lso ss whr th optiml siz xs th onvntionl hoi n thus orrts for inquy of th lttr. mong th xtrm intrvls, I P,2 is prtiulrly ppling. For, in ition to th ov vntgs, it hs goo on-si ovrg uris, njoys stl stimt of, n is oth trnsformtionrspting n rng-prsrving. Th xtrm ootstrp-t intrvl I T,2, howvr, rquirs stl stimt of to yil stisftory rsults. Not furthr tht th xtrm intrvl n points r ll sujt to Mont rlo vrin of th sm orr, O p {n 1 (log ) 1 }, s th stnr pprohs unr th ssumptions of Proposition 3, spit thir rlin on n mpiril trmintion of. Our mpiril finings onfirm this osrvtion. y its vry ntur, th xtrm prntil mtho my onsirly fft y outlir ootstrp rsmpls. It n m mor roust though, t th

20 XTRM OOTSTRP PRNTILS 493 xpns of littl xtr omputtionl ffort. Inst of rwing th rquir ootstrp rsmpls, w rw =(1+ψ) rsmpls for som smll ψ>0. Th xtrm ootstrp prntil is lult for h sust of siz mong th rsmpls. Th onfin intrvl is thn riv from ithr th vrg or th min of ths xtrm prntils, hn iminishing th fft of outlir rsmpls. Th min i of our mtho, nmly, th ptiv justmnt of Mont rlo ffort to orrt for thortil ovrg rror, n implmnt y ltrntiv pprohs. For instn, th optiml n riv from th symptoti xpnsions for th ovrgs of th stnr Mont rlo pproximt ootstrp intrvls. Hll (1986) givs suh n xpnsion, in th sns of ltting oth n n tn to infinity, for th ovrg proility of th stnr lowr ootstrp-t onfin limit. n pproprit my otin y quting th xpnsion to th sir nominl ovrg lvl. Its vlu pns on th prtiulr intrmit ootstrp-t prntil onsir in th xpnsion. It sms tht th siz of my ru y foring this prntil to th xtrm on. Howvr, Hll s (1986) xpnsion os not onvnintly tr to th lttr s. Our Proposition 2 rsolvs th prolm n givs th smllst possil. s hs n point out in Stion 2.3, our prour yils vlus of whih my lso initiv of th miniml Mont rlo ffort rquir of th stnr ootstrp pprohs. prt from thir mthoologil implitions, w fin in suh vlus of pplitions of ignosti kin. Hug sizs of sn n lrming signl out th possil filur of th ootstrp ing ppli to th qustion in hn. W s from Tl 1 tht th thortil vlu of inrss rpily s α pprohs on. For xmpl, in th s of th oul xponntil vrin, th stnr prntil n ootstrp-t mthos woul typilly rquir t lst 37,187 n 18,308, rsptivly, in orr to giv n urt 99% two-si onfin intrvl. Hr th onvntionl hoi of, whih is muh smllr, woul not work stisftorily. Our isussion hs thus fr n rstrit to th smooth funtion mol for slr prmtr. It is lr from th proofs in th ppnix tht our prour gnrlizs to ny situtions whr gworth xpnsions n foun for th ootstrp istriutions. Ths inlu th s whr θ is von-miss funtionl in prtiulr. Diiio n fron (1996) isuss vrity of situtions mnl to gworth xpnsions. n importnt n nturl xtnsion of our prour is to th multivrit stting whr onfin rgion woul riv from th onvx hull of ll th ootstrp t. This ypsss th prtil iffiulty prtining to th orring of multivrit t whih is gnrlly m ssntil to ny stnr ootstrp pproh. This is topi of futur rsrh.

21 494 STPHN M. S. L knowlgmnts Th uthor grtfully knowlgs two rviwrs for thir vlul ommnts n suggstions. ppnix Proof of Proposition 1. Lt Ĵ th onitionl istriution funtion of n 1/2 (ˆθ 1 ˆθ)/ˆσ givn X. Dnot y φ n Φ th stnr norml nsity n istriution funtions rsptivly. Dfin z β =Φ 1 (β). Not tht pr (Ĵ( T ) u) is th tru ovrg of th stnr prntil mtho intrvl of nominl ovrg u. It follows from Hll (1992, 3.5 tht pr (Ĵ( T ) u) =u + n 1/2 R n (u), (.1) whr R n (u) =φ(z u )p n (z u )+O(n (M+1)/2 ), for som polynomil p n n som suffiintly lrg M to trmin. Not tht R n (0) = R n (1) = 0. Hn w hv pr (θ ˆθ () )=(1 Ĵ( T) ) 1 =1 ( +1) 1 n 1/2 u R n (u). 0 (.2) Lt G th stnr Guml istriution funtion, G(x) =xp( x ), n G th istriution of th mximum of inpnnt uniform [0, 1] rnom vrils. Using stnr symptoti rsults for xtrm orr sttistis s givn, for xmpl, in Riss (1989, 5.2, w otin 1 0 u R n (u)= = 1 0 R n (u) G (u) φ( 1 v + 1 )p n ( 1 v + 1 ) G(v){1+O( 2 )} + O(n (M+1)/2 ) = φ( 1 )p n ( 1 ){1+O( 2 )} + O(n (M+1)/2 ), (.3) whr stisfis φ( 1 )=. Th lst qulity follows y xpning φp n out 1 n th ft tht v G(v) = 1. pplying Hll s (1992, 3.5 gworth xpnsions for ovrgs of stnr ootstrp intrvls n noting n δ n,whv p n ( 1 )= (1/6)σ {1+O( 2 )} + o(n 1/2+ɛ ), (.4) for ny ɛ>0. Th xprssion (1) thn follows y stting M 2 1n sustituting (.3) n (.4) in (.2). Th proof of (2) is ntirly nlogous with

22 XTRM OOTSTRP PRNTILS rpl y 1. For th two-si s, not first tht pr (ˆθ (1) θ ˆθ () ) [ { } =1 2( +1) 1 +n 1/2 φ( 1 ) p n ( 1 ) p n ( 1 ) {1+O( 2 )} ] + O(n (M+1)/2 ), (.5) whih follows from (.2) n its lowr til vrsion. xpnsions show tht, for ny ɛ>0, gin, Hll s gworth p n ( 1 ) p n ( 1 ) = (1/36)n 1/2 σ {1+O( 2 )} + o(n 3/2+ɛ ). (.6) Sustituting (.6) in (.5) n hoosing M 2, w prov (3). Proof of Proposition 2. Th proof is similr to tht of Proposition 1. Dfin S n y pr { pr (T 1 T X ) u } = u + n 1/2 S n (u). W thn hv pr (θ <ˆθ n 1/2ˆσT 1 () )=( +1) 1 + n 1/2 u S n (u), n hn (5) follows from rgumnts similr to thos us for proving (1). W n prov (4) in th sm fshion n (6) follows y omining (4) n (5) in trivil wy. ProofofProposition3. Th rsults follow immitly y noting tht = O{(log n) 1/2 } n tht stisfis n δ n provi n 1 1 α n δ 1 for n suffiintly lrg. Rfrns rn, R. (1987). Prpivoting to ru lvl rror of onfin sts. iomtrik 74, htthry, R. N. n Ghosh, J. K. (1978). On th vliity of th forml gworth xpnsion. nn. Sttist. 6, ooth, J. G. n Hll, P. (1994). Mont rlo pproximtion n th itrt ootstrp. iomtrik 81, ooth, J. n Prsnll,. (1998). llotion of Mont rlo rsours for th itrt ootstrp. J. omput. Grph. Sttist. 7, Diiio, T. J. n fron,. (1996). ootstrp onfin intrvls (with Disussion). Sttist. Si. 11, fron,. (1979). ootstrp mthos: nothr look t th jkknif. nn. Sttist. 7, fron,. (1981). Nonprmtri stnr rrors n onfin intrvls (with Disussion). n. J. Sttist. 9, fron,. (1987). ttr ootstrp onfin intrvls. (with Disussion). J. mr. Sttist. sso. 82,

23 496 STPHN M. S. L Hll, P. (1986). On th numr of ootstrp simultions rquir to onstrut onfin intrvl. nn. Sttist. 14, Hll, P. (1992). Th ootstrp n gworth xpnsion. Springr, Nw York. Hinkly, D. V. n Wi,.. (1984). Improvmnt of jkknif onfin limit mthos. iomtrik 71, L, S. M. S. n Young, G.. (1995). symptoti itrt ootstrp onfin intrvls. nn. Sttist. 23, L, S. M. S. n Young, G.. (1999). Th fft of Mont rlo pproximtion on ovrg rror of oul-ootstrp onfin intrvls. J. Roy. Sttist. So. Sr. 61, Polnsky,. M. (1997). nwith sltion for th smooth ootstrp prntil mtho. Prprint. Polnsky,. M. n Shuny, W. R. (1997). Krnl smoothing to improv ootstrp onfin intrvls. J. Roy. Sttist. So. Sr. 59, Riss, R.-D. (1989). pproximt Distriutions of Orr Sttistis: With pplitions to Nonprmtri Sttistis. Springr, Nw York. Sho, J. n Tu, D. (1995). Th Jkknif n ootstrp. Springr, Nw York. Tu, D. (1992). pproximting th istriution of gnrl stnriz funtionl sttisti with tht of jkknif psuovlus. In xploring th Limits of ootstrp, (ityr. LPg n L. illr), Wily, Nw York. Dprtmnt of Sttistis n turil Sin, Th Univrsity of Hong Kong, Pokfulm Ro, Hong Kong. -mil: smsl@hkusu.hku.hk (Riv pril 1998; pt Jun 1999)