Robust Estimation for ARMA models

Transcription

1 Robust Estimtio for ARMA modls Nor Mulr, Dil Pñ y d Víctor J. Yohi z Uivrsidd Torcuto di Tll, Uivrsidd Crlos III d Mdrid d Uivrsidd d Buos Airs d CONICET. Novmbr 8, 007 Abstrct This ppr itroducs w clss of robust stimts for ARMA modls. Thy r M-stimts, but th rsiduls r computd so tht th ct of o outlir is is limitd to th priod whr it occurs. Ths stimts r closly rltd to thos bsd o robust ltr, but thy hv two importt dvtgs: thy r cosistt d th symptotic thory is trctbl. W prform Mot Crlo whr w show tht ths stimts compr fvorbl with rspct to stdrd M-stimts d to stimts bsd o digostic procdur. Itroductio Thr r two mi pprochs to dl with outlirs wh stimtig ARMA modls. Th rst pproch is to strt stimtig th modl prmtrs usig mximum liklihood d th lyzig th rsiduls with digostic procdur to dtct outlirs. Amog othrs, digostic procdurs for dtctig outlirs wr proposd by Fox [0], Chg, Tio d Ch [5], Tsy [4], Pñ [3], d Ch d Liu [6]. Howvr digostic procdurs su r from th mskig problm: wh thr r svrl outlirs which hv similr cts, th outlirs my ot b dtctd. A scod pproch is to us robust stimts, tht is, stimts which r ot much i ucd by outlyig obsrvtios. A dtild rviw of robust procdurs for ARMA modls c b foud i Chptr 8 of Mro, Mrti d Yohi [7]. I this Chptr it is show tht i th cs of AR(p) modl, o outlir t obsrvtio t c ct th rsiduls corrspodig to priods t 0 ; Supportd i prt by Grt PAV from ANPCYT. y Supportd i prt by Grt SEJ by MEC, Spi. z Supportd i prt by Grt X094 from th Uivrsity of Buos Airs, Grt PICT 407 from ANPCYT Argti d Grt PIP 5505 from CONICET.

2 t t 0 t + p; d i th cs of ARMA(p; q) modl with q > 0 it c ct ll rsiduls corrspodig to priods t 0 t. For this rso stimts bsd o rgulr rsiduls (for xmpl M- or S-stimts) r ot vry robust. O wy to improv th robustss of th stimts is to comput th rsiduls usig th robust ltr itroducd by Msrliz []. This robust ltr pproximts th o stp prdictor i ARMA modls with dditiv outlirs. Svrl uthors hv proposd stimts tht us rsiduls computd with th Msrliz ltr. For istc, Mrti, Smrov d Vdl [0] proposd ltrd M-stimts, Mrti d Yohi [9] ltrd S-stimts d Bico, Grci B, Mrtiz, d Yohi [] ltrd stimts. Howvr, w c mtio two shortcomigs of th stimts bsd o ltrd rsiduls. Th rst o is tht ths stimts r symptoticlly bisd. Th scod o is tht thr is ot symptotic thory for ths stimtors, d thrfor ifrc procdurs lik tsts or co dc rgios r ot vilbl. I this ppr w propos w pproch to void th propgtio of th ct of o outlir wh computig th iovtio rsiduls of th ARMA modl: w d ths rsiduls usig uxiliry modl. For this purpos w itroduc th boudd iovtio propgtio ARMA (BIP-ARMA) modls. With th hlp of ths modls, w r bl to d stimts for th ARMA modl which r highly robust wh th sris cotis outlirs. W show tht th mchisms of th proposd stimts to void th propgtio of th outlirs r similr to thos bsd o robust ltrs. Howvr, th dvtg of ths stimts ovr thos bsd o th robust ltrs is tht thy r cosistt d symptoticlly orml udr prfctly obsrvd ARMA modl. Th proposd stimts c b cosidrd s grliztio of th MMstimts itroducd by Yohi [6] for rgrssio. I th rst stp w d highly robust rsiduls scl d i th scod stp w us rdscdig M-stimt which uss this scl. For brvity sk w hv omittd i this ppr som of th proofs. All th proofs c b foud i Mulr, Pñ d Yohi []. Th ppr is orgizd s follows. I Sctio w itroduc th w fmily of modls d show tht th corrspodig rsiduls r similr to thos obtid with robust ltr. I Sctio 3 w itroduc th proposd stimts. I Sctio 4 w stblish th mi symptotic rsults: cosistcy d symptotic ormlity. I Sctio 5 w discuss th computtio of th proposd stimts. I Sctio 6 w discuss robustss proprtis of th proposd stimts. I Sctio 7 w prst th rsults of Mot Crlo study. I Sctio 8 w show th prformc of th di rt stimts for ttig mothly rl sris. I Sctio 9 w mk som cocludig rmrks. Sctio 0 is Appdix with th mi proofs of th symptotic rsults.

3 A Nw Clss of Boudd Nolir ARMA modls. BIP-ARMA modls W r goig to cosidr sttiory d ivrtibl ARMA modl tht c b rprstd by (B)(x t ) = (B) t () whr t r i.i.d. rdom vribls with symmtric distributio P d whr p (B) d (B) P r polyomil oprtors giv by (B) = i= ib i d q (B) = i= ib i with roots outsid th uit circl. If t hs rst momt w hv tht E(x t ) = : Lt (B) = (B)(B) = + P i= ib i d cosidr th MA() rprsttio of th ARMA procss x t = + t + X i t i : () W c modl cotmitd ARMA procsss with frctio " of outlirs by z t " = ( " t )x t + " t w t (3) whr y t is th ARMA modl, w t is rbitrry procss d " t is procss tkig vlus 0 d such tht lim! =( P i= " t ) = ": For xmpl " t my b sttiory procss such tht E( " t ) = " : Th cs of dditiv outlirs corrspods to w t = x t + t, whr x t d v t r idpdt procsss. Rplcmt outlirs corrspod to th cs tht th procsss x t d w t r idpdt. Accordig to th dpdc structur of th procss " t w c hv dditiv outlirs or ptchy outlirs. For dtil, s Mrti d Yohi [8]. Robustss is rltd with th possibility of ccurtly stimtig th prmtr of th ctrl modl x t wh w obsrv th cotmitd procss z t ". Aothr typ of outlirs r iovtio outlirs. A ARMA procss with iovtio outlirs occurs wh w obsrv ARMA procss stisfyig () but th iovtios t hv hvy-tild distributio. Rgulr M-stimts c cop with this typ of outlirs. S for xmpl Mro t l. [7]. W will us th followig fmily of uxiliry modls y t = + t + X i= i= t i i ; (4) whr th t s r i.i.d. rdom vribls with symmtric distributio d is robust M-scl of t which coicids with th stdrd dvitio i th cs tht th t s r orml, th i s r th co cits of (B)(B) d (x) is odd d boudd fuctio. A M-scl of t is d d s th solutio of th qutio E ( ( t =)) = b:. W cll this modl, th boudd iovtio propgtio utorgrssiv movig vrg modl (BIP-ARMA). 3

4 To obti robust d cit stimts w will choos boudd d such tht thr xists k with (x) = x for jxj k: Mor dtils o how to choos d b d r giv i Sctios 3. d 6. Not tht i this modl th lg ct of lrg iovtio i priod t hs boudd ct o y t+j for y j 0 d sic j! 0 xpotilly wh j!, this ct will lmost disppr i fw priods. Not tht (4) c lso b writt s y t = + t t d multiplyig both sids by (B) w gt + (B)(B) t (B)y t = ( which is quivlt to i ) + (B) t i= (B) t + (B) t y t = t + + i (y t i ) i= rx i= t i t i + ( i i ) i ; (5) whr r = mx(p; q): If r > p, p+ = = r = 0 d if r > q; q+ = = r = 0.. Robust ltrs d BIP-ARMA modls Lt us lyz th rltioship of th BIP-ARMA modl d ARMA modl with dditiv outlirs. Th BIP-ARMA modl c b lso b writt s y t = ( " t )x t + " t (x t + t ); whr x t = y t t + t is ARMA modl stisfyig (B)(x t ) = (B) t, t = ( t =); t = t t ; " t = I(j t j k) d " = P ((j t j k): Howvr, i th BIP-ARMA modl " t d t r ot idpdt d thy r lso ot idpdt of x t : W will show tht th o-stp forcst i th BIP-ARMA modl is similr to th forcst obtid by usig th robust ltr for ARMA modls itroducd by Msrliz []. Th Msrliz ltr ws proposd s pproximtio to o-stp prdictor for dditiv modls of th form (3), whr x t is Gussi ARMA modl, " t r i.i.d. Broulli vribls with P ( " t = ) = " d t r i.i.d. orml rdom vribls. Suppos tht w hv ARMA sris y ; :::; y d tht w suspct tht it is cotmitd with dditiv outlirs. Assum rst tht w kow th prmtrs ; ; d of th ARMA modl. Th robust ltr computs cl sris yt, d ltrd iovtios rsiduls b t tht r obtid by th followig rcursiv procdur. Suppos tht th cld sris y; :::; yt ; d th ltrd iovtio rsiduls b ; :::; b t prvious to tim t r computd. Sic y t = P i= i(y t i ) + t ; whr (B) = (B) (B) = + P i= ib i, th o-stp hd robust forcst of y t ; is obtid by rplcig th y t i s by 4

5 th cld vlus y t i s, i.., th o-stp robust forcst of y t is obtid by by t = X i (yt i ) = ((B) (B) )(yt ); (6) i= whr yt = for t 0: Th ltrd iovtio rsidul for priod t is computd by b t = y t by t d th cld vlu yt by b yt = by t + s t t b = y t b t + s t t ; (7) s t s t whr s t is stimt of th o-stp prdictio rror scl d whr hs th sm proprtis s thos sttd for, i prticulr for som k > 0 it holds tht (u) = u for juj k: Obsrv tht if jb t j k; th s t (b t =s t ) = b t d y t = y t : Rcursiv formul for s t c b foud i Mrti t l. [0]. W c sily driv from (6) d (7) tht by t = + i (yt i ) i= qx b i s t t i= s t i : (8) Now, from (5), th o stp forcst for y t i th BIP-ARMA modl is giv by by t = + i= t i y t i t i + i qx i= t i i ; (9) which is similr to (8) tkig s th cld sris y t = y t t + ( t = ) : (0) Th mi di rc is tht hr s t is tk is tk costt d qul to : Thus, th ltrd rsiduls usd by Mrti t l. [0] d Bico t l. [] r vry similr to thos of BIP-ARMA modl. I th xt Sctio w will us th modl (4) to d robust stimts of th prmtrs of ARMA modl tht my coti dditiv outlirs. 3 Boudd MM-stimts for ARMA modls Assum tht y ; :::; y r obsrvtios corrspodig to BIP-ARMA modl d tht th dsity of t is f(u): Th coditiol log liklihood fuctio of y p+ ; :::; y giv y ; :::; y p d th vlus b p r+(;) = 0; :::; b p(;) = 0, whr r = mx(p; q), c b writt s L c (;) = t=p+ log f b t(;) ; () 5

6 whr from (5), th fuctios b t(;) r d d rcursivly for t p + by rx b t(;) = y t i (y t i )+ i b b t i t i (;) + ( i i ) (;)!! : i= i= () I th cs of pur ARMA modl, i.., whr (u) = u; () rducs to t () = y t qx i (y t i ) + i t i (): (3) i= i= Sic ML-stimts r ot robust, w will cosidr M-stimts which miimizs M() b = b t (;b) ; (4) p b t=p+ whr is boudd fuctio, d b is stimt of. W obsrv tht th M-stimts d d i (4), rquir to hv stimt b of : This ld us to d i Sctio 3. two stp procdur for stimtig tht w cll MM-stimts. 3. M-stimts of scl Hubr [3] itroducd th M-stimts of scl. Giv smpl u = (u ; :::; u ); u i R, M-stimt of scl S (u) is d d by y vlu s (0; ) stisfyig ui = b; (5) s i= whr is fuctio stisfyig th followig proprty P giv blow. P: (0) = 0; (x) = ( x); (x) is cotiuous, o-costt d odcrsig i jxj. W c d two symptotic brkdow poits of M-stimt of scl: th miimum frctio of outlirs which r rquird to mk th stimt i ity, ; d th miimum frctio of ilirs tht my tk th stimt to zro, 0. Hubr [4] shows tht = b= d 0 = b= whr = mx. Th, th globl brkdow poit of ths stimts is = mi( ; ) d so tkig b = = w gt mximum brkdow poit of 0:5. To mk th M- scl stimt cosistt for th stdrd dvitio wh th dt r orml, w rquir tht E ((x)) = b whr is th stdrd orml distributio: 3. MM-Estimts Th MM-stimts for rgrssio wr itroducd by Yohi [6] to combi high brkdow poit with high cicy udr orml rrors. Th ky id of th MM-stimts is to comput i th rst stp highly robust stimt of th rror scl, d i th scod stp this scl stimt is usd to comput 6

7 M-stimt of th rgrssio prmtrs. For tim sris modls ths two stps r ot ough to gurt robustss. This is du to th fct tht outlir i o priod, dos ot oly ct th rsidul corrspodig to this priod, but it my lso ct ll th subsqut rsiduls. To void this propgtio w d MM-stimts for th ARMA modl, whr th rsiduls r computd s i th BIP-ARMA modl istd s i th rgulr ARMA modl. Th, th procdur for computig MM-stimts is s follows. Stp. I this stp w obti stimt of : For this purpos w cosidr two stimts of ; o usig ARMA modl d othr usig BIP-ARMA modl d choos th smllst o. Lt b boudd fuctio stisfyig P d such tht if b = E( (u)); th b= mx (u) = 0:5: This gurts tht for orml rdom smpl th M-scl stimtor s bsd o covrgs to th stdrd dvitio d tht th brkdow poit of s is 0:5. Put B 0; = (; ) R p+q : jzj + holds for ll th roots z of (B) d (B) : (6) Lt us cll B 0 = B 0; for som smll > 0 d B = B 0; R. Th, w d stimt of b S = rg mi S ( ()) (7) B d th corrspodig stimt of s = S ( ( b S )); (8) whr () = ( p+ (); :::; ()) d S is th M-stimt of scl bsd o d b: Lt us dscrib ow th stimt corrspodig to th BIP-ARMA modl. D b b S = ( b b S; b b S; b b s) by th miimiztio of S ( b (;b(; ))) ovr B: Th vlu b(; ) is stimt of computd s if (; ) wr th tru prmtrs d th t s wr orml. Th sic i this cs th M-scl coicids with th stdrd dvitio of t, from (4) w hv = y + P i= ; i (; ) whr =vr(( t =)) d y =vr(y t ): Lt b y b robust stimt of y d =vr((z)) whr z hs N(0,) distributio. Th, w d b (; ) = b y + P i= : (9) i (; ) Th scl stimt s b corrspodig to th BIP-ARMA modl is d d by b b S = ( b b S; b b S; b b s) = rg mi B S ( b (;b(; ))) (0) 7

8 d s b = S b b b S; b( b b S; b b S) ; () whr b (; ) = ( b p+(; ); :::; b (; )). Our stimt of is s = mi(s ; s b ): () As w will s i th xt sctio, if th smpl is tk from ARMA modl without outlirs, symptoticlly w obti s < s b : W should poit out tht dspit th fct tht ws computd s if th t s wr orml, th symptotic proprtis of th stimtors r ot goig to dpd o this ssumptio. Stp. Cosidr boudd fuctio such tht stis s P d : This fuctio is chos so tht th corrspodig M-stimt is highly cit udr orml iovtios. Lt d M () = M b () = p p t=p+ t=p+ t () s (3) b t (;s ) s : (4) W d th stimts b M d b b M by th miimiztio ovr B of M () d M b () rspctivly. Th, th MM-stimt b M is qul to b M if M ( b M ) M b ( b b M ) d is qul to b b M if M ( b M ) >M( b b b M ): For istc w c tk (x) = (x) with 0 < <. If 00 (0) > 0; will b clos to qudrtic fuctio wh tds to 0. Rmrk. O importt problm tht will b oly bri y mtiod hr is tht of th robust modl slctio. O possibility to xplor is to dpt to ARMA modls th robust it prdictio rror (RFPE) slctio critrio giv i Sctio 5. of Mro t l. [7] for rgrssio. I th xt sctio w will show tht wh th smpl is tk from ARMA modl without outlirs, for lrg th stimt will choos b M = b M ; d i our Mot Crlo study of Sctio 7 w obsrv tht if th smpl hs ough dditiv outlirs w my hv b M = b b M. This implis tht b M d b b M hv th sm symptotic distributio for y. Howvr, th cicy of b b M for it smpl siz dpds of : If th itrvl whr coicids with th idtity icrss, th cicy for it smpl siz of b b M will icrs too, but th propgtio of th outlirs ct will gi importc d so th stimt will los robustss. 4 Asymptotic rsults Th mi rsults of this Sctio, sttd i Thorms 4 d 6, r th cosistcy d symptotic ormlity of th BMM-stimtors for ARMA modls. Ths 8

9 thorms rquir to prov rst th cosistcy of S- d th cosistcy d symptotic ormlity of MM-stimtors. W sttd ths rsults i Thorms, 3 d 5 rspctivly. Th lik tht rlts th proprtis of S- d MM- to thos of BMM-stimts r Thorms d 4. Cosidr th followig ssumptios: P. Th procss y t is sttiory d ivrtibl ARMA (p; q) procss with prmtr 0 = ( 0 ; 0 ; 0 ) B d E(log + j t j) <, whr log + = mx(log ; 0). Th polyomils 0 (B) d 0 (B) do ot hv commo roots. P3. Th iovtio t hs bsolutly cotiuous distributio with symmtric d strictly uimodl dsity. P4. P ( t C) < for y compct C. P5. Th fuctio is cotiuous, v d boudd. Th followig Thorm stblishs th cosistcy of th S-stimts bsd o ARMA modls. Thorm. Assum tht y t stis s P with iovtios t stisfyig P3. Assum lso tht is boudd d stis s P with > b, d tht = 0 is boudd d cotiuous. Th, (i) Th stimt b S d d i (7) is strogly cosistt for 0. (ii) 9

10 Lt s b th scl stimt d d i (8). Th s! s 0 :s: whr s 0 is d d by E ( ( t =s 0 )) = b: Th xt Thorm stblishs tht udr rgulr ARMA modl b S d b b S r symptoticlly quivlt. Thorm. Assum tht y t stis s coditio P, with iovtios t stisfyig P3 d P4. Assum lso tht is boudd d stis s P with > b, tht = 0 is boudd, cotiuous d tht stis s P5. Th if y t is ot whit ois, with probbility thr xists 0 such tht b b S = b S for ll 0 d th s d d i () vri s s! s 0.s.. Th rso why th bov thorm rquirs tht y t is ot whit ois is tht i tht cs both modls: th rgulr ARMA d th BIP-ARMA with y fuctio ; coicids. Thrfor, i this cs it dos ot mttr which of th two modl is chos. Th followig Thorms shows th cosistcy of th MM-stimt. Thorm 3. Assum tht y t stis s coditio P, with iovtios t stisfyig P3. Assum lso tht i, i=,, r boudd d stisfy P, i = 0 i ; i=, r boudd d cotiuous d tht > b: Th b M! 0.s.. Th xt Thorm shows tht symptoticlly udr rgulr ARMA modl b M d b b M r quivlt. Thorm 4. Suppos tht th ssumptios of Thorm 3, P4 d P5 hold: Th if y t is ot whit ois, with probbility thr xists 0 such tht b b M = b M for ll 0 d th b M! 0.s.. Th followig Thorm shows th symptotic ormlity of th MM-stimts. Thorm 5. Suppos tht th ssumptios of Thorm 3 hold: Morovr ssum tht 0 d 00 r cotiuous d boudd fuctios d = E( t ) < : Th w hv whr ( p) = ( b M 0 )! D N(0; D); D = s 0E F0 ( t =s 0 ) E F 0 ( 0 ( t =s 0 )) 0 = C d C = (c ij ) is th (p + q + ) (p + q + ) mtrix giv by c i;j = c p+i;p+j = i= qx i= 0i 0i X k k+j i if i j p k=0 X $ k $ k+j i if i j q; k=0 0 ; (5)

11 c i;p+j = c i;p+j = X $ k k+j i if i p; j q; i j k=0 X k $ k+i j if i p; j q; j i ; k=0 whr 0 (B) = + P i= ib i d 0 (B) = + P i= $ ib i : Obsrv tht wh th t s r orml, = : Rmrk. Wh (u) = u ; b M is th coditiol mximum liklihood stimt corrspodig to orml rrors. Lt F 0 b th distributio of t ; th, i this cs s 0E F0 ( t =s 0 ) =EF 0 ( 0 ( t =s 0 )) = Thrfor th symptotic cicy of th MM-stimt with rspct to th orml coditiol mximum liklihood stimt wh th iovtios hv distributio F 0 is EFF ( ; F 0 ) = EF 0 ( 0 ( t =s 0 )) s 0 E F : (6) 0 ( t =s 0 ) Choosig covitly w c mk this cicy s clos to o s dsird for th cs tht F 0 is orml. Rmrk 3. Th rltiv cicy of th MM- d BMM- stimts giv by (6) is th sm th th o of th M-stimts of loctio with rspct to th m. This implis th wll-kow fct tht M-stimts r robust for iovtio outlirs, tht is wh y t ; t ; corrspod to prfctly obsrvd ARMA modl, but th distributio F 0 of t is hvy tild. Rmrk 4. Wh E( t ) =, th rt of covrgc of M-stimts of d my b lrgr th =, d th symptotic distributio o-orml. This cs ws studid by svrl uthors. S for xmpl Dvis, Kight d Liu [8] d Dvis [9]. Rmrk 5. Thorms -5 us P3 oly to gurt tht for ll ; th fuctio g()= E((( t )=) hs uiqu miimum t = 0: If g() hs uiqu miimum t 6= 0; th th stimts of d r still cosistt, but th stimt of will covrg to 0 +. Filly from Thorms 4 d 5 w driv th followig Thorm. Thorm 6. Suppos tht th ssumptios of Thorm 5, P4 d P5 hold. Th ( p) = ( b M 0 ) covrgs i distributio to N(0; D) distributio, whr D is d d i (5): Not tht th ssumptios of Thorms d 4 iclud coditio P4. Howvr this coditio is ot strictly cssry d is icludd oly to simplify th proofs. All th symptotic thorms of this Sctio ssum tht th procss is ARMA modl. W cojctur tht similr rsults, cosistcy d symptotic ormlity hold wh th obsrvtios follows BIP-ARMA modl. Th mi di culty to prov ths rsults is to show tht th distributio of b t(; ) is symptoticlly sttiory.

12 5 Computtio W will discuss hr how to comput th MM-stimt. W strt computig th stimts of stp, b S d b b S: Accordig to (5) w c writ S ( () ) = P t=p+ r t (); whr r t () = S ( () ) ( p) = = b= t () S ( ()) Th to comput b S w c us y o-lir lst squrs lgorithm, for xmpl Mrqurd lgorithm. Similrly w c trsform th miimiztio of S ( b (;b (; ) )) i o-lir lst squrs problm. Not tht o-lir lst squrs lgorithms rquir good strtig poit. Sic th fuctios w r miimizig r o-covx d thy my hv svrl locl miim, th choic of th strtig poit is crucil. If th modl hs fw prmtrs (.g., p + q 3), o wy to obti th strtig poit is to grt grid of vlus of th prmtr d choos s iitil stimt th o miimizig th objctiv fuctio. Not tht th cs of p + q 3 is vry frqut i th cs of ARMA pplictios, whr th us of prsimoious modls is rcommdd. Bico t l. [] gv lgorithm to comput highly robust strtig poit wh thr r mor prmtrs. I th scod stp, to comput b M d b b M w c us Mrqurd lgorithm usig similr id d tkig s iitil stimt th bst stimt computd i stp. I our simultios th stimts wr d d tkig 8 < (x) = : 0:5x if jxj 0:00x 8 0:05x 6 + 0:43x 4 0:97x + :79 if < jxj 3 3:5 if jxj > 3; (x) = (x=0:405) d = 0 :Not tht d r smooth fuctios which r qudrtic i th itrvls ( 0:8; 0:8) d ( ; ) rspctivly. Th fuctio ws chos so tht if w tk b = mx = th th scl is cosistt to th stdrd dvitio for orml smpls Not tht is rdscdig fuctio. For ttig ARMA(,) modl to 000 obsrvtios usig MATLAB progrm, with iitil solutio computd with grid of 0 vlus i ch prmtr, th computig tim of BMM-stimt with th choics of i ; i = ; d giv bov is pproximtly 0 scods i PC computr with AMD Athlo.8 GHz procssor. For ttig AR(3) modl udr th sm coditios, th computig tim is miut 0 scods. 6 Robustss proprtis Svrl robustss msurs c b usd for stimts of tim sris prmtrs. Hmpl [] itroducd th i uc curv to msur th robustss of :

13 stimt udr i itsiml outlir cotmitio i th frmwork of i.i.d. obsrvtios. Küsch [5], Mrti d Yohi [8] d Mcii, Rochtti d Troji [6] giv grliztios of th i uc curv for stimtig tim sris prmtrs. Howvr, bcus of its i itsiml chrctr, th i uc curv my ot b good msur of th robustss wh thr is positiv frctio of outlir cotmitio. For xmpl, it c b provd tht for vry smll mout of cotmitio th MM- d BMM-stimts symptoticlly coicid d thrfor thir i uc curvs lso coicid. Howvr, w will s blow i this Sctio d i Sctio 7 tht th BMM-stimt is mor robust th th MM-stimt. I uc fuctios for th M-stimts of ARMA modls c b foud i Mrti d Yohi [8]. A mor rlibl msur of th robustss of stimt to cop with positiv frctio " of cotmitio is th symptotic mximum bis. Cosidr fmily of " cotmitd procss z "k t = ( " t )x t + " t w k t (7) s i (3) whr k K d (x t ; " t ; wt k ) is sttiory. Suppos lso tht th distributio of th ucotmitd procss y t dpds o prmtr R j. As xmpl w c cosidr th fmily of dditiv outlirs modls which is obtid tkig wt k = x t + k, with k R Suppos tht for ch smpl siz w hv stimt b of d lt b (L) = (b (L); :::; b j (L)) b th lmost sur limit of b wh pplid to procss with distributio L: Th bis of th i-th compot b wh pplid to z t "k s d d i (7) is B(b i ; ;"; k) = jb i (L(z "k t )) i j; whr L(z t "k ) dots d distributio of th procss z t "k : Th mximum symptotic bis of th i-th compot is d d by MB(b i ; ;") = kk B(b i ; ;"; k): W hv pproximtly computd th mximum bis curvs of th MM- d BMM-stimts for Gussi AR() d MA() modls with dditiv outlirs (w k t = x t + k) d whr th " t r i.i.d. Broulli vribls. To simplify th computtio w limit th itrcpt from ths modls by ssumig it to b kow d ull. Th symptotic vlu of th stimt is pproximtd usig smpls of siz W foud tht for smpls siz lrgr th 0000 th chgs i th stimt r gligibl. I Figur w show th bis curvs of th MM- d BMM-stimts for th AR() modl with = 0:5 d " = 0:. I Figur w show th mximum biss curvs for th MM- d BMM-stimts udr th sm modl I Figur 3 w show th mximum bis curv for th BMM-stimt udr MA() modl with prmtr = 0:5. 3

14 bis MM BMM k Figur : Bis curv for th AR() modl with = :5 d 0% of dditiv outlirs whr k is th outlir siz mximum bis MM BMM prctg of cotmitio Figur : Mximum Bis for th AR() modl with = :5 4

15 mximum bis MM BMM prctg of cotmitio Figur 3: Mximum Bis of th BMM for th MA() modl with = :5 I both css w obsrv tht th BMM-stimt hs smllr mximum bis th th MM-stimt. W lso obsrv tht th bhvior of th MMis di rt from th BMM-stimt. Aftr th cotmitio is lrgr th som vlu " th mximum bis of th MM-stimt is costtly qul to th vlu of th stimtd prmtr. This ms tht th symptotic vlu of th stimt bcoms 0 idpdtly of th tru vlu of th prmtr. This vlu " corrspod to th brkdow poit otio proposd by Gto d Lucs []. For th AR() modl th vlu " dpds o. For th MA() modl " = 0: Istd, th bhvior of th BMM-stimt is di rt d pprtly th stimt dos ot brk dow. A surprisig ftur of its mximum bis curv is tht for vry lrg " th mximum bis strts dcrsig. This c b xplid s follows: wh " is lrg, th probbility of obtiig ptch of two or mor outlirs icrss. Th ct of ptch of outlirs is to icrs th corrltio of th sris d thrfor, i th cs of th AR() modl with positiv d MA() with gtiv it prvts tht th prmtr furthr pproximts to zro for outlirs with xd siz k.w lso computd th mximum biss curvs for othr vlus of prmtrs d d th rsults wr similr. W cojctur tht th robust bhvior of th BMM-stimt udr dditiv outlir cotmitio lso holds wh w obsrvd y cotmitd procss z t " s giv i (3). Th rso is tht sic th BIP-ARMA modl icluds built-i ltrig to comput th rsiduls, smll frctio of outlirs will ct oly smll frctio of rsiduls. Thrfor, sic th loss fuctio of th BMM-stimt is boudd, th stimt will ot b lrgly ctd by smll frctio of lrg rsiduls. W comput mximum bis curvs for th cs 5

16 Estimt AR() M A() = 0:5 = 0:5 MLE 0:07 0:0036 0:00 0:004 MM 0:08 0:0045 0:0 0:0046 BMM 0:08 0:004 0:0 0:005 CTC B 0:07 0:0036 0:0 0:004 CTC 3 0:07 0:0036 0:0 0:0044 FTAU 0:09 0:0054 0:0 0:0065 Tbl : MSE of th AR() d MA() modls without outlirs of rplcmt outlirs (wt k dditiv outlirs. = k) obtiig similr rsults tht for th cs of 7 A Mot Crlo Study W hv prformd Mot Crlo study to compr svrl stimts for ARMA modls. W hv simultd thr Gussi sttiory ARMA modls cosidrig th cs tht th sris do ot coti outlirs d th cs tht th sris hv 0% of qully spcd i tim dditiv outlirs. Th smpl siz i th simultios is 00 d th Mot Crlo study ws do with 500 rplictios. Th stimts cosidrd i this study r (i) th orml coditiol mximum liklihood stimt (MLE), (ii) MM-stimt whr th rsiduls r computd s i rgulr ARMA modl (MM), (iii) MM-stimt whr th rsiduls r comprd with th os of BIP-ARMA modl (BMM), (iv) stimt bsd o th digostic procdur proposd by Chg, Tio d Ch [5] d which is furthr dscribd i Ch d Liu [6]. Th cuto poit for outlir rjctio is chos by th Bofrroi iqulity s c = ( (0:05=)); whr is th N(0; ) distributio fuctio. W dot this stimt by (CTC B ). (v) Th sm s i (iv) but th cuto poit is c = 3 (CTC 3 ). (vi) Th tu ltrd stimt proposd by Bico t l. []. W dot this stimt by (FTAU). Th stimts MM d BMM r bsd o th fuctios d d dscribd i Sctio 5 I Tbl w show th MSE for th six stimts wh th obsrvtios com from AR() d MA() modl without outlirs. Tbl show th MSE of th sm stimts for ARMA(,) modl without outlirs. Th rltiv cicy with rspct to th MLE vris from 80% to 90% for th stimt BMM i th cs of d, from 80 % to 94% for th stimt MM, is prcticlly 00% for th CTC stimts d vris from 65% to 84% for th FTAU. Th cicy of ll th stimts of is vry high. I Tbls 3 d 4 w show th M Squr Error of stimtio (MSE) of th six stimts d th thr modls with 0% of dditiv outlirs of siz 6

17 Estimt = 0:5 = 0:5 MLE 0:045 0:006 0:006 MM 0:050 0:0073 0:0075 BMM 0:05 0:0069 0:0075 CTC B 0:045 0:006 0:006 CTC 3 0:047 0:006 0:0064 FTAU 0:054 0:0074 0:008 Tbl : MSE for th ARMA(,) modls without outlirs AR() MA() Estimt = 0:5 = 0:5 MLE 0:89 0:03 0:78 0:8 MM 0:04 0:085 0:05 0:5 BMM 0:0 0:04 0:056 0:05 CTC B 0:85 0:03 0:74 0:30 CTC 3 0:48 0:096 0:36 0:5 FTAU 0:03 0:0 0:00 0:03 Tbl 3: MSE of th AR() d MA() modls with 0 prct of qully spcd dditiv outlirs of siz 4 4 d i Tbls 5 d 6 w show th MSE of th six stimts d th thr modls with 0% of dditiv outlirs of siz 6. W obsrv tht for ll modls th stimt BMM of d bhvs much bttr th thos corrspodig to th stimts MM, CTC B d CTC 3. Th prformc of th stimts FTAU d BMM r comprbl. Th rrors of th MSEs show o ths tbls r smllr th 5% with probbility Howvr sic th ll th stimt wr computd with th sm smpls, th rrors of th di rcs btw th MSE of y two stimts r much smllr mkig comprisos possibl. Estimt = 0:5 = 0:5 MLE 0:04 0:03 0:99 MM 0:093 0:0 0:3835 BMM 0:088 0:07 0:060 CTC B 0:03 0:03 0:300 CTC 3 0:83 0:0 0:33 FTAU 0:08 0:08 0:040 Tbl 4: MSE of th ARMA(,) modls with 0 prct of qully spcd dditiv outlirs of siz 4 7

18 AR() MA() Estimt = 0:5 = 0:5 MLE 0:394 0:89 0:380 0:5 MM 0:08 0:3 0:06 0:59 BMM 0:09 0:0048 0:0 0:0065 CTC B 0:364 0:89 0:345 0:8 CTC 3 0:057 0:047 0:04 0:064 FTAU 0:08 0:0076 0:07 0:0 Tbl 5: MSE of th AR() d MA() modls with 0 prct of qully spcd dditiv outlirs of siz 6 Estimt = 0:5 = 0:5 MLE 0:40 0:070 0:374 MM 0:57 0:034 0:585 BMM 0:065 0:0 0:0 CTC B 0:393 0:069 0:378 CTC 3 0:8 0:04 0:334 FTAU 0:093 0:03 0:05 Tbl 6: MSE of th ARMA(,) modls with 0 prct of qully spcd dditiv outlirs of siz 6 8 A xmpl This xmpl dls with mothly sris of iwrd movmt of rsidtil tlpho xtsios i xd gogrphic r from Jury 966 to My 973 (RESEX). Th sris ws lyzd by Brubchr [3] d by Mrti, Smrov d Vdl [0], who idti d AR() modl for th di rcd sris y t = x t x t ; whr x t is th obsrvd sris. Tbl 7 displys th vlu of th stimts MLE, MM, BMM, CTC 3 d th FTAU togthr with th MAD-scl of th rsiduls. W c s tht th stimtd vlus of th prmtrs of th MLE d th CTC 3 r quit di rt from th robust stimts MM, BMM d FTAU. Th stimt CTC B givs th sm rsult s CTC 3 (it dtcts th sm outlirs) d it is omittd from th tbl. Figur 4 shows th dt y t obtid di rtitig th obsrvd dt s y t = x t x t d th cld vlus s i (0), which r s to b lmost coicidt xcpt t outlir loctios. 8

19 Estimts MAD MLE :69 0:48 0:7 :70 MM :8 0:34 0:3 :43 BMM :74 0:4 0:36 :4 CTC 3 3:44 :4 0:74 :86 FTAU :7 0:7 0:49 :0 Tbl 7: Estimts of th prmtrs of th RESX sris diffrcd RESEX idx Figur 4: Di rcd RESEX Sris: Obsrvd (solid li) d Filtrd (circls) vlus 9 Cocludig rmrks W hv prstd two fmilis of stimts for ARMA modls: MM-stimts d BMM-stimts. Th BMM-stimts uss mchism tht voids th propgtio of th full ct of th outlirs to th subsqut rsidul iovtios. To mk this mchism comptibl with cosistcy wh th tru modl is ARMA, w cosidr two stimts: o is obtid ttig rgulr ARMA modl d th othr ttig BIP-ARMA modl, whr th propgtio of th ct of outlirs is boudd. Th, th stimt which ts bttr to th dt is slctd. W hv show i Sctios 6 d 7 tht, t lst for dditiv outlirs, th BMM-stimts r much mor robust th th MM-stimts d quit comprbl with th FTAU-stimts. Th mi dvtg of th BMM-stimts ovr th FTAU-stimts is tht symptotic thory is ow 9

20 vilbl d this mks ifrc with BMM-stimts possibl. Th Mot Crlo rsults of Sctio 7 lso show tht th BMM-stimt comprs fvorbly with th stimt bsd o th Ch d Liu [6] digostic procdur. 0 Appdix Suppos tht w hv th i it squc of obsrvtios Y t = (:::; y t k ; ::::; y t ; y t ) grtd by sttiory d ivrtibl ARMA(p; q) procss up to tim t with prmtr 0. Giv y = (; ;) such tht th polyomils (B) d (B) hv ll th roots outsid th uit circl, lt us d t () = (B)(B)(y t ): (8) Th t ( 0 ) = t d t () s stisfy th followig rcursiv rltioship t () = y t i (y t i ) + i= qx i t i(): I th cs tht t hs it rst momt, w hv tht t () = y t E(y t jy t ), whr th coditiol xpcttio is tk ssumig tht th tru vlu of th prmtr vctor is : It is strightforwrd to driv th followig formuls for th for th rst d scod drivtivs of t t i = (B)(y t i ); i p; (9) t j = (B) t j() (30) = (B)(B)(y t j ), j t = i= i ; (3) i t j = 0; i p; j p; t j = (B)(y t j i ), i p; j q; (33) 0

21 @ t j = 3 (B)(B)(y t i j ), i q; j q; t = P q i= ; i p; (35) i t = i= ; i q; (36) ( i ) t = 0: (37) W will us th followig ottio. Giv fuctio g(u) : R k! R;w d rg(u) s th colum vctor of dimsio k whos i-th lmt is r i g(u) =@g(u)=@u i d r g(u) is th k k mtrix whos (i; j) lmt is r ijg(u) g(u)=@u j : W strt provig th followig Lmm. Lmm Assum y t stis s P. Th for y d > 0 w hv: (i) Thr xists sttiory procss W 0t such tht B0[ j t ()j W 0t d if E(jy t j ) < ; th E(jW 0t j ) < : (ii) Thr xists sttiory procss W t such tht B0[ kr t ()k W t d if E(jy t j ) < ; th E(jW t j ) < : (iii) Thr xists sttiory procss W t such tht B0[ r t () W t ;whr jjajj dots th l orm of mtrix A:If E(jy t j ) < ; th E(jW t j ) < : Proof. Sic B 0 [ d; d]; usig (8), it is sy to show tht thr r positiv costts k 0, k d 0 < < ;such tht B 0[ j t ()j k 0 + k X i=0 i jy t i j : P D W 0t = k 0 +k i=0 i jy t i j : Th by Lmm of Yohi d Mro [7], W 0t is it. To prov tht if E(jy t j ) < th E(W0t) < ; it is ough to show tht E X i jy t i j! < ; i=0

22 d this follows from E X X X i jy t i j! i+i E(jy t i y t i j) i=0 i =0 i =0 X X i =0 i =0 i+i E = (jy t i j )E = (jy t i j )! X X X = E(jy t j ) i+i = E(jy t j ) i : i =0 i =0 i=0 Thrfor (i) follows. From (9)-(3) d (3)-(37) w c prov (ii) d (iii) rspctivly usig th sm rgumts s i th proof of (i). I th xt Lmm w prov th Fishr Cosistcy of th S-stimt wh w hv ll th pst obsrvtios. Lmm Assum tht y t stis s coditio P with iovtios stisfyig P3. Assum tht is boudd fuctio stisfyig coditio P, d s() by E t () = b: (38) s() Th if B d 6= 0 w hv s 0 = s( 0 ) < s(). Proof. Lt = (; ;) 6= 0 = ( 0 ; 0 ; 0 ):W hv whr t () = (B)(B)(y t ) = (B)(B)(y t 0 ) + (B)(B)( 0 ) =!(B) t + c( 0 ); (39)!(B) = (B) 0 (B)0 (B)(B) X = +! i B i i= d c = P p i= i P q i= i 6= 0: Put X t () =! i t i + c( 0 ): (40) i=

23 Th, from (39) w obti E t () =E s ( t + t ()) : 0 s 0 Lt S(p; q) t + p S(p; q) = E ; q d obsrv tht S(p; q) is dcrsig i q: Lmm 3. of Yohi [5], tht if P d P3 hold th for ll p 6= 0 d q 6= 0 showd S(0; q) < S(p; q): (4) Th, sic t d t () r idpdt w hv E t () = E(S( t (); s 0 )) s 0 E(S(0; s 0 )) t = E = b s 0 d th qulity holds if d oly if t () = 0.s.. Bcus of th idti bility of th ARMA modl, this occurs if d oly if = 0 : Th 6= 0 implis E t () = E(S( t (); s 0 )) > b s 0 d thrfor w hv s() > s 0 = s( 0 ): Th xt two Lmms r vry wll kow proprtis of di rc qutio. Thy r provd for sk of compltss. Lmm 3. Cosidr th di rc qutio z t = kx i z t i + d (4) i= d ssum tht P k i= i 6= : Lt z t ; t t 0 + b solutio of (4) corrspodig to giv iitil vlus z t0 ; z t0 ; z t0 k. Th it holds jz t z j A() t t 0 (jzt0 z j + ::: + jz t0 k+ z j ) = ; (43) whr z = d=( P k i= i), 0 3 ::: k k 0 0 ::: 0 0 A() = B 0 0 ::: 0 ::: ::: ::: ::: ::: ::: ::: 0 C A (44) 3

24 d jjajj dots th l orm of th mtrix A. Proof. It is ough to show th Lmm for t = t 0 + : Cll z t = (z t ; z t ; :::z t k+ ) 0 ; z = (z ; :::; z ) 0 ; d = (d; 0:::0) 0 : Th it is sy to chck tht z t = A()z t + d; z = A()z + d d th Thrfor w hv tht z t z = A()(z t z ): jjz t z jj ka()k jjz t z jj d th (43) holds for t = t 0 +. This provs th Lmm. Giv = ( ; :::; k )R k ; lt (x) b th polyomil (x) = kx i x i : i= Lmm 4 Giv " > 0; lt C " b th st of ll = ( ; :::; k ) such tht th polyomil (B) = B ::: k B K hs ll its roots hv modulus lrgr or qul th + " d lt A() b th mtrix s d d i (44). Th, thr xists 0 < < d positiv costt C such tht Proof. Usig th Jord coicl form w c writ A() t C t : (45) C " A() = B ()(D() + N())B(); whr D() is digol mtrix tht hs th igvlus of A() i th digol, d N() is ilpott mtrix of 0 0 s d 0 s stisfyig tht N() k is th ull mtrix. d such tht jjn()jj :MorovrN() d D() commut. Put C() = D() + N(); th A() t = B t ()C t ()B t (): (46) Sic N() d D() commut d N() k is th ull mtrix, w gt C t () C " C " kx j=0 t kn()k j kd()k t j (47) j 4

25 Lt i (); i k; b th roots of (B):Sic th igvlus of A() r = i (); i k; from (47) w obti C t () C " C " kx + " + " t j t (48) j + " j=0 t k+ k+ t : (49) + " For 0 t k, w c writ C t () C " tx C " j=0 tx j=0 t j t kn()k j kd()k t j ( + ") t j t : (50) Th from (49) d (50) it is sy to prov tht thr xists C such tht C t () C t ; : (5) C " whr = =( + "). Put C = C" kb()k d C 3 = C" B () d C = C C C 3 : Th, from (46) d (5) w gt (45). Lmm 5 Udr th ssumptios of Thorm, for y d > 0 w hv, j lim! B 0[ js ( () ) s()j = 0.s.. Proof. It is sy to show tht s() is cotiuous d positiv. Lt h = if s(), h = s(): B 0[ B 0[ Th h > 0 d h < : From Lmm of Mulr d Yohi [] w hv tht lim! B 0[ ;c[h =;h ] t=p+ ( t ()=c) p E t () c = 0.s.. (5) Sic is boudd d cotiuous, thr xists C > 0 such tht for t p +, 5

26 B 0[ ;c[h =;h ] C t=p+ B 0[ t=p+ t () c t () c j t () t ()j : (53) Put g t () = t () t (); it is sy to vrify tht for t p + ; qx g t () = i g t i= i () with g q+ i () = q+ i (); i q: Th by th d itio of B 0 d Lmms 3 d 4 thr xists 0 < < d positiv costt C such tht for t p + ; B 0[ jg t ()j C t B 0[ d by Lmm (i) w gt for t p + ; B 0[ whr Z is th rdom vribl C ( lim! j t () t ()j C t ( ( qx i= q+ i()) = ; qx W0;q+ i) = i= t Z; (54) qx W0;q+ i )= : i= Thrfor from (5), (53) d (54) w hv B 0[ ;c[h =;h ] t=p+ ( t ()=c) p E t () c = 0.s.. Lt 0 " h = d d g () = E t () ; g () = E t () s() + " : s() " (55) By (38) w hv tht g () < b d g () > b: Sic B 0 is compct st d g d g r cotiuous, w hv = B 0[ g () < b; = if g () > b: B 0[ 6

27 Lt = mi (b ; b) : From (55) thr xists 0 such tht for ll 0 ; t () p E t () c c : B 0[ ;c[h =;h ] t=p+ Thrfor, for ll 0 w gt, if B 0[ p t=p+ t () s() " b + : Similrly w hv, B 0[ p t () + s() + " b t=p+ ; d hc from th mootoicity of (juj) w gt, B 0[ js ( ()) s()j ": This provs th Lmm. Lmm 6. Udr th ssumptios of Thorm, thr xists d > 0 stisfyig lim if if S ( ()) > s 0 +.s..! jj>d;(;)b 0 Proof. Giv = (; ; ) with (; ) B 0, lt us cll # t () = t () t (; ;0). From (3), it is sy to show tht # t () stisfy for t p + th di rc qutio,! qx # t ()= i # t i () + i (56) i= with iitil coditios # p+ i () = 0; i q. Morovr, it is sy to vrify tht # t () = # t (; ;): Usig th d itio of B 0 ; thr xists > 0 d K > 0 such tht for ll (; ) B 0, i= qp i i= K qp : (57) i i= 7

28 Th, by Lmms 3 d 4 thr xists 0 < < d K > 0 such tht for t p + ; qp i i= # t (; ;) (;)B 0 qp t K ; i d so thr xists t 0 such tht for t t 0 ; w hv qp i i= # t (; ;) (;)B 0 qp : (58) Th, from (57) d (58) w gt, i= i i= if j# t ()j jj : (59) (;)B 0 From (54) thr xists rdom vribl Z, d 0 < < such tht for ll t p + it holds d by Lmm (i) w obti j t (; ;0) t (; ;0)j t Z (60) (;)B 0 j t (; ; 0)j W 0;t ; (6) (;)B 0 whr W 0;t is sttiory procss. Sic > b d lim x! (jxj) =, thr xist k 0 > 0 d > such tht for ll x stisfyig jxj k 0 it holds Lt m b such tht (x) b: (6) P (W 0;t < m= ) > ; (63) m k = mx s 0 + ; k 0 (64) d d 4(s 0 + )k=: Th usig (59) w gt for ll t t 0 if j# t ()j (s 0 + )k: (65) jj>d;(;)b 0 Sic stis s proprty P, it holds 8

29 if jj>d;(;)b 0 p p t=p+ t () s 0 + if t () s 0 + I(A t )I(B t ); (66) t=p+ jj>d;(;)b 0 whr A t = fw 0t < m=g d B t = f t Z < m=g: From (60) d (6) w c writ, j t ()j j# t ()j (W 0;t + t Z): Th, from (64) d (65) w obti for ll t t 0 tht if j t ()j > k(s 0 + ) jj>d;(;)b 0 W 0;t + C t Z < k(s 0 + ) A t \ B t : (67) Sic 0; d (juj) is o dcrsig, from (66) d (67), w gt p t=p+ if jj>d;(;)b 0 t () s 0 + p (k) I(A t )I(B t ): (68) t=t 0 With probbility, thr xists t t 0 such tht I(B t ) = for ll t t :Th (k) p I(A t )I(B t ) (k) p t=t 0 (k) p t=t + t=p+ I(A t ) I(A t ) t p p (k): (69) Sic W 0t is rgodic d sttiory procss, from (63) w hv lim! p I(A t ) = P (A t ) > : t=p+ Th, from (6), (64), (66), (68) d (69) w gt lim if if! jj>d;(;)b 0 p t=p+ t () (k) b s 0 + 9

30 d th Lmm follows. Proof of Thorm. Tk " > 0 rbitrrily smll d lt d b s i Lmm 6. By th domitd covrgc thorm it is sy to show tht s() is cotiuous. Th by Lmm, thr xists 0 < < such tht mi s() s 0 + : B 0[ ;jj 9 jj" Thrfor by Lmm 5, thr xist such tht for mi S () s 0 + = B 0[ ;jj 0 jj" d S ( 0 ) s 0 + =4: Morovr by Lmm 6, thr xists such tht for if S ( () > s 0 +.s.. jj>d;(;)b 0 Thrfor, for mx( ; ) it holds tht jj b S 0 jj < " d this provs th Thorm. Th xt thr Lmms will b usd to prov Thorm. Lmm 7. Assum tht y t stis s coditio P. Giv d > 0 d > 0, thr xist costts C > 0 d 0 < < such tht B 0[ b t (;) y t C + t 0<! jy i j ; t p + : Proof. Giv B 0 [ d; d] d, lt us d for t p +, i= rx t (;) = b t(;) y t, D t (;) = ( i i ) b t i(;)= (70) d t (;) = y t for t p. From (), t (;) stisfy for t p + th rcursiv qutio, i= t (;) = D i t i (;) + D t (;) ( i= i ); t p + : (7) i= t (;) = ( t (;); t (;); :::; t p+ (;)) 0 ; D t (;) = (D t (;) ( i ); 0:::0) 0 30 i=

31 d A() s d d i (44). Th, t (;) = A() t (;) + D t (;) d it is sy to show iductivly tht for t p + ; t Xp t (;) = A() t p p + A i ()D t i (;): (7) Sic B 0 is compct ; is boudd, d jj d it follows tht thr xists costt D such tht for t p + ; i=0 B 0[ B 0[ 0< 0< kd t (;)k jd t (;)j + ( i ) jj D + d: (73) i= Th, from (6), (7), Lmms 3 d 4 w hv tht thr xists positiv costts C, d 0 < < such tht for t p + ; B 0[ 0< k t (;)k C t k p k + C (D + d) : (74) This provs th Lmm. Lmm 8. Udr th ssumptios of Thorm, giv d > 0; thr xists > 0 such tht lim if! if S b (; b(; )) > s 0 +.s.. B 0[ Proof. Sic by (9), b(; ) b y, usig Lmm 7 w c d costt C > 0 d 0 < < such tht! b t(;b(; )) y t C b y + t jy i j : B 0[ Sic lim! b y = y.s., with probbility o thr xists t 0 lrg ough such tht for ll t t 0 ; b y + t i= jy i j y.s.. (75) i= Th, cllig D = C y w hv for ll t t 0 ; B 0[ b t(;b(; )) y t D: (76) 3

32 W c writ y t = 0 + t + v t ; whr v t is sttiory procss tht dpds o t 0; t 0 < t: Sic y t is ot whit ois d th distributio of t is uboudd, w hv tht v t hs uboudd distributio too. Put u t (; ) = 0 + v t + ( b t(;) y t ): (77) Th, from (), u t (; ) lso dpds o t 0; t 0 < t: W c writ b t(;) = y t + ( b t(;) y t ) = t + u t (; ); (78) d obsrv tht (76) d (77) imply tht for t t 0 w hv if B 0[ ju t (; )j fjv t j > D + j 0 j + g: (79) Sic v t hs uboudd distributio, w hv tht = P (jv t j D + j 0 j + ) > 0: Accordig to d itio of s 0 i Thorm, w hv tht E ( ( t =s 0 )) = b d i Lmm 3. of Yohi [5] it is show tht for ll u 6= 0. This implis tht Th, sic E ( (( t + u)=s 0 ) ) > b if E ( (( t + u)=s 0 ) ) > b: juj t ( )E + if s E t + u > b; 0 juj s 0 w c d > 0 such tht t ( )E + if s 0 + E t + u b + : (80) juj s 0 + Put d d t + u h(u) = E s 0 + b R t (; ) = t (;) s 0 + = t + u t (; ) s 0 + h(u t (; )) h(u t (; )) 3

33 It is sy to vrify tht R t (; ) is boudd mrtigl di rc squc. Th, by th lw of lrg umbrs for mrtigl di rcs, s for istc Thorm 0.0 of Dvidso [7], w gt tht p R t (; ) = 0.s.. (8) t=p+ Usig compctss rgumt for ll " > 0 w c d ( i ; i ; i ); i m 0 ;with i B 0 [ d; d]; i y, such tht if w d V i = f(; ) : jj i jj + j i j i g; w hv tht [ m0 i= V i B 0 [ d; d] [0; y ] d (R t (; ) R t ( p i ; i )) " (;)V i t=p+ This lst iqulity d (8) imply tht lim p! B 0[ ; y t=p+ R t (; ) ".s., d sic this hold for ll " > 0; w gt lim! B 0[ ; y p t=p+ R t (; ) = 0.s.. (8) Put Th, w gt = p I(jv t j D + j 0 j + ): t=p+ B 0[ ; y p t=p+ d thrfor sic! ;.s. by (80) w hv lim if! B 0[ ; y p d th from (8) w hv lim! if B 0[ h(u t (; )) ( )h(0) + if juj h(u); t=p+ h(u t (; )) ( )h(0) + if juj h(u) ; y t=p+ b +.s. b t (;) b + : s

34 Th, sic b(; ) b y by (75), th Lmm follows. Lmm 9. Udr th ssumptios of Thorm, thr xists d > 0 such tht, lim if if S ( b (;b(; )) ) > s 0 +.s..! jj>d;(;)b 0 Proof. Dot by A k i () d Ak i;j () th i-th rw d (i; j) lmt of th mtrix A k () rspctivly, whr A() is d d i (44). Th, ccordig to (7) w hv b t(;b(; )) y t t Xp = A t p () p + A i ;() D t i (; b(; )) ( i=0! i ) ; i= whr p d D t (; b(; )) r d d i (70). Th, b t(;b(; )) t A t p () p Xp A i ;()D t i (; b(; ) i=0 t Xp + A i ;()( i ) jy t j i=0 A t i=0 p () kp k t Xp + A i ;()( i= t Xp i=0 A i () jdt i (;b(; ))j i ) i= jy t j : (83) From Lmm 4 thr xists positiv costt C d 0 < < such tht for t p + A() t p k p k + t Xp i=0 t Xp C t k p k + i jd t i=0 A i () jd t i (;b(; ))j i (;b(; ))j Sic B 0 is compct, is boudd, d ccordig to (9), 0 < b(; ) b y ; w hv tht thr xists positiv costt C such tht for t p + ;! : B 0[ jd t (;)j C b y 34

35 d th thr xists positiv costt C such tht A() t t 0 kp k + B! C b y + t jy t j D for t p + ; t= t Xp i=0 A i () jdt i (;b(; ))j : (84) with p = 0; p t (;) = i t i (;) + ( i ) i= i= = 0; ; = 0. It is sy to show tht t (;) = ( i= t Xp i ) A i ;(): Th, from Lmm 3 d usig rgumt similr to th o usd to prov (59) i Lmm 6, thr xists " > 0 d t 0 such tht for ll t t 0 ; Th, from (83), (84) d (85), for ll t t 0 ; i=0 if j t (;)j " jj : (85) (;)B 0 if b t (;b(; ) " (;)B 0 jj jy tj C b y + t! jy t j : (86) Sic > b d lim x! (jxj) =, thr xists k 0 d > such tht for ll jxj k 0 ; (x) b: (87) Sic lim! b y = y.s. d lim t! t P p i= jy ij = 0.s., with probbility o thr xists t t 0 such tht for ll t t ; b y + t P p i= jy ij y : Tk k such tht th st R t = fjy t j k C y g (88) stis s P (R t ) =:Tk k = mx(k = (s 0 + ) ; k 0 ) d d such tht d > 4k(s 0 + )=": Th, from th d itio of k, (86) d (88) i R t ; for ll t t if b t (;b(; )) > k(s0 + ): (89) jj>d;(;)b 0 Sic stis s proprty P, w hv, t= 35

36 if jj>d;(;)b 0 p p t=t + b t (;b(; )) s 0 + if b t(;b(; )) s 0 + I Rt : (90) t=p+ jj>d;(;)b 0 From (87) d (89) d sic R t is sttiory d rgodic with P (R t ) = w gt lim if! p lim if (k)! p = lim if (k)! p t=t + t=t + t=p+ I Rt I Rt if jj>d;(;)b 0 b t(;b(; )) s 0 + lim (k)! p (t p) b.s. I Rt d th from (90) w hv, lim if if! jj>d;(;)b 0 p t=p+ b t (;b(; )) b.s.. s 0 + This provs th Lmm. Proof of Thorm. From Lmms 8 d 9 w hv tht thr xists > 0 such tht lim if! if B S ( b (;b(; ) ) > s 0 +.s.. But, by Thorm -(ii) w hv tht b S stisfy lim S ( ( b S )) = s 0.s..! This provs th Thorm. Th followig four Lmms will b usd to prov Thorm 3. Lmm 0 Assum tht y t stis s coditio P with iovtios stisfyig P3 d ssum tht stisfy coditio P with boudd. Lt us cll m() = E( ( t ()=s 0 ));th 0 = rg mi B m(). Proof. Similr to Lmm. Lmm Assum tht y t stis s coditio P d coditio P. D 36

37 Th, w hv; lim M() = p! B 0[ M () t=p+ t () s : (9) E t () = 0.s. s 0 for ll d > 0: Proof. By th domitd covrgc thorm, it is sy to show tht M(; v) = E t () v is cotiuous. Th, giv > 0; thr xists with 0 < < s 0 such tht B 0[ ;[s 0 ;s 0+] E t () v E t () < s 0 : (9) Sic t () is sttiory d cotiuous d boudd, by Lmm 3 of Mulr d Yohi [] w hv whr lim! (;v)c 0 p t=p+ t () E t () = 0.s., (93) C 0 = f(;v) : B 0 [ d; d]; [s 0 ;s 0 + ]g By Thorm, lim! s = s 0.s.. Th, with probbility o thr xists 0 such tht for ll 0 w hv s [s 0 ;s 0 + ] d B 0[ ;[s 0 ;s 0+] p t=p+ t () E t () < : Hc, from (9) d (94) w hv tht for 0 ; t () B 0[ p s E t () s 0 t=p+ (94) B 0[ ;[s 0 ;s 0+] p t=p+ t () E t () 37

38 + B 0[ ;[s 0 ;s 0+] E t ( E t ( < ": s 0 This provs th Lmm. Lmm. Udr th ssumptios of Thorm 3, w hv lim! B 0[ jm () M ()j = 0.s.. Proof. W hv M () M () = p t=p+ t () s t () s : From (54), sic 0 is boudd d lim! s = s 0 > 0.s. thr xists k > 0, 0 < <, rdom vribl Z d 0 such tht for ll 0 t () s t () s k j t () t ()j k t Z. (95) Th, lim! B 0[ lim! kz p t=p+ p t=p+ t () s t kz lim! ( p)( ) t () s = 0. (96) This provs th Lmm. Lmm 3. Udr th ssumptios of Thorm 3, thr xists d > 0 d > 0 such tht lim if! if M () m( 0 ) +.s., jj>d;(;)b 0 whr m( 0 ) is d d i Lmm 0. Proof. Sic th iovtio t stisfy P3, th m( 0 ) = E ( ( t =s 0 )) <. Sic lim x! (jxj) = ; usig similr rgumts to thos usd i Lmm 6, w hv tht thr xists d > 0 d > such tht lim if if! jj>d;(;)b 0 p t=p+ t () s > m( 0 ).s.: (97) d so th Lmm follows. Proof of Thorm 3. Follows from Lmms 0-3 usig similr rgumts s thos usd i th proof of Thorm. 38

39 Th xt two Lmms will b usd to prov Thorm 4. Lmm 4. Udr th ssumptios of Thorm 3, for ll d > 0 thr xists > 0 such tht lim if! if M () b m( 0 ) +.s., B 0[ whr m( 0 ) is d d i Lmm 0. Proof. It is similr to th proof of Lmm 8. Lmm 5. Udr th ssumptios of Thorm 3, thr xists d > 0 d > 0 such tht lim if! if M() b m( 0 ) +.s., jj>d;(;)b 0 whr m( 0 ) is d d s i Lmm 0. Proof. Sic th iovtios t stisfy P3, th m( 0 ) = E ( ( t =s 0 )) < :Sic lim x! (jxj) = ; usig similr rgumts to thos usd i Lmm 9 w hv tht thr xists d > 0 d > ; lim if! if M() b > m( 0 ).s.. (98) jj>d;(;)b 0 d so th Lmm follows. Proof of Thorm 4 From Lmms 4 d 5 w hv tht thr xists > 0 such tht lim if! if B M b () m( 0 ) + : Thorm 3 implis tht lim! M ( b M ) = m( 0 ).s.. This provs th Thorm. Th xt v Lmms will b usd to prov Thorm 5. Lmm 6. Udr th ssumptios of Thorm 5, w hv whr ( p) = t=p+ r t ( 0 )! D N (0; V 0 ) ; s 0! V 0 = E r t ( 0 ) 0 r t ( 0 ) s : (99) 0 s 0 Proof. W c writ Sic is odd r t ( 0 ) = ( t =s 0 ) r t ( s 0 s 0 ): (00) 0 d th distributio of t is symmtric, 39

40 E t From Lmm (ii) d th fct tht E(y t ) < w gt s 0 V 0 = E(r t ( 0 )r t ( 0 ) 0 ) < : = 0: (0) Thrfor, from (00) d (0) d sic r t () dpds oly o Y t =(y t ; y t ; :::) for y colum vctor c 6= 0 i R p+q+ w hv tht c 0 r t ( 0 ) = c 0 r t ( 0 ) s 0 s 0 is sttiory mrtigl di rc squc. Th, pplyig th Ctrl Limit Thorm for Mrtigls (s Thorm 4.3, Dvidso [7]) w hv tht This implis tht ( p) = ( p) = t=p+ c 0 r t ( 0 )! D N(0; c 0 V 0 c): s 0 t=p+ r t ( 0 )! D N(0; V 0 ) s 0 provig th Lmm. Lmm 7. Udr th ssumptios of Thorm 5 w hv lim! ( p) = t=p+ r t ( 0 ) s r t ( 0 ) s 0 = 0 i probbility. Proof. Th proof is similr to th o of Lmm 5. i Yohi [5] for MM-stimts i th cs of rgrssio. W c writ, = ( p) = ( p) = t=p+ t=p+ r t ( 0 ) s ( t =s ) s r t ( 0 ) s 0 ( t =s 0 ) r t ( s 0 ): 0 (0) D for 0 v ; j p + q + ; 40

41 A ;j (v) = ( p) = t=p+ t (0:5 + v) s 0 r j t ( 0 ): Sic from Thorm lim! s = s 0.s. i ordr to prov th Lmm is ough to show tht A ;j (v) j p + q +, r tight. Usig Thorm.3 of Billigsly [] it will b ough to show th followig two coditios, (i) A ;j (0) is tight. (ii)for y 0 v v d y > 0 w hv tht thr xists costt k such tht P (ja ;j (v ) A ;j (v )j ) k (v v ) : (i) follows from Lmm 6. Lt us prov ow (ii). W c writ for j p + q +, Put G(; v) = : (0:5 + v) s 0 Th = = E (A ;j (v ) 0 p p A ;j (v ))! (G( t ; v ) G( t ; v ))r j t ( 0 ) A t=p+ E(B r C r B t C t ); (03) t=p+ r=p+ whr d B t = t (0:5 + v ) s 0 t (0:5 + v ) s 0 (04) C t = r j t ( 0 ): (05) Sic r t ( 0 ) dpds o Y t =(y t ; y t ; :::), w hv tht B t is idpdt of C r for ll r t: Morovr ll B t s r idpdt. Th if r < t, usig tht E(B t jy t ) = E(B t ) = 0; w obti Morovr E(B r B t C r C t ) = E(E(B r B t C r C t jy t )) = E(E(B t jy t )C t C r B r ) From (03), (04), (05), (06) d (07) w obti = 0: (06) E(B t C t ) = E(B t )E(C t ): (07) 4

42 E (A ;j (v ) A ;j (v )) t = E (0:5 + v ) s 0 t (0:5 + v ) s 0 Lt v < v < v :Th, usig th M Vlu Thorm w gt E(r j t ( 0 )) :(08) = E t (0:5 + v ) s 0 (v v ) E s 0 (0:5 + v)4 t 0 t (0:5 + v ) s 0! t (0:5 + v) s 0 : Th, sic 0 is boudd; t hs scod momt d s 0 > 0 w c coclud tht thr xists k 0 > 0 such tht t t E k 0 (v v ) : (09) (0:5 + v ) s 0 (0:5 + v ) s 0 Th sic E(yt ) < ; by Lmm (ii) w hv tht E (r j t ( 0 )) < : Th from (08) d (09) thr xists k > 0 such tht E (A ;j (v ) A ;j (v )) k (v v ) : Hc, (ii) follows from th Chbyshv s iqulity. Lmm 8. Udr th ssumptios of Thorm 5, w hv t ( lim! ( p) = r 0 ) s r t ( 0 ) s = 0.s.. Proof. W c writ t=p+ t ( r 0 ) s r t ( 0 ) s = t ( 0 ) t s s r t ( 0 ) s = t ( 0 ) s s (r t ( 0 ) r t ( 0 )) t ( 0 ) t + s s s r t ( 0 ) r t ( 0 ): (0) 4

43 By Lmm (ii), (9), (30) d (3), usig similr rgumts to thos ldig to th proof of (54) i Lmm 5, w c prov tht thr xists 0 < < d rdom vribl W such tht d thrfor lim! ( p) = kr t ( 0 ) r t ( 0 )k t W () t=p+ kr t ( 0 ) r t ( 0 )k = 0.s.. Th, sic is boudd d by Thorm (ii) w hv lim! s = s 0 > 0.s., lim! ( p) = s t=p+ t ( 0 ) (r t ( 0 ) r t ( 0 )) = 0.s.. () s Usig tht 0 is boudd d th M Vlu Thorm w c foud costt k > 0 such tht ( p) = s k ( p) = s t=p+ t=p+ t ( 0 ) s t s r t ( 0 ) j t ( 0 ) t ( 0 )j kr t ( 0 )k (3) From (54), thr xists 0 < < d rdom vribl Z such tht t=p+ j t ( 0 ) t ( 0 )j kr t ( 0 )k Z From (9)-(3) w hv tht W = P t=p+ t=p+ t (kr t ( 0 )k) : (4) t (kr t ( 0 ) k) is wll d d. Th, from (3), (4) d th fct tht lim! s = s 0.s., w obti lim t ( 0 ) t! ( p) = s s s r t ( 0 ) lim! t=p+ k ZW = 0.s.. (5) ( p) = s Th th Lmm follows from (0), () d (5). Lmm 9. Udr th ssumptios of Thorm 5 w hv for ll d > 0, (i) lim! B 0[ p t=p+ r t () s E r t () s 0 = 0.s., 43

44 whr jjajj dots th l orm of th mtrix A. (ii) E r t ( 0 ) = 0 t s 0 s E E r t ( 0 s 0 )r t ( 0 ) 0 0 d this mtrix is o-sigulr. Proof. Th proof of (i) is similr to th o of Lmm. W ow prov (ii). Usig tht E( ( t =s 0 )) = 0 d tht, ccordig to (9)- (37), both r t () d r t () dpd o Y t =(y t ; y t ; :::), w obti E r t ( 0 ) = s 0 s E 0 0 t s 0 E r t ( 0 )r t ( 0 ) 0 : Sic E( 0 ( t =s 0 )) > 0 d E r t ( 0 )r t ( 0 ) 0 is o-sigulr mtrix (s Bustos d Yohi [4]) w obti (ii). Lmm 0. Udr th ssumptios of Thorm 5, w hv, lim r t ()! B 0[ p s r t () s = 0.s. for ll d > 0: Proof. Put t=p+ V () =r t ()r t () 0 : Di rtitig r ( t ()=s ) w obti r t () s = ( t ()=s ) s r t ()+ 0 ( t ()=s ) (s ) V (): (6) Lt us d G t () = (s ) d d th 0 H t () = s t () s r t () s r t ()r t () 0 0 t () s t () r t ()r t () 0 s r t () t () s r t () ; r t () = G t () + H t (): (7) s 44