Technical Note: Auto Regressive Model

Transcription

1 Techncal Ne: Au Regressve Mdel We rgnall cmsed hese echncal nes afer sng n n a me seres analss class. Over he ears, we ve mananed hese nes and added new nsghs, emrcal bservans and nuns acqured. We fen g back hese nes fr reslvng develmen ssues and/r rerl address a rduc sur maer. In hs aer, we ll g ver anher smle, e fundamenal, ecnmerc mdel: he au regressve mdel. Make sure u have lked ver ur rr aer n he mvng average mdel, as we buld n man f he cnces resened n ha aer. Ths mdel serves as a crnersne fr an serus alcan f ARMA/ARIMA mdels. Backgrund The au regressve mdel f rder (.e. AR( )) s defned as fllws: Where x x x... x a a ~..d ~ N(0,) a s he nnvans r shcks fr ur rcess s he cndnal sandard devan (aka vlal) Essenall, he AR( ) s merel a mulle lnear regressn mdel where he ndeenden (exlanar) varables are he lagged edns f he uu (.e. x, x,..., x ). Kee n mnd ha x, x,..., x ma be hghl crrelaed wh each her. Wh d we need anher mdel? Frs, we can hnk f an AR mdel as a secal (.e. resrced) reresenan f a MA( ) rcess. Le s cnsder he fllwng sanar AR () rcess: x x a x x a ( L)( x ) a Nw, b subracng he lng run mean frm he resnse varable ( x ), he rcess nw has zer lngrun (uncndnal/margnal) mean. Techncal Ne Auregressve Mdel Sder Fnancal Cr, 04

2 0 Nex, he rcess can be furher smlfed as fllws: ( L)( x ) ( L) z a a z L Fr a sanar rcess, he a N N z ( L L L... L...) a L In sum, usng he AR() mdel, we are able reresen hs MA( ) mdel usng a smaller srage requremen. We can generalze he rcedure fr a sanar AR() mdel, and assumng an MA( ) reresenan exss, he MA ceffcens values are slel deermned b he AR ceffcen values: x x x... x a x... x x... x a LL L x a a (... )( ).. Once agan, b desgn, he lng run mean f he revsed mdel s zer Hence, he rcess can be reresened as fllws: ( LL... L ) z a a a ( x ) z... ( L)( L)..( L) L L L Techncal Ne Auregressve Mdel Sder Fnancal Cr, 04

3 B havng, {,,.., }, we can use he aral fracn decmsn and he gemerc seres reresenan; we hen cnsruc he algebrac equvalen f he MA( ) reresenan. Hn: B nw, hs frmulan lks enugh lke wha we have dne earler n he MA echncal ne, snce we nvered a fne rder MA rcess n an equvalen reresenan f AR( ). The ke n s beng able cnver a sanar, fne rder AR rcess n an algebracall equvalen MA( ) reresenan. Ths rer s referred as causal. Causal Defnn: A lnear rcess { X } s causal (srcl, a causal funcn f{ a }) f here s an equvalen MA( ) reresenan. Where: ( ) 0 X La La Causal s a rer f bh{ X } and{ a }. In lan wrds, he value f { X } s slel deenden n he as values f{ a }. IMPORTANT: An AR() rcess s causal (wh resec { a }) f and nl f he characerscs rs (.e. ) fall usde he un crcle (.e. ). Le s cnsder he fllwng examle: ( L)( x ) ( L) z a z z a Nw, le s re rganze he erms n hs mdel: z (z a) " z z a Techncal Ne Auregressve Mdel Sder Fnancal Cr, 04

4 z ( z a ) a z a a z z z... z " " " " " " " z a a a N N " N " " " N an an a a " " " N " a a a an " The rcess abve s nn causal, as s values deend n fuure values f { a } bservans. Hwever, s als sanar. Gng frward, fr an AR (and ARMA) rcess, sanar s n suffcen b self; he rcess mus be causal as well. Fr all ur fuure dscussns and alcan, we shall nl cnsder sanar causal rcesses. Sabl Smlar wha we dd n he mvng average mdel aer, we wll nw examne he lng run margnal (uncndnal) mean and varance. () Le s assume he lng run mean ( ) exss, and: Ex [ ] Ex [ ]... Ex [ ] Nw, subrac he lng run mean frm all uu varables: x ( x ) ( x )... ( x ) a ( x ) ( x ) ( x )... ( x ) a + (... ) Take he execan frm bh sdes: E[ x ] E[ ( x ) ( x )... ( x ) a] + (... ) 0 (... )... In sum, fr he lng run mean exs, he sum f values f he AR ceffcens can be equal ne. Techncal Ne Auregressve Mdel 4 Sder Fnancal Cr, 04

5 () T examne he lng run varance f an AR rcess, we ll use he equvalen MA( ) reresenan and examne s lng run varance. x... a LL L a (... ) a L L L... Usng aral fracn decmsn: c c c... a L L L Fr a sable MA rcess, all characerscs rs (.e. ) mus fall usde he un crcle (.e. ): ( c c... c ) ( c c... c ) L ( c c... c ) L... a Nex, le s examne he cnvergence rer f he MA reresenan: lm c c... c 0 k k k k Fnall, he lng run varance f an nfne MA rcess exss f he sum f s squared ceffcens s fne. Var[ x ] ( ( c c... c ) ( c c... c )... Tk k k k k + ( c c... c)...) ( cc... c) ( cjj) j Furhermre, fr he AR() rcess be causal, he sum f abslue ceffcen values s fne as well. k c jj k j Techncal Ne Auregressve Mdel 5 Sder Fnancal Cr, 04

6 Examle: AR() ( L ) a L ( L L...) a 4 6 Var[ ] (...) a Assumng all characersc rs ( ) fall usde he un crcle, he AR() rcess can be vewed as a weghed sum f sable MA rcesses, s a fne lng run varance mus ex. Imulse Resnse Funcn (IRF) Earler, we used AR() characerscs rs and aral fracn decmsn derve he equvalen f an nfne rder mvng average reresenan. Alernavel, we can cmue he mulse resnse funcn (IRF) and fnd he MA ceffcens values. The mulse resnse funcn descrbes he mdel uu rggered b a sngle shck a me. a a... k k k k k The rcedure abve s relavel smle (cmuanall) erfrm, and can be carred n fr an arbrar rder (.e. k). Ne: Recall he aral fracn decmsn we dd earler: c c c... a L L L Techncal Ne Auregressve Mdel 6 Sder Fnancal Cr, 04

7 We derved he values fr he MA ceffcens as fllws: ( c c... c ) ( c c... c ) L ( c c... c ) L... a In rncle, he IRF values mus mach he MA ceffcens values. S we can cnclude: () The sum f denmnars (.e. c ) f he aral fracns equals ne (.e. c ). () The weghed sum f he characerscs rs equals (.e. c ). () The weghed sum f he squared characerscs rs equals (.e. c ). Frecasng Gven an nu daa samle{ x, x,..., x }, we can calculae values f he mvng average rcess fr fuure (.e. u f samle) values as fllws: E [ ]... T a T T T... T T T T T T E [ ] E [ ]... T T T T = ( ) ( )... ( ) T T T T We can carr hs calculan an number f ses we wsh. Nex, fr he frecas errr: Var[ ] Var[... a ] T T T T T T T T T T Var[ ] Var[... a ] ( ) Var[ T] Var[ T T... T at] Var[ (... a )... a ] Var[( ) ] ( ( ) ) T T T T T T T T at at As he number f ses ncrease, he frmulas becme mre cumbersme. Alernael, we can use he MA( ) equvalen reresenan and cmue he frecas errr. Techncal Ne Auregressve Mdel 7 Sder Fnancal Cr, 04

8 IRF={ z} z ( LL...) a And he frecas errr s exressed as fllws: Var[ ] T Var[ ] ( ) T Var[ ] ( )... T Var[ ] (... )... Tk k Var[ ] (...) Tk k Ne: The cndnal varance grws cumulavel ver an nfne number f ses reach s lng run (uncndnal) varance. Crrelgram Wha d he au regressve (AR) crrelgram ls lk lke? Hw can we denf an AR rcess (and s rder) usng nl ACF r PACF ls? Frs, le s examne he ACF fr an AR rcess: Where: k ACF(k) k E[( x )( x )] (cvarance fr lag j) j j E x Le s frs cmue he au cvarance funcn j. [( ) ] (lng-run varance) E[( x )( x )] E[ zz ] E[( z z.. z a ) z ]... ( )... E[( z z.. z a ) z ] ( )... ( ) ( ) Techncal Ne Auregressve Mdel 8 Sder Fnancal Cr, 04

9 Nex, fr he rd lag cvarance; E[( z z.. z a ) z ] ( ) ( ) ( ) In sum, fr an AR() rcess, we need cnsruc and slve lnear ssems cmue he values f he frs au cvarances ( ) ( 4) ( 5) ( 5) ( 6) ( 7) ( 4 6) ( 7) ( 8) ( 9) ( ) ( ) ( ) The au cvarance fr lags greaer han s cmued eravel as fllws: k k k k k Examle: Fr an AR(5) rcess, he lnear ssem f equans f he au cvarance funcns s exressed belw: Q: wha d he lk lke n he ACF l? 4 5 ( ) ( 4) ( 5) 6 0 ( 5) 4 4 Due he causal effec, ACF values f a rue AR rcess dn dr zer a an lag number, bu raher al exnenall. Ths rer hels us qualavel denf he AR/ARMA (vs. MA) rcess n he ACF l. Deermnng he acual rder (.e. ) f he underlng AR rcess s, n ms cases, dffcul. Techncal Ne Auregressve Mdel 9 Sder Fnancal Cr, 04

10 Examle: Le s cnsder he AR() rcess: z z a Ez [ z ] Ez [ z ]... k k k The ACF fr an AR() rcess can be exressed as fllws: k k ACF(k) The ACF values dn dr zer a an lag number, bu raher declne exnenall. Q: Wha abu a hgher rder AR rcess? The ACF l can ge ncreasngl mre cmlex, bu wll alwas al exnenall. Ths s due he mdel s causal rer. We can ell he dfference beween an MA rcess and an AR/ARMA rcess b hs qualave dfference. We need a dfferen l r l hel denf he exac rder f he AR rcess and s rder: a l ha drs zer afer he h lags when he rue mdel s AR(). Ths l r l s he aral aucrrelan l (PACF). Paral au crrelan funcn (PACF) The aral au crrelan funcn (PACF) s nerreed as he crrelan beween x and where he lnear deendenc f he nervenng lags ( x, x,..., x ) has been remved. h PACF( h) Crr( x, x x, x,..., x ) h h x h, Ne ha hs s als hw he arameers f a mulle lnear regressn (MLR) mdels are nerreed. Examle: x x Techncal Ne Auregressve Mdel 0 Sder Fnancal Cr, 04

11 In he frs mdel, s nerreed as he lnear deendenc beween and x. In he secnd mdel, he s nerreed as he lnear deendenc beween and x, bu wh he deendenc beween and x alread accuned fr. In sum, he PACF has a ver smlar nerrean as he ceffcens n he mulle regressn suans and he PACF values are esmaed usng hse ceffcen values. () Cnsruc a seres f regressn mdels and esmae he arameers values: x 0,,x a x 0,,x,x a x 0,,x,x,x a x x x x x a 0,4,4,4,4 4, x x x x... x a 0, k, k, k, k k, k k Nes: () The PACF(k) s esmaed b kk,. () T esmae he PACF f he frs k lags, we d need slve k regressn mdels, whch can be slw fr larger daa ses. A number f algrhms (e.g. Durbn Levensn algrhm and Yule Walker esmans) can be emled exede he calculans. () The PACF can be calculaed frm he samle au cvarance. Fr examle, esmae he PACF(), we slve he fllwng ssem: Fr PACF (), we slve he fllwng ssem:,,,,, Usng he Durbn Levensn algrhm mrves he calculan seed dramacall b re usng rr calculans esmae curren nes. E[( x )( x )] j j Techncal Ne Auregressve Mdel Sder Fnancal Cr, 04

12 B defnn, he au cvarance f lag rder zer ( ) s he uncndnal (margnal) varance. B desgn, fr a rue AR() rcess, he crresndng PACF l drs zer afer lags. On he her hand, he ACF l als (declnes) exnenall. Usng nl he PACF l, I shuld be able cnsruc an AR mdel fr an rcess, rgh? N. The PACF l manl examnes wheher he underlng rcess s a rue AR rcess and denfes he rder f he mdel. Cnclusn T reca, n hs aer, we lad he fundan fr a slghl mre cmlex mdel: he auregressve mdel (AR). Frs, we resened he AR rcess as a resrced frm f an nfne rder MA rcess. Nex, armed wh a few mahemacal rcks (.e. IRF, aral fracn decmsn and gemerc seres), we ackled man mre cmlex characerscs f hs rcess (e.g. frecasng, lng run varance, ec.) b reresenng as an MA rcess. Laer n, we nrduced a new cnce: Causal. A rcess s defned as causal f and nl f s values { X } are deenden n he rcess s as shcks/nnvans{ a, a, a,...}. We shwed ha sanar s n a suffcen cndn fr ur mdels; he mus be causal as well. Fnall, we delved n AR rcess denfcan usng crrelgram (.e. ACF and PACF) ls. We shwed ha he ACF f an AR rcess des n dr zer, bu raher als exnenall n all cases. Furhermre, we lked n PACF ls and ulned ha fac ha PACF, b desgn, drs zer afer lags fr a rue AR rcess. As we g n dscuss mre advanced mdels n fuure echncal nes, we wll fen refer he MA and AR rcesses and he maeral resened here. References Hamln, J.D.; Tme Seres Analss, Prncen Unvers Press (994), ISBN D. S.G. Pllck,; Handbk f Tme Seres Analss, Sgnal Prcessng, and Dnamcs, Academc Press (999), ISBN: Bx, Jenkns and Resel; Tme Seres Analss: Frecasng and Cnrl, Jhn Wle & SONS. (008) 4h edn, ISBN: Techncal Ne Auregressve Mdel Sder Fnancal Cr, 04