Section 11 Basics of Time-Series Regression

Transcription

1 Secin Baic f ime-serie Regrein Wha differen abu regrein uing ime-erie daa? Dynamic effec f X n Y Ubiquiu aucrrelain f errr erm Difficulie f nnainary Y and X Crrelain i cmmn ju becaue bh variable fllw rend hi can lead puriu regrein if we inerpre he cmmn rend mvemen a rue crrelain Fcu i n eimaing he prperie f he daa-generaing prce raher han ppulain parameer Variable are fen called ime erie r ju erie Lag and difference Wih ime-erie daa we are fen inereed in he relainhip amng variable a differen pin in ime. Le X be he bervain crrepnding ime perid. he fir lag f X i he preceding bervain: X. We meime ue he lag perar L(X ) r LX X repreen lag. We fen ue higher-rder lag: L X X. he fir difference f X i he difference beween X and i lag: ΔX X X = ( L)X Higher-rder difference are al ued: Δ X = Δ(ΔX ) = (X X ) (X X ) = X X + X = ( L) X = ( L + L )X Δ X = ( L) X Difference f he lg f a variable i apprximaely equal he variable grwh rae: Δ(lnX ) = lnx lnx = ln(x /X ) X /X = ΔX / X Lg difference i exacly he cninuuly-cmpunded grwh rae he dicree grwh-rae frmula ΔX / X i he frmula fr nce-perperid cmpunded grwh Lag and difference in Saa Fir yu mu define he daa be ime erie: e year hi will crrecly deal wih miing year in he year variable. Can define a variable fr quarerly r mnhly daa and e frma prin u apprpriaely. Fr example, uppe yur daa have a variable called mnh and ne called year. Yu wan cmbine in a ingle ime variable called ime. gen ime = ym(year, mnh) ~ ~

2 hi variable will have a %m frma and will prin u like m4 fr April. Yu can hen d e ime Once yu have he ime variable e, yu can creae lag wih he lag perar l. and difference wih d. Fr example, la perid value f x i l.x he change in x beween nw and la perid i d.x Higher-rder lag and difference can be bained wih l3.x fr hird lag r d.x fr ecnd difference. Aucvariance and aucrrelain Aucvariance f rder i cv(x, X ) We generally aume ha he aucvariance depend nly n, n n. hi i analgu ur Aumpin #: ha all bervain fllw he ame mdel (r were generaed by he ame daa-generaing prce) hi i ne elemen f a ime erie being ainary Aucrrelain f rder (ρ ) i he crrelain cefficien beween X and X. cv ( X, X ) ρ= var X ( X X+, )( X X, ) = + We eimae wih ρ= ˆ, where ( X X, ) X, X i he mean f X ver he range f bervain + = deignaed by he pair f ubcrip. We meime ignre he differen fracin in frn f he ummain ince heir rai ge a ge. Univariae ime-erie mdel We meime repreen a variable ime-erie behavir wih a univariae mdel. Whie nie: he imple univariae ime-erie prce i called whie nie Y = u, where u i a mean-zer IID errr (uually nrmal). he key pin here i he aucrrelain f whie nie are all zer (excep, f cure, fr ρ, which i alway ). Very few ecnmic ime erie are whie nie. Change in ck price are prbably ne. We ue whie nie a a baic building blck fr mre ueful ime erie: Cnider prblem f frecaing Y cndiinal n all pa value f Y. = ~ 3 ~

3 [,, ] Y = E Y Y Y + u Since any par f he pa behavir f Y ha wuld help predic he curren Y huld be accuned fr in he expecain par, he errr erm u huld be whie nie. he ne-perid-ahead freca errr f Y huld be whie nie. We meime call hi freca-errr erie he fundamenal underlying whie nie erie fr Y r he innvain in Y. he imple aucrrelaed erie i he fir-rder auregreive (AR()) prce: Y =β +β Y + u where u i whie nie., In hi cae, ur ne-perid-ahead freca i [ ] freca errr i u. E Y Y =β +β Y and he Fr impliciy, uppe ha we have remved he mean frm Y ha β =. Cnider he effec f a ne-ime hck u n he erie Y frm ime ne n, auming (fr impliciy) ha Y = and all ubequen u value are al zer. Y =β + u = u Y =β Y + u =β u Y3 =β Y + u3 =β u Y. =β u hi hw ha he effec f he hck n Y ge away ver ime nly if β <. he cndiin β < i neceary fr he AR() prce be ainary. If β =, hen hck Y are permanen. hi erie i called a randm walk. he randm walk prce can be wrien Y = Y + u r Δ Y = u. he fir difference f a randm walk i ainary and i whie nie. If Y fllw a ainary AR() prce, hen ρ = β, ρ = β,, ρ=β. One way aemp idenify he apprpriae pecificain fr a imeerie variable i examine he aucrrelain funcin f he erie, which i defined a ρ cnidered a a funcin f. If he aucrrelain funcin decline expnenially ward zer, hen he erie migh fllw an AR() prce wih piive β. A erie wih β < wuld cillae back and frh beween piive and negaive repne a hck. ~ 4 ~

4 he aucrrelain wuld al cillae beween piive and negaive while cnverging zer. he AR(p) prce We can generalize he AR() allw higher-rder lag: Y =β +β Y +β Y + +β Y + u p p We can wrie hi cmpacly uing he lag perar nain: Y =β +β LY +β LY + +β LY + u Y LY LY LY u p p p β β β p =β + p ( ) β L β L β L Y =β + u p β LY =β + u, where β(l) i a degree-p plynmial in he lag perar wih zer-rder cefficien f. We can again analyze he dynamic effec n Y f a hck u m eaily by auming a zer mean (β = ), n previu hck (Y = Y = = Y p = ), and n ubequen hck (u = u 3 = = ). I ge cmplicaed very quickly: Y = u Y =βu Y3 =β u +βu Y =β β +β u +β β u +β u 4 3 We can m eaily ee he behavir f hi by uilizing he plynmial in he lag perar: β ( LY ) = u Y = u β ( L) where he diviin i he equain i plynmial diviin. Yu can eaily verify ha fr he AR() prce, β ( L) = βl i i = +β L+β L + = L ( L) β β i = i u = βu i β L i = Y+ + = β u i he effec f he hck in perid n he value f Y perid laer. Ne ha hi prce i ainary nly if β <. ~ 5 ~

5 Cnider lving fr he r f he equain β(l) =. In he AR() cae, hi i β L = β L = L =. β hu he r f β(l) = mu be greaer han in ablue value if he prce i be ainary. A uni r mean ha β =, which i he randm walk prce. hi paern f aciain beween he ainariy f he AR prce and he r f he equain β(l) = carrie ver in higher-rder prcee. Wih a higher-rder prce, he r f he equain can be cmplex raher han real. he crrepnding ainariy cndiin i ha he r f β(l) = mu lie uide he uni circle in he cmplex plane. A we will dicu a mre lengh laer, fr each uni r lying n he uni circle in he cmplex plane we mu difference Y nce achieve ainariy. Wih r ha lie inide he uni circle, he mdel i irrerievably nnainary. Mving-average prcee We wn ue hem r udy hem much, bu i wrh briefly dicuing he pibiliy f puing lag n u in he univariae mdel. Y =α + u +α u +α u + +αqu q =α +α( L) u i called a mving-average prce f rder q r MA(q) Ne ha he cefficien f u i nrmalized ne. Finie MA prcee are alway ainary becaue he effec f a hck u die u afer q perid. he aucrrelain f an MA(q) prce are zer fr lag greaer han q. When we divided by β(l) lve he AR prce, we creaed an infinie MA prce n he righ ide. We can cmbine AR and MA prcee ge he ARMA(p, q) prce: Y =β +β Y + +β Y + u +α u + +α u p p q q ~ 6 ~

6 ( LY ) β =β +α Lu, which lve Y β β ( L) = + α β ( L) ( L). Sainariy f he ARMA prce depend enirely n he r f β(l) = lying uide he uni circle becaue all finie MA prcee are ainary. Cniderain in eimaing ARMA mdel Auregreive mdel can in principle be eimaed by OLS, bu here are me pifall. By including enugh lagged Y erm we huld be able eliminae any aucrrelain in he errr erm: make u whie nie. If we have aucrrelain in u, hen u will be crrelaed wih u, which i par f Y. S Y i crrelaed wih he errr erm and OLS i biaed and incnien. If Y i highly aucrrelaed, hen i lag are highly crrelaed wih each her, which make mulicllineariy a cncern in rying idenify he cefficien. MA mdel cann be eimaed by OLS becaue we have n value fr lagged u erm hrugh which eimae he α cefficien. Bx and Jenkin develped a cmplicaed ieraive prce fr eimaing mdel wih MA errr. Fir we eimae a very lng AR prce (becaue an MA can be expreed a an infinie AR ju a an AR can be expreed a an infinie MA) and ue i calculae reidual. We hen plug in hee reidual ( back-caing mve generae he pre-ample bervain) and eimae he α cefficien. hen we recalculae he reidual and ierae under he eimae f he α cefficien cnverge. Saa will eimae ARMA (r ARIMA) mdel wih he arima cmmand. Regrein wih erially crrelaed errr erm Befre we deal wih iue f pecificain f Y and X, we will hink abu he prblem ha erially crrelaed errr erm caue fr OLS regrein. Can eimae ime-erie regrein by OLS a lng a Y and X are ainary and X i exgenu. Exgeneiy: ( ) E u X, X, =. Sric exgeneiy: E( u X X X X X ),,,,,,. + + = ~ 7 ~

7 Hwever, nearly all ime-erie regrein are prne having erially crrelaed errr erm. Omied variable are prbably erially crrelaed hi i a paricular frm f vilain f he IID aumpin. Obervain are crrelaed wih he f nearby perid A lng a he her OLS aumpin are aified, hi caue a prblem n unlike heerkedaiciy OLS i ill unbiaed and cnien OLS i n efficien OLS eimar f andard errr are biaed, cann ue rdinary aiic fr inference me exen, adding mre lag f Y and X he pecificain can reduce he everiy f erial crrelain. w mehd f dealing wih erial crrelain f he errr erm: GLS regrein in which we ranfrm he mdel ne whe errr erm i n erially crrelaed hi i analgu weighed lea quare (al a GLS prcedure) Eimae by OLS bu ue andard errr eimae ha are rbu erial crrelain GLS wih an AR() errr erm One f he lde ime-erie mdel (and n ued much anymre) i he mdel in which u fllw and AR() prce: Y =β +β X + u u =φ u +ε, where ε i a whie-nie errr erm and < φ <. In pracice, φ > nearly alway GLS ranfrm he mdel in ne wih an errr erm ha i n erially crrelaed. Le Y,, Y φ = = X Y φ Y, =,3,,, u φ, =, u = u φ u, =,3,,. Y = φ β +β X + u. hen X φ, =, = X φ X, =,3,,, he errr erm in hi regrein i equal ε fr bervain hrugh and i a muliple f u fr he fir bervain. ~ 8 ~

8 By aumpin, ε i whie nie and value f ε in perid afer are uncrrelaed wih u, here i n erial crrelain in hi ranfrmed mdel. If he her aumpin are aified, i can be eimaed efficienly by OLS. Bu wha i φ? Need eimae φ calculae feaible GLS eimar. radiinal eimar fr φ i φ= ˆ crr ( uˆ, ˆ u ) uing OLS reidual. hi eimain can be ieraed ge a new eimae f φ baed n he GLS eimar and hen re-d he ranfrmain: repea unil cnverged. w-ep eimar uing FGLS baed n ˆφ i called he Prai-Winen eimar (r Cchrane-Orcu when fir bervain i drpped). Prblem: φ i n eimaed cnienly if u i crrelaed wih X, which will alway be he cae if here i a lagged dependen variable preen and may be he cae if X i n rngly exgenu. In hi cae, we can ue nnlinear mehd eimae φ and β jinly by earch. hi i called he Hildreh-Lu mehd. In Saa, he prai cmmand implemen all f hee mehd (depending n pin). Opin crc de Cchrane-Orcu; eearch de Hildreh-Lu; and he defaul i Prai-Winen. Yu can al eimae hi mdel wih φ a he cefficien n Y in an OLS mdel, a in S&W Secin 5.5. HAC cnien andard errr (Newey-We) A wih Whie heerkedaiciy cnien andard errr, we can crrec he OLS andard errr fr aucrrelain a well. We knw ha ( X X) u ˆ i = β =β +. ( X X) i = In hi frmula, plim X = μx, plim X X X. =σ i = plim ( X μ ) u plim( v ) plim β β = =, where v = v and S ( ˆ ) X i = σx σx. v X μ u X i = ~ 9 ~

9 And in large ample, ( ˆ ) ~ ~ ( v ) v var var β = var. = 4 σx σx σv Under IID aumpin, var ( v) = var ( v ) =, and he frmula reduce ne we knw frm befre. Hwever, erial crrelain mean ha he errr erm are n IID (and X i uually n eiher), hi den apply. In he cae where here i erial crrelain we have ake in accun he cvariance f he v erm: v + v + + v var ( v ) = var = E( vivj) i= j= = var ( vi) + cv ( vi, vj) i= j i = var v + cv v, v + cv v, v + + cv v, v σv = f, where j f + crr ( v, v j) j = j = + ρj. j = hu, ( ˆ ) ( ) ( ) ( ) σ v var β = f, 4 which expree he variance a he prduc f he σx n-aucrrelain variance and he f facr ha crrec fr aucrrelain. In rder implemen hi, we need knw f, which depend n he aucrrelain f v fr rder hrugh. hee are n knwn and mu be eimaed. Fr ρ we have l f infrmain becaue here are pair f value fr (v, v ) in he ample. Fr ρ, here i nly ne pair (v, v ( ) ) namely (v, v ) n which bae an eimae. he Newey-We prcedure runcae he ummain in f a me value m, we eimae he fir m aucrrelain f v uing he OLS m reidual and cmpue ˆ m j f ˆ = + ρj. j = m

10 m mu be large enugh prvide a reanable crrecin bu mall enugh relaive allw he ρ value be eimaed well. 3 Sck and Wan ugge ching m=.75 a a reanable rule f humb. implemen in Saa, ue hac pin in xreg (wih panel daa) r peimain cmmand newey, lag(m) Diribued-lag mdel Univariae ime-erie mdel are an inereing and ueful building blck, bu we are alm alway inereed n ju in Y behavir by ielf bu al in hw X affec Y. In ime-erie mdel, hi effec i fen dynamic: pread u ver ime. hi mean ha anwering he quein Hw de X affec Y? invlve n ju Y/ X, bu a mre cmplex e f dynamic muliplier Y / X, Y + / X, Y + / X, ec. We eimae he dynamic effec f X n Y wih diribued-lag mdel. In general, he diribued-lag mdel ha he frm Y = β + β ix i + u. Bu f cure, we cann eimae an infinie number f lag cefficien β i, we mu eiher runcae r find anher way apprximae an infinie lag rucure. We can eaily have addiinal regrer wih eiher he ame r differen lag rucure. Finie diribued lag Simple lag diribuin i imply runcae he infinie diribuin abve and r + eimae Y =β + β ix i + u by OLS. i = Y Dynamic muliplier in hi cae are X = β, + fr =,,, r and herwie. Cumulaive dynamic muliplier (effec f permanen change in X) are β +. v= Kyck lag: AR() wih regrer Y =β +β Y +δ X + u Y =δ X Y X + =βδ i = ~ ~

11 Y + =βδ X Y+ =βδ X hu, dynamic muliplier ar a δ and decay expnenially zer ver infinie ime. hu, hi i effecively a diribued lag f infinie lengh, bu wih nly parameer (plu inercep) eimae. Y+ v v Cumulaive muliplier are = δ β. X v= v= v δ Lng-run effec f a permanen change i δ β =. β ~ ~ v= Eimain ha he penial prblem f incniency if u i erially crrelaed. hi i a eriu prblem, epecially a me f he e aiic fr erial crrelain f he errr are biaed when he lagged dependen variable i preen. Kyck lag i parimniu and fi l f lagged relainhip well. Wih muliple regrer, he Kyck lag applie he ame lag rucure (rae f decay) all regrer. I hi reanable fr yur applicain? Example: delayed adjumen f facr inpu: can p uing expenive facr mre quickly han yu ar uing cheaper facr. ARX(p) Mdel We can generalize he Kyck lag mdel lnger lag: Y Y Y X u =β +β + +β p p +δ +. Same general principle apply: Wrry abu ainariy f lag rucure: r f β(l) If u i erially crrelaed, OLS will be biaed and incnien Dynamic muliplier are deermined by cefficien f infinie lag plynmial [β(l)] If mre han n X, all have ame lag rucure Hw deermine lengh f lag p? Can keep adding lag a lng a β p i aiically ignifican Can che max he Akaike infrmain crierin (AIC): SSR p AIC( p) = ln + ( p+ ). Can che max he Bayeian (Schwarz) infrmain crierin (BIC): ( p) SSR p BIC = ln + + ( p ) ln.

12 Ne ha regrein can ue a many a p bervain, bu huld ue he ame number fr all regrein wih differen p value in aeing infrmain crieria. AIC will che lnger lag han BIC. AIC came fir, i ill ued a l BIC i aympically unbiaed Saa calculae inf crieria by ea ic (afer regrein) Can al include mving-average erm ge ARMAX(p, q) ADL(p, q) Mdel: Rainal lag We can al add lag he X variable() Y =β +β Y + +β Y +δ X +δ X + +δ X + u p p q q Ne ha S&W mi curren effec here. Can add mre X variable wih varying lag lengh β ( LY ) =β +δ ( L) X + u, β δ ( L) u Y = + X +. β L β L β L Sainariy depend nly n β(l), n n δ(l). Can eaily eimae hi by OLS auming: E( u Y Y Y p X X X q),,,,,,, = (Y, X ) ha ame mean, variance, and aucrrelain fr all (Y, X ) and (Y, X ) becme independen a All variable have finie, nn-zer furh mmen N perfec mulicllineariy hee are general aumpin ha apply m ime-erie mdel. Granger caualiy Granger aemped e caualiy by eing wheher a variable δ(l) plynmial wa zer. (Need leave X u f regrein here.) F e f e f cefficien n all lag f X, given effec f lagged Y and any her regrer. Rejecin mean X caue Y. Shuld n be called caualiy: he real quein being anwered i wheher X help predic Y given he pah ha Y wuld fllw baed n i wn lag (and perhap n her regrer) Nnainariy due rend Many macrecnmic variable have rend: upu, price, wage, mney upply, prduciviy, ec. all end increae ver ime. ~ 3 ~

13 Deerminiic rend are cnan increae ver ime, hugh he variable may flucuae abve r belw i rend line randmly. Y =β +β + u u i ainary errr erm If he cnan rae f change i in percenage erm, hen we culd mdel lny a being linearly relaed ime. Schaic rend allw he rend change frm perid perid be randm, wih given mean and variance. Randm walk i imple verin f chaic rend: Y = Y + u where u i whie nie. Randm walk wih drif allw fr nn-zer average change: Y =β + Y + u Difference beween deerminiic and chaic rend Cnider large negaive hck u in perid In deerminiic rend, he rend line remain unchanged. Becaue u i aumed ainary, i effec evenually diappear and he effec f he hck i emprary In chaic rend, he lwer Y i he bai fr all fuure change in Y, he effec f he hck i permanen. Which i mre apprpriae? N clear rule ha alway applie Schaic rend are ppular righ nw, bu hey are cnrverial Ne ha he randm-walk mdel i ju he AR() mdel wih β =. hi i a uni r he β(l) = equain. Impac f chaic rend Eimae f cefficien f an auregreive prce will be biaed dwnward in mall ample. Can e β = wih uual e in OLS auregrein Diribuin f aiic are n r cle nrmal Spuriu regrein Nn-ainary ime erie can appear be relaed wih hey are n. Shw he Granger-Newbld reul/able eing fr uni r Since he deirable prperie f OLS (and her) eimar depend n he ainariy f Y and X, i wuld be ueful have a e fr a uni r. Dickey-Fuller e i uch a e. We can ju run Y =β +β Y + u and e β =, becaue he diribuin i n reliable under he null hyphei f nnainariy. If we ubrac Y frm bh ide, we ge Δ Y =β + β Y + u. ~ 4 ~

14 If he null hyphei i rue (β = ) hen he dependen variable i nnainary and he cefficien n he righ i zer. Under he null hyphei, Y fllw a randm walk wih drif (β ). We can e hi hyphei wih an OLS regrein, bu becaue he regrer i nnainary (under he null), he aiic will n fllw he r aympically nrmal diribuin. Inead, i fllw he Dickey- Fuller diribuin, wih criical value much higher han he f he nrmal. If he DF aiic exceed he criical value a ur deired level f ignificance, hen we rejec he null hyphei f nn-ainariy and cnclude ha he variable i ainary. Ne ha a ne-ailed e i apprpriae here becaue β huld alway be negaive. Oherwie, i wuld imply β >, which i nn-ainary in a way ha cann be recified by differencing. Can we be ure ha u i n erially crrelaed? Prbably n, bu by adding me lag f ΔY n he RHS we can uually eliminae he erial crrelain f he errr. Δ Y =β + β Y +γδ Y + +γ Δ Y + u i he mdel fr he p p Augmened Dickey-Fuller (ADF) e, which i imilar bu ha a differen diribuin ha depend n p. We can al ue an ADF e e he null hyphei ha he erie i ainary arund a deerminiic rend ( rend ainary ) by adding a linear rend erm δ he equain. Inuiin f DF and ADF e: Sainary erie are mean-revering. hey end cme back heir mean afer a diurbance, he fuure change depend n he curren level f he erie. If he level Y affec he change in ime (negaively), hen he erie will end rever a fixed mean and i ainary. If he change in ime i independen f he level, hen he erie i flaing away frm any fixed mean and i nnainary. Saa de DF and ADF e wih he dfuller cmmand, uing he lag(#) pin add lagged difference. An alernaive e i ue Newey-We HAC rbu andard errr in he riginal DF equain raher han adding lagged difference eliminae erial crrelain f u. hi i he Phillip-Pern e: pperrn in Saa. Nnainary v. brderline ainary erie Y Y u = + i a nnainary randm walk Y.999Y u = + i a ainary AR() prce ~ 5 ~

15 hey are n very differen when <. Shw graph f hree erie Can we hpe ha ur ADF e will dicriminae beween nnainary and brderline ainary erie? Prbably n wihu lnger ample han we have. Since he null hyphei i nnainariy, a lw-pwer e will uually fail rejec nnainariy and we will end cnclude ha me highly perien bu ainary erie are nnainary. Ne: he ADF e de n prve nnainariy; i fail prve ainariy. Nnainariy due break Break in a erie/mdel are he ime-erie equivalen f a vilain f Aumpin #. he relainhip beween he variable (including lag) change eiher abruply r gradually ver ime. Wih a knwn penial break pin (uch a a change in plicy regime r a large hck ha culd change he rucure f he mdel): Can ue Chw e baed n dummy variable e fr abiliy acr he break pin. Inerac all variable f he mdel wih a ample dummy ha i zer befre he break and ne afer. e all ineracin erm (including he dummy ielf) = wih Chw F aiic. If breakpin i unknwn: Quand likelihd rai e find he large Chw-e F aiic, excluding (rimming) he fir and la 5% (r mre r le) f he ample a penial breakpin make ure ha each ub-ample i large enugh prvide reliable eimae. QLR e aiic de n have an F diribuin becaue i i he max f many F aiic. ~ 6 ~