CH4. Auto Regression Type Model

Transcription

1 CH4. Auo Regression Type Model Saionary Processes ofen a ime series has same ype of random behavior from one ime period o he nex ouside emperaure: each summer is similar o he pas summers ineres raes and reurns on equiies saionary sochasic processes are probabiliy models for such series process saionary if behavior unchanged by shifs in ime a process is weakly saionary if is mean, variance, and covariance are unchanged by ime shifs hus X, X, is a weakly saionary process if EX ( i ) Var( X i ) (a consan) for all i (a consan) for all i Corr( X, X ) ( i j ) for all i and j for some funcion i j he correlaion beween wo observaions depends only on he ime disance beween hem (called he lag) example: correlaion beween X and X 5 = correlaion beween X 7 and X 0 is he correlaion funcion Noe ha ( h) ( h) covariance beween X and X h is denoed by ( h) () is called he auocovariance funcion Noe ha ( h) ( h) and ha (0) since (0) ( h) ( h) many financial ime series no saionary bu he changes in hese ime series may be saionary: z y y or z ( B) y Lag operaor B. Whie Noise simples example of saionary process: No Correlaion Case X, X, is Whie noise or EX ( i ) for all i WN(, ) if Var( X i ) Corr( X, X ) 0 for all i j i (a consan) for all i j Noe: Disribuion specificaion is no required. I does no have o be a normal disribuion If X, X, IID normal hen process is Gaussian whie noise process whie noise process is weakly saionary wih (0) and ( ) 0 if 0 so ha (0) and ( ) 0 if 0 WN is unineresing in iself bu is he building block of imporan models usually used as error erms.

2 I is ineresing o know if a financial ime series, e.g., of ne reurns, is WN. Esimaing parameers of a saionary process observe y,..., y n esimae and wih Y and esimae auocovariance wih nh ( h) n ( y jh y)( y j y) j esimae () wih ˆ( h) ˆ( h), h =,, : ˆ(0) s Auo Regressive (AR) processes AR() processes ime series models wih correlaion buil from WN in AR processes y is modeled as a weighed average of pas observaions plus a whie noise error AR() is simples AR process,,... are WN(0, ) yy...,, is an AR() process if y ( y ) for all hree parameers: : mean, : variance of one-sep ahead predicion errors : a correlaion parameer y ( ) y compare wih linear regression model, y x 0 0 ( ) is called he consan in compuer oupu is called he mean in he oupu When hen y... h h h0 infinie moving average [MA( )] represenaion If, hen y,... is a weakly saionary process h since, 0 as he lag h Noe: if { } is a sricly independen zero-mean random variable, hen a saionary ime series { y } is linear. Oherwise he series is nonlinear. Properies of a saionary AR() process When (saionariy), hen

3 Ey ( ) (0) Var( y ) h ( h) Cov( y, y h) h ( h) Corr( y, y ) h Only if and only for AR() processes if, hen he AR() process is nonsaionary, and he mean, variance, and correlaion are no consan Formulas 4 can be proved using h... h h0 y For example h h Var( y ) Var( h) h0 h0 Also, for h > 0 i i h j i0 j0 3 h j ( h) Cov(, ) disinguish beween Non-saionary AR() processes Random Walk if = (uni roo case) hen y y no saionary random walk process = variance of,,... and (0) = variance of y, y,... y y ( y )... y... = y0 0 i sar a he process a an arbirary poin y 0 hen E( y y0) y0 for all and Var( y y ) 0 A shock on on ime 0 is ransien in he saionary process, whereas i is permanen in he uni roo series. Saionary processes end o have mean reversion, whereas uni roo processes end o move irregularly off he mean (here is even no saionary level of mean o rever). Uni roo processes include he unpredicable sochasic rend. When, an AR() process has explosive behavior Suppose an explosive AR() process sars a y0 0 and has 0 Then y y ( y ) y i

4 ... Therefore, E( y ) y0 and... y0 4 ( ) Var( y ) (... ) Since, variance increases geomerically fas a increasing variance makes he random walk wander AR() processes when Explosive AR processes no widely used in economerics since economic growh usually is no explosive. (n = 00) Uni Roo Tess Tes for H : v.s. H : 0 H : 0 H : saionary Dickey-Fuller Tes y y, where ~ WN(0, ) If, y are referred o as inegraed wih order one or I(). y y y Noe: is no inegraed, i.e., I(0). Order of inegraion: I(d). I repors he minimum number of differences required o obain a saionary series. A ime series is inegraed of order d if ( B) d X yields a saionary process. 4

5 Augmened Dickey-Fuller Tes p y y y, where i i i ~ WN(0, ) Three ypes Zero mean: y y Single mean: y y Trend: y y The following SAS code performs augmened Dickey-Fuller ess wih auoregressive orders and 5. proc arima daa=es; idenify var=x saionariy=(adf=(,5)); run; Phillips-Perron es is using nonparameric esimaion skill Unlike he null hypohesis of he Dickey-Fuller and Phillips-Perron ess, he null hypohesis of he KPSS saes ha he ime series is saionary. R-code for uni roo ess library(series) x = rnorm(000) # no uni-roo y = diffinv(x) # conains a uni-roo adf.es(x) pp.es(x) kpss.es(x) Esimaion AR() model is a linear regression model one creaes a lagged variable in y and uses his as he x-variable The leas squares esimaion: minimize n y ( y ) If he errors are Gaussian whie noise hen OLS =MLE In SAS, use he AUTOREG or he ARIMA procedure SAS provides maximum likelihood esimaes (ML), uncondiional leas squares esimaes (ULS), Yule-Walker esimaes (YW: defaul), ieraive Yule-Walker esimaes (ITYW) proc auoreg daa=b; model y = ime / nlag= mehod=ml oupu ou=p p=yha pm=yrend lcl=lcl ucl=ucl; run; backsep; 5

6 The oupu daa se includes all he variables in he inpu daa se, he forecas values (YHAT), he prediced rend (YTREND), and he upper (UCL) and lower (LCL) 95% confidence limis. Backsep: sepwise variable selecion mehod For more abou Yule-Walker esimaes, see Gallan and Goebel (976) Residuals ˆ y ˆ ˆ ( y ˆ ) esimae,,..., n since y ( y ) used o check ha y, y,..., y n is an AR() process auocorrelaion in residuals evidence agains AR() assumpion o es for residual auocorrelaion use SAS s auocorrelaion plos can also use he Ljung-Box es null hypohesis is ha auocorrelaions up o a specified lag are zero SAS code for SACF and PSACF compues he exended sample auocorrelaion funcion uses hese esimaes o enaively idenify he auoregressive and moving average orders of mixed models. The following code generaes an ESACF able wih dimensions of p=(0:7) and q=(0:8). proc arima daa=call; idenify var=vol esacf p=(0:7) q=(0:8); run; Auocorrelaion Tess Durbin-Wason Tes Suppose e e, where wih is iid normal disribued. Tes H : 0 is esing no firs order auocorrelaion in 0 e. R-code: Auocorrelaion es for OLS residual library(lmes) dwes(revenue~asses, daa=bankda) The number of lags can be specified using he max.lag argumen library(car) resuls =lm(y ~ x + x) durbin.wason(resuls,max.lag=) SAS code: Auocorrelaion es for OLS residual 6

7 proc auoreg daa=a; model y = ime / dw=4 dwprob; run; Box-Pierce and Ljung-Box Tes library(s) y = arima.sim(lis(order = c(,,), ar = c(0.7, 0.), ma=(0.3)), n = 00) #y: simulaed ime series from arima(,,) model s.plo(y) a =arima(y,order=c(,,0)) Box.es(a$resid) Box.es(a$resid, ype="ljung-box") Example: GE daily reurns The SAS oupu comes from running he following program (proc auoreg) Here is he SAS oupu SAS uses he model 7

8 y y ˆ = -0.5 and sandard deviaion of ˆ is value for esing H : 0 versus 0 H : 0 is null hypohesis: log reurns are whie noise, and alernaive is ha hey are correlaed small p-value is evidence agains he geomeric random walk hypohesis h ( h) correlaion beween successive log reurns In summary, AR() process fis GE log reurns beer han whie noise no proof ha he AR() fis hese daa o check ha he AR() fis well, look sample auocorrelaion funcion (SACF) of he residuals plo of he residual SACF is available from SAS SACF of he residuals from he GE daily log reurns shows high negaive auocorrelaion a lag 6 ˆ(6) is ouside he es limis so is significan a.05 SACF of residuals from an AR() fi o GE daily log reurns he more conservaive Ljung-Box simulaneous es ha ()... () 0 has pvalue =.0 (in MINITAB) since he AR() model does no fi well, one migh consider more complex models hese will be discussed in following secions can also esimae and es ha is zero -value for esing ha is zero is very small p-value is near one large values of he p-value are insignifican Example: Cree Daily Log Reurns The SAS oupu comes from running he following program (proc arima) 8

9 Here is he SAS oupu AR(p) models y is AR(p) process if ( y ) ( y ) ( y )... ( y ) p p here,..., n is WN(0, ) muliple linear regression model wih lagged values of he ime series as he x-variables model can be re-expressed as y y y 0... p p here 0 (... p) leas-squares esimaor minimizes n y ( 0 y... p y p) p 9

10 leas-squares esimaor can be calculaed using a muliple linear regression program one mus creae x-variables by lagging he imeseries wih lags hrough p easier o use he ARIMA command in SAS s AUTOREG procedures hese do he lagging auomaically Sepwise regression applied o AR processes sepwise regression: looks a many regression models see which ones fi he daa well will be discussed laer backwards regression: sars wih all possible x-variables eliminaes hem one a ime sop when all remaining variables are significan can be applied o AR models SAS s AUTOREG procedure allows backsepping as an opion Example proc auoreg daa=a; model y = ime / mehod=ml nlag=5 run; backsep; The following SAS program sars wih an AR(6) model and backseps Here is he SAS oupu 0

11

12 Forecasing AR models can forecas fuure values consider forecasing using an AR() process have daa y,..., y n and esimaes ˆ and ˆ remember yn ( yn ) n and E( n y,..., yn) 0 so we esimae yn by yˆ ˆ ˆ ˆ n : ( yn ) and ˆ ˆ ˆ ˆ ˆ ˆ ˆ ˆ ˆ n y : ( y ) ( y ) ec. y by n n n in general, ˆ ˆ ˆk y ( ˆ nk yn ) if ˆ < hen as k increases forecass decay exponenially fas o ˆ forecasing general AR(p) processes is similar Example: for an AR() process yn ( yn ) ( yn ) n herefore yˆ ˆ ˆ ˆ ˆ n : ( yn ) ( yn ) yˆ : ˆ ˆ ( yˆ ˆ ) ˆ ( y ˆ ), ec. n n n he forecass can be generaed auomaically by saisical sofware SAS SAS program: AR() using proc Auoreg proc auoreg daa=b; model y = / nlag= mehod=ml oupu ou=p p=yha pm=yrend lcl=lcl ucl=ucl; run; backsep; The oupu daa se includes all he variables in he inpu daa se, he forecas values (YHAT), he prediced rend (YTREND), and he upper (UCL) and lower (LCL) 95% confidence limis.

13 Appendices Co-inegraion Tess If we find he response and predicor variables are inegraed (non-saionary), hen we migh suspec he spurious regression problem (see Granger and Newbold, 974) from he models. A regression model involving he non-saionary series can spuriously lead o a significan relaionship beween unrelaed series. a spurious regression problem However, Engle and Granger (987) claimed ha if he error erm is saionary, in which case, he non-saionary ime series are said o be co-inegraed. Then he relaionship beween variables is inerpreed o be in long-run equilibrium. Technically, Hamilon has shown ha he OLS esimaes for he coefficiens of he regression model are consisen under he exisence of he co-inegraion (Hamilon, 994, pp. 590~59). Tes for he non-saionairy of each response and predicor variable es for a uni roo of each variable Tes for he co-inegraing relaionship beween variables es for a uni roo in he residuals of he co-inegraion regression Le y and x be inegraed (non-saionary), and y X e If y and x are co-inegraed, hen esimaes of e would be I(0). If no, esimaes of e would be also non-saionary for some. Evidence of co-inegraion implies ha a variable capures he dominan source of persisen innovaions in he oher variable over his period ineresed in long erm relaionship Phillips-Ouliaris Co-inegraion Tes Compues he Phillips-Ouliaris es for he null hypohesis ha x is no co-inegraed. The uni roo is esimaed from a regression of he firs variable (column) of x on he remaining variables of x wihou a consan and a linear rend. R-code x = diffinv(rnorm(000)) y =.0-3.0*x+rnorm(x,sd=5) z = s(cbind(x,y)) # co-inegraed po.es(z) If he co-inegraion relaionship is deeced, he error correcion model (ECM) is usually applied o model he dynamic relaionship among he co-inegraed variables. ECM employs he differenced variables o ransform he original series ino a saionary process. Example k k k i i i i i i Y Y RE I u Granger Causaliy For some k 0, if E( y E( y F )) E( y E( y )), k k k k 3

14 Random Walk hen we say ha x Granger-causes y Noe: F denoes he informaion se of x and y available a ime denoes he informaion se of y available a ime Packe lmes in R: Currenly, he mehods for he generic funcion grangeres only perform ess for Granger causaliy in bivariae series. The es is simply a Wald es comparing he unresriced model in which y is explained by he lags (up o order order) of y and x and he resriced model in which y is only explained by he lags of y. Null hypohesis: x does no Granger-cause y R-code ## Which came firs: he chicken or he egg? library(lmes) daa(chickegg) grangeres(egg ~ chicken, order = 3, daa = ChickEgg) grangeres(chicken ~ egg, order = 3, daa = ChickEgg) ## alernaive ways of specifying he same es grangeres(chickegg, order = 3) grangeres(chickegg[, ], ChickEgg[, ], order = 3) R-code # simulaed ime series y from AR() model and esimae & forecasing library(s) y = arima.sim(lis(order = c(,0,0), ar = c(0.7, 0.)), n = 00) #simulaion s.plo(y) a=arima(y,order=c(,0,0)) #esimaion (fiing) predic(arima(lh, order = c(,0,0)), n.ahead = ) # forecasing up o -ime ahead # load a random walk daa from R packe, and plo i library(tsa) daa(rwalk) plo(rwalk, ype='o', ylab='random Walk') # mulivariae case example library(vars) ## No run: Time 4

15 daa(canada) var.c <- VAR(Canada, p =, ype = "cons") plo(var.c) ## Diagnosic Tesing ## ARCH es arches <- arch.es(var.c) plo(arches) ## Normaliy es normaliyes <- normaliy.es(var.c) plo(normaliyes) ## serial correlaion es seriales <- serial.es(var.c) plo(seriales) ## Predicion var.c.prd <- predic(var.c, n.ahead = 8, ci = 0.95) plo(var.c.prd) ## Sabiliy var.c.sabil <- sabiliy(var.c, ype = "Rec-CUSUM") plo(var.c.sabil) ## End(No run) Reading liss [] Cryer, J.D., Chan, K. (008), Time Series Analysis wih Applicaions in R, Springer, New York, USA. [] Enger RF and Granger CWJ (987). Coninegraion and error correcion: represenaion, esimaion, and esing. Economerica 55: [3] Gallan, A. R. and Goebel, J. J. (976) Nonlinear Regression wih Auoregressive Errors, Journal of he American Saisical Associaion, 7, [4] Granger CWJ and Newbold P (974). Spurious regressions in economerics. J Economerics : -0. [5] Hamilon J (994). Time Series Analysis. Princeon, New Jersey. [6] Tsay, R.S. (005), Analysis of Financial Time Series, Wiley, New Jersey, USA. [7] 경제시계열분석 (00), 박준용, 장유순, 한상범, 경문사 5