1. Joint stationarity and long run effects in a simple ADL(1,1) Suppose Xt, Y, also is stationary?

Transcription

1 HG Third lecure - 9. Jan. 04. Join saionariy and long run effecs in a simple ADL(,) Suppose X, Y are wo saionary ime series. Does i follow ha he sum, X Y, also is saionary? The answer is NO in general. There may be non-saionariy presen in he covariance srucure beween he wo series. On he oher hand, if we require ha X, Y are joinly saionary, hen X Y is saionary. DEF. (C.f. Hamilon secion 0.) Consider he vecor ime series V X Y. Then we define join (covariance) saionariy for X, Y, o mean ha he vecor ime series V is covariance saionary meaning ha i. EX x EV is consan, and EY y ii. cov( X, X h) cov( X, Y ) h ( h) cov( V, Vh) cov( Y, X h) cov( Y, Y h) x( h) xy( h), h 0,, yx( h) y( h) does no depend on. Noe ha, in conras o univariae ime series where he auocovariance saisfies ( h) ( h), we do no have he same propery for vecor ime series, i.e., in general, ( h) ( h) - due o he fac ha, in general, we may have ( h) ( h) xy. If he vecor ime series V is covariance saionary, i follows ha any linear combinaion ax by is saionary as well. yx

2 Now suppose X, Y are joinly saionary and saisfy he following dynamic specificaion. () Y 0 Y 0X X, where ~ WN(0, ) We assume ha (i) is an innovaion - meaning ha is uncorrelaed wih he prehisory of Y, i.e., Y, Y,. This can be achieved by requiring ha he soluion of Y from () should be causal, i.e., requiring ha, and (ii) ha X is a causal saionary (i.e., ha 0 ), and (iii) ha X is exogenous in he srong sense ha is independen of he whole series X. This is a special case of an auoregressive disribued lag model of order and (an ADL(,) model). Noe. Such a model can be esimaed by simple OLS which provides consisen esimaes for all 5 parameers. In his conex Y, Y, are called predeermined, which plays he role of a weaker exogeneiy assumpion ensuring ha OLS works - reaing Y as an explanaory variable. If he error is replaced by a MA error, for example by u, hen Y can no longer be reaed as an explanaory variable in () since i will be correlaed wih he error, u. Also () is no longer an ADL model, bu a more general ARMAX model ha needs oher esimaion mehods han OLS. Wih hese assumpions we can easily find he long-run effecs involved here. There are a leas wo ways of hinking abou his: Mehod A: Find he long-run effecs from he (causal) soluion of (), which exiss since. Wrie firs () in erms of lag polynomials () ( L) Y 0 ( L) X, where ( L) L and ( L) 0 L

3 Now, since, he infinie order linear filer ( L) is well defined, and we find he soluion of () simply as ( L) 0 ( L) (3) Y 0 X X ( L) ( L) ( L) ( L) ( L) From his we see ha we have long-run effecs here, () he effec of a uni change in leaving all oher s unchanged. We found in LN ha his is () 3. Similarly (), since ( L) ( L) 0 L L he long-run ( L) effec on Y of a uni change in X leaving all oher X s unchanged, j0 () 0 j () () Mehod B: (quie common in economerics). Here we imagine ha he dynamic specificaion in () implies a long-run seady sae obained when all errors (including hose for X ) are se = 0. If he errors are se o zero, boh X and Y will, due o saionariy, ake he expeced values, seady sae relaionship becomes, and x respecively, and he y y 0 y 0 x x ( ) ( ) y 0 0 x 0 0 (4) y x so a uni shif in he long-run seady sae of of 0 in he seady sae for Y. X will be accompanied by a shif This secion is a pre-ase of he sudy of relaionships beween ime series presened by Ragnar laer. We reurn o he univariae case.

4 . Auocorrelaion funcion for a causal ARMA(p,q) ARMA(p,q): 4 p q (5) ( L) Y 0 ( L), where ( L) L pl, ( L) ( L ql ) and where ~ WN(0, ). We assume ha all roos of () z are ouside he uni circle, in which case (5) has a causal saionary soluion, wih expeced value, EY 0 ( ) (). Inroducing he cenered series, y Y, exercise 3 of seminar shows ha (5) is equivalen wih (6) ( L) y ( L) wihou he consan 0, and where ( ) 0 Ey. The auocovariance funcion, is now ( h) ( h) E( y y h), h 0,,,3, and he auocorrelaion funcion (acf), ( h) ( h) ( h) (0), h 0,,,3, where (0) var( Y) var( y ) and (0). Case, p = 0 (MA(q). We find, puing 0, q q q j jh, 0hq ( h) E( y y h) j j kk j0 j0 k0 0 h q giving he acf q j jh ( ) j0 0 q h hq 0 h q

5 5 Case, p > 0 (ARMA(p,q)). The main poin is ha boh ( h) ( h) saisfy he same homogeneous difference equaion as he dynamic mulipliers j and j we found in LN, i.e., and (7) ( L) ( h) 0 and ( L) ( h) 0 for h max( p, q ) wih slighly differen iniial values for h max( p, q ). For compleeness sake: For ( h), he iniial condiions can, in general, be expressed by he j s and he dynamic mulipliers from ( L) ( L) ( L) L L (8) 0 ( h) ( h ) ( h p) p h 0 h q qh for 0 h max( p, q ) (Noe ha in he AR(p) case ( L) ( L) ( L) L L ) 0 In he AR(p) =ARMA(p,0) case (8) reduces o somehing slighly simpler (9) for h 0 ( h) ( h ) p ( h p) 0 for h 0 For small values of p and q i is no necessary o go hrough j or j bu easier o calculae he iniial values direcly (see, e.g., Hamilon a he end of secion 3.4 for he AR() case.) Having found ( h) ( h) ( h) (0). ( h), i is easy o find he acf ( h) using

6 Since he general soluion for 6 ( h) is equal o he general soluion for h, we see ha ( h) ( h) (0) approaches 0 jus as fas as h when h, i.e., exponenially fas. 3. Esimaion of an causal ARMA(p,q) model (a rough survey) Suppose Y ~ ARMA( p, q ) such ha ( L) Y 0 ( L) ha we rewrie in he form (see Ex.3 in seminar ) (0) ( L)( Y ) ( L), where EY ( ) 0 (), and ~ WN(0, ). We have observaions of Y, Y and wan o esimae he parameers in (0)., T There are several more or less asympocically equivalen esimaion procedures available. I urns ou ha maximum likelihood mehods based on Gaussian join disribuions usually work well even if he Y s are no normally disribued. A few mehods can be menioned Full maximum likelihood (mle) based on Gaussian disribuion, Condiional mle based on condiional (Gaussian) join condiional disribuion of Y,, Y T given fixed sar value Y. Uncondiional leas squares esimaion. Condiional leas squares given fixed Y. Yule-Walker esimaion in he AR(p) case which are he same as he momen esimaors (mme). If he parameer vecor (excep ) is ' (,,, p,,, q), hen all hese esimaors lead o he same asympoic normal disribuion (even if he observaions are no Gaussian) F ˆ ( ) (0, ) T T N C where C is an asympoic covariance marix derived from a Gaussian Fisher informaion marix, and which can be

7 7 consisenly esimaed. Deails (somewha complicaed) can parly be found in Hamilon and he more advanced ime series lieraure (and will no be discussed here). The defaul esimaion procedure used by Saa is he full mle, bu one may order some of he oher mehods as well. The full mle mehod are based on he facorizaion of he join Gaussian disribuion f ( y,, y ) T f( y ) f ( y y ) f ( y3 y, y) f ( yt y,, yt ) The simple AR() case may illusrae he differen mehods: Then (in he Gaussian case), is independen of he prehisory, Y, Y, and Y ( Y ) implies ha f ( y y,, y ) f ( y y ), which is simply he N( ( y ), ) densiy (pdf). The sar, Y, has a differen normal pdf, by saionariy, N wih parameer deermined (, y ) E( Y ), y var( Y ) (he las follows since Y ( Y ) y y ) The full log likelihood hen becomes () l full T consan ln( ) ln( ) S(, ) where S(, ) ( )( y0 ) ( y ) ( y ) () T is called he uncondiional sum of squares.

8 8 Full mle (Saa s defaul) means maximizing (), which requires ieraions, S and hen esimaing ( ˆ ˆ, ˆ ). T Uncondiional leas squares means minimizing () firs, which also requires S ieraions, and hen esimaing ( ˆ ˆ, ˆ ). T Condiional mle uses he simpler condiional log likelihood (3) where T S c 0 l(,, y ) cons. ln( ) (, ) T Sc (, ) ( y ) ( y ) (4) is called he condiional sum of squares (being equal o a usual sum of squares). The condiional mle maximizes (3) and he condiional leas squares minimizes (4) giving he esimaors ˆ y ˆ y ˆ, where T T T and T T y Y y Y (showing ˆ Y Y T, i.e., he mme = he Yule-Walker esimaor) and ˆ T T ( Y Y )( Y Y ) ( Y Y), which is quie close o he usual OLS esimaor (where we would use he same mean Y insead of Y, Y ). Noe. In principle, using OLS on an AR(p) would give consisen and asympoic normally disribued esimaors for he same reason as given for he ADL model in secion. However, i was discovered ha he OLS esimaors as

9 9 well as he condiional leas squares esimaors has a endency of producing a bias, he so called Hurwiz-bias, of size in moderae samples. This is T nicely described and illusraed in Ragnar s firs lecure noes 0. This is probably he main reason why modern sofware programs usually prefer he full mle esimaion o compensae for his bias. The necessiy of ieraions is hardly an issue any more ) The Yule-Walker esimaor is he momen mehod based on (9) reproduced here ( h) ( h ) p ( h p) for h 0 which in marix noaion becomes ( h ) p ( h p) for h 0, where p p, and (0) ' p ( k j ) p jk, p is a p p marix, (,, p )' a p vecor, and ( (0),, ( p))' is a p vecor. The mehod of momens consiss in p replacing all ( h) by sample esimaes ˆ( h) and solve ˆ ˆ ˆ ˆ, ˆ ˆ(0) ˆ ˆ p p p p p

10 -.0 0 ln BNP FN, D.0.04 ln BNP FN A pracical example he BNP daa from LN. ln(bnp) 980q 990q 000q 00q Dela ln(bnp) Firs difference 980q 990q 000q 00q

11 The Box-Jenkins idenificaion process (described below) leads o he decision ha he bes model appears o be a (causal) ARMA(,) process for Y ln BNP. This we express by saying ha Y ln BNP is an ARIMA(,,). d (DEF: X ~ ARIMA( p, d, q ) means ha X ~ ARMA( p, q) (causal)) The esimaion of his model is done by Saa, using he arima command:. arima y,arima(,,) (I have defined y = ln(bnp) in my daase) (seing opimizaion o BHHH) Ieraion 0: log likelihood = Ieraion : log likelihood = Ieraion : log likelihood = Ieraion 3: log likelihood = Ieraion 4: log likelihood = (swiching opimizaion o BFGS) Ieraion 5: log likelihood = Ieraion 6: log likelihood = Ieraion 7: log likelihood = Ieraion 8: log likelihood = ARIMA regression Sample: 978q - 03q Number of obs = 4 Wald chi(3) = 96. Log likelihood = Prob > chi = OPG D.y Coef. Sd. Err. z P> z [95% Conf. Inerval] y _cons ARMA ar L ma L L /sigma

12 0 Densiy Noe: The es of he variance agains zero is one sided, and he wo-sided confidence inerval is runcaed a zero. Having esimaed an ARIMA model one should always check if here is any evidence in he daa agains he model (specificaion esing). The usual way o aack ha is o sudy he residuals ˆ ha are predicions of he unobservable error erms, o see if hey behave like whie noise. To ge he residuals I use he arima pos-esimaion command predic wih opion residual:. predic resdy,residual (This generaes resdy = prediced ( missing value generaed) error erms (residuals)) Do hey seem o be Gaussian? Prediced error erms for lnbnp (residuals) residual, one-sep There seems o some mild skewness o he lef in relaion o a normal pdf. Is his endency significan? A formal es is, e.g., he Shapiro-Wilk es:

13 3. swilk resdy Shapiro-Wilk W es for normal daa Variable Obs W V z Prob>z resdy So here is evidence agains he normaliy assumpion (5% significance). By reesimaing he model using so called robus sandard errors, I, o a cerain degree, compensae for lack of normaliy:. arima y, arima(,,) vce(robus) ARIMA regression Sample: 978q - 03q Number of obs = 4 Wald chi(3) = Log pseudolikelihood = Prob > chi = Semirobus D.y Coef. Sd. Err. z P> z [95% Conf. Inerval] y _cons ARMA ar L ma L L /sigma The p-values were no affeced, so i does no seem o be oo serious his lack of normaliy. The nex quesion we should check (which is more imporan) is: Is here any evidence of auocorrelaion lef in he residuals? There are various ways of checking his. The firs naural hing o do is o plo he acf and pac (he parial auocorrelaion funcion) which is an inegraed par of he Box-Jenkins diagnosic machinery. So I need firs o explain he concep of he parial auocorrelaion funcion and is usefulness for ime series: DEF. The parial correlaion beween wo variables, Y, Y, conrolling for some oher variables, X, X,, X k, is defined as he correlaion beween Y andy, in he condiional join disribuion of Y, Y given all he

14 4 X, X,, X k. This correlaion can be calculaed by drawing ou all influence of X, X,, X k on Y, Y before he correlaion is calculaed. I.e., ake separaely he regression of Y on X, X,, X k, and of Y on X, X,, X k, ake he correlaion beween he wo ses of residuals obained. The parial auocorrelaion (pac) beween Y and Y h in a saionary ime series is he parial correlaion conrolling for all inermediae Y s, Y, Y,, Y. h Formulas for pac are no given here bu can be found in Hamilon (chap. 3). The poin is, however, ha he pair acf(h) and pac(h) can idenify a pure AR(p) process and a pure MA(q) process: For MA(q), acf(h) = 0 for h > q, while pac(h) ends o 0 slower even for h >q. For AR(p) i is opposie, acf(h) ends o 0 slower and even for h>p, while pac(h) = 0 for h > p. For an ARMA(p,q) neiher acf(h) nor pac(h) ge =0 suddenly. They behave similarly. For whie noise boh acf(h) and pac(h) are 0 for all h >0. The asympoical disribuion for he esimaed pac(h) have imes sandard deviaion T ha we use as a benchmark for random variaion in he esimaed pac when he rue pac(h) = 0.

15 The commands ac resdy and pac resdy give plos: ACF residuals from lnbnp model Lag Barle's formula for MA(q) 95% confidence bands Parial auocorrelaions of resdy PACF residuals lnbnp Lag 95% Confidence bands [se = /sqr(n)] The grey area represens he naural variaion (95%) for he esimaed acf and pac when he rue acf and pac are 0. Excep for possibly one h, he plos does no seem provide any evidence agains he wo funcions of h being zero. The command corrgram gives more info on his wih he Pormaneau es Q for esing he whie noise null hypohesis, H 0, ha acf(h) = 0 for all 0 h m. m m ( ) ˆ ( ) T j j Q T T j Under H 0, i can be proven ha Q F (chi-square disr. wih m degrees of m T freedom. Rejec H 0 for large values of Q m. m

16 6. corrgram resdy, lags(5) LAG AC PAC Q Prob>Q [Auocorrelaion] [Parial Auocor] We see no no evidence agains whie noise her even when he suspicious values a lags 8 and are included in Q. There are oher ess as well (e.g., he runs es for independence no shown here) and he Barle es shown below:

17 Cumulaive Periodogram Whie-Noise Tes ARMA(,) for dela lnbnp Frequency Barle's (B) saisic = 0.54 Prob > B = Under he whie noise hypohesis he residuals should lie along he sraigh line, which seems o be he case for his model. The p-value abou 0.9. In conras I show he same es for anoher enaive model, i.e., he AR(): Cumulaive Periodogram Whie-Noise Tes AR() for dela lnbnp Frequency Barle's (B) saisic =.3 Prob > B = 0.096

18 Idenificaion So how did I arrive a he conclusion ha he ARMA(,) is he bes ARMA model for ln BNP? The apparaus for he Box-Jenkins idenificaion procedure is more or less esablished excep for one poin I am coming o he use of informaion crieria. Firs I look a he acf for he original series Y ln BNP : Acf for y= ln BNP Lag Barle's formula for MA(q) 95% confidence bands This is a ypical acf.graph for a non-saionary process a causal ARMA should have an acf ha ends o zero in an exponenial manner. This is clearly no he case here indicaing differencing as he firs sep.

19 -0.50 Parial auocorrelaions of y Acf and pac for Dela lnbnp Acf for Dela Y = Dela ln BNP Pac for y = ln BNP Lag Barle's formula for MA(q) 95% confidence bands Lag 95% Confidence bands [se = /sqr(n)] The acf/pac graph does no show any clear preference for a pure AR or pure MA, so I will ry several ARMA(p,q). My sraegy is o ry all 5 ARMA(p,q) modeler med 0 pq, 4. I was early recognized ha i is no a good idea o compare models jus by looking a he max log likelihood value, since his auomaically increases when new parameers are added o he model. I urns ou ha over-fiing (oo many parameers) in a model seriously increases uncerainy in paricular in connecion wih predicion which is he mos common use of arima models. So crieria have been developed ha combine he informaion in he log likelihood wih a punishmen due o number of parameers. Two crieria have urned ou o work well in ime series conexs i.e., he Akaike informaion crierion (AIC) and he Schwarz Bayesian informaion crierion (B): AIC: Choose a model o minimize AIC ln(likelihood) k BIC: Choose a model o minimize BIC ln(likelihood) ln( T) k `where T number of observaions k number of parameers

20 0 Simulaions show ha AIC someimes lead o larger han necessary models. However, from oher principles boh crieria are well founded. There are also oher crieria suggesed in he lieraure, bu he wo menioned are he mos common. In Saa he arima pos-esimaion command esa ic provide boh. The following able show he AIC and he BIC crieria for he 5 models: AIC MA AR order BIC MA order AR order The winner model for AIC is ARMA(,3), and he winner for BIC is ARMA(,). Using he principle ha, when, forecasing is he main purpose of he model, hen hen he fewer parameers, he beer. So I wen for he ARMA(,) and subjeced i o some specificaion esing as done above o decide if i is accepable. I passed he ess, so I decided ARMA(,) as my model for forecasing. 6. Forecasing. Here I jus show some example forcasing graphs produced by Saa for he BNP daa and leave he heoreical jusificaion o nex lecure.

21 The arima pos-esimaion commands give many forcasing possibiliies. In order o enable he arima pos-esimaion commands, you have run he arima esimaion command firs. Which I did showing he oupu above.. arima y,arima(,,) (I have defined y = ln(bnp) in my daase) (---- Oupu shown on page ----). predic resdy,residual (This generaes resdy = prediced ( missing value generaed) error erms (residuals)). predic preddy,xb (The opion xb generaes preddy = (one-sep ahead) prediced dela(lnbnp)). predic yha,y (The opion y generaes yha = ( missing value generaed) (one-sep ahead) prediced y=lnbnp ). lis y yha D.y preddy resdy in / residual y=lnbnp yha D.y= dela y preddy resdy

22 One-sep ahead predicion of ln(bnp) 005q3 007q3 009q3 0q3 03q3 ln BNP FN y predicion, one-sep 7. Dynamic forcasing in Saa. You need dynamic forcasing if you wish o forcas more han one period ahead. So, firs I use only daa up o 00q o esimae my model and predic he res of he period from hese. Then I can compare hese predicions wih he acual observaions:. arima y if in(,00q), arima(,,) ARIMA regression Sample: 978q - 00q Number of obs = 8 Wald chi(3) = 8.63 Log likelihood = Prob > chi = OPG D.y Coef. Sd. Err. z P> z [95% Conf. Inerval] y _cons ARMA ar L ma L L /sigma

23 3. predic shory,y (This generaes one-sep ahead predicions for he ( missing value generaed) whole period unil 03q. predic shordy, dynamic(q(00q)) y ( missing value generaed). woway (sline y if >q(008q), recas(scaer)) (sline shory if >q(008q)) (sline shordy if >q(008q)) From he Saa pdf-documenaion on arima posesimaion: dynamic(q(00q)) produces dynamic (also known as recursive) forecass. ime consan specifies when he forecas is o swich from one sep ahead o dynamic. In dynamic forecass, references o evaluae o he predicion of for all periods a or afer ime consan; hey evaluae o he acual value of for all prior periods. One-sep and mulisep (dynamic) forcasing afer 00q Unil 00q he wo are he same q 009q 00q 0q 0q 03q ln BNP FN y predicion, dyn(q(00q)) y predicion, one-sep

24 4 8 Redundan parameers (he common facor problem) In he nex example eps is 4 observaions of whie noise ~ N(0,). Fiing an ARMA(,) gives (spuriously) significan parameers. Rule: Cancel common facors in ( L) and ( L) in an ARMA-model in order o avoid spuriously significan parameers. So, when we alk abou a proper ARMA model we implicily assume ha ( L) and ( L) have no common facors.. summarize eps Variable Obs Mean Sd. Dev. Min Max eps arima eps, arima(,0,) ARIMA regression Sample: 978q - 03q Number of obs = 4 Wald chi() =.46 Log likelihood = Prob > chi = OPG eps Coef. Sd. Err. z P> z [95% Conf. Inerval] eps _cons ARMA ar L ma L /sigma Noe ha in his model we know ha (*) Y ~ WN(0,). This implies ha for any consan, c, we have (**) cy c, Subracing (**) from (*), we ge ( cl) Y ( cl), so ha he ARMA(,) is rue for any c. By running he ARIMA(,0,) we see ha we ge significan values for og, which clearly would be misleading if we did no know ha Y were whie noise.

25 5 Hence, in order ha an ARMA(p,q) model does no give misleading resuls, we should require ha he wo polynomials, ( L) and ( L) doe no have any common facors. In oher words: Cancel all common facors in he raio ( L), in order o have a proper ARMA- ( L) model!