Applied Time Series Notes White noise: e t mean 0, variance 5 2 uncorrelated Moving Average

Transcription

1 Applied Time Series Noes Whie noise: e mean 0, variance 5 uncorrelaed Moving Average Order 1: (Y. ) œ e ) 1e -1 all Order q: (Y. ) œ e ) e â ) e all 1-1 q -q ( 14 ) Infinie order: (Y. ) œ e ) 1e -1 ) e - ) 3e -3 â all Have o be careful here - may no "converge" i.e. may no exis. Example: Y. œ e +e -1+e -+e -3+ â has infinie variance Pr {Y>C} = Pr{ Z > (Y-.)/_ } = Pr{ Z > 0 } =1/ for any C (makes no sense!) Example: Y. œ e + 3e e e -3+ â Variance{Y} = 5 ( â ) = 5 /(1 3 ) for 3 <1. Y. œ e + 3e e e -3+ â Y -1. œ e e -+ 3 e e -4+ â (Y. )- 3 (Y. )=e +0 Auoregressive - AR(1) -1 (Y. ) œ 3 (Y. ) e all 1 E(Y ). œ 3 [E(Y ). ] 1 Saionariy: E(Y ) consan all (call i.) Cov(Y, Y j) œ #(j) œ funcion of j only E(Y ) œ. (if ± 3 ± 1) Assuming ± 3 ± 1 (y. ) œ 3 (Y. ) e 1 œ 33 [ (Y. ) e ] e 1 œ 333 [ [ (Y. ) e ] e ]+e ec. 1 (Y. ) œe 3e 3 e Again, E(Y ) œ. 3 1 Var (Y ) œ5 ( ) œ5 (1 3 ) 4 6 j Cov (Y, Y j) œ 35 (1 3)

2 Applied Time Series Noes Example: Plo of #(j) versus j is Auocovariance Funcion ( 15 ) j= #(j) / (1) Find 3, variance of Y variance of e and.: 3 = 0.5 (geomeric decay rae) Variance of Y is 64. Variance of e: [Using #(0) = 64 = 5 /(1-3 ) = 5 /(1-.5)] 5 = 64(.75) = 48. Covariances have no informaion abou.. Forecas: Y œ. 3 (Y. ) e n 1 n n+1. œ 90 known (or esimaed) Daa Y 1, Y,, Y n wih Yn œ106. We see ha 3 =0.5 Ys œ 90.5(106 90) œ 98 error Y Ys œ e n 1 n 1 n 1 n+1 n n n n 1 Y œ. 3 (Y. ) e 3e Y s œ. 3 (Y. ) œ 94, error œ e 3e n n n n 1 Y s j j 1 œ. 3 (Y. ) error œe + 3e 3 e n j n n j n j 1 n 1 98, 94, 9, 91, 90.5, forecass. Forecas inervals (large n).., 3 known (or esimaed and assumed known) n 1 n n 1 n 3 n n 1 Forecas errors e, e 3e, e 3e 3 e, (1) Can' know fuure e's j () Can esimae variance 5 (1 3 3 ). (3) Esimae 5 : Use r œ (Y. ) 3 (Y. ) hen 5s œ Dr n or 1 Ge S œ D(Y Y ) n hen 5s œ S (1 3 ) y y Ys 1.96 É5s (1 3 3j ) n j

3 Esimaing an AR(1) Applied Time Series Noes ( 16 ) Y =.(1-3) + 3Y -1 + e = - + 3Y -1 + e Looks _ like a regression: _ Regress Y on 1, Y Y -Y on Y -Y (noin) -1-1 or n _ n! 5 (Y - Y) converges o E{ (Y -.) } = #(0) = = _. Èn [ (Y -Y) e /n ] is Èn imes a mean of (Y -.) e erms! -1-1 uncorrelaed (bu no independen) _ Neverheless È n [!(Y -1 -Y) e /n ] converges o N(0,? ) where -1-1 variance is E{ (Y -.) e } = E{ (Y -.) }E{ e }= #(0) 5 _ n _ 3. Èn ( ^- ) = È n [! -1 (Y -Y) e /n ] / [ n! (Y - Y) ] = in he limi his is N(0, #(0) 5 / #(0) ) = N(0, 1-3 ). EXAMPLE: Winning Percenage (x 10) for baseball's Naional League pennan winner. Regression: Year Y Y ã ã ã ã PROC REG: Y ^ = Y -1 + e, s = (66.06) (.108) Y ^ = (Y ) + e Year Forecas Forecas Sandard Error 1994: (64) = È863.7 = : (64.46) = È863.7(1+.44 ) = 3.10 or 1995: ( ) : (41.38) = =. ^ È863.7/(1-.44 ) = È# ^(0) so long erm forecas is jus mean. Theory assigns sd. error È(1-3)/n o 3^. We have È(1-.44 )/73 =.105

4 Applied Time Series Noes Idenificaion - Par 1 Auo correlaion 3 (j) œ #(j) #(0) ( 17 ) (For AR(1), 3(j) œ 3 j ) ACF Parial Auocorrelaion Regress Y on Y, Y,, Y 1 j Las coefficien C is called j h j parial auocorrelaion PACF. 1 More formally, ^# (j) = n D (Y. )(Y. ) esimaes #(j) 1 n XX w regression marix looks like j 1 n Ô D(Y 1 Y ) D(Y 1 Y )(Y Y ) D(Y 1 Y )(Y Y ) D(Y Y ) Õ ã Ø w w so formally, XXbœ XYis analogous o he populaion equaion (also "bes predicor" idea) Ô #(0) #(1) #(j 1) Ô b1 Ô #(1) #(1) #(0) #(j ) b #() Ö Ù Ö Ùœ Ö Ù ã ã Õ# (j-1) #(j-) #(0) Ø Õb( œ C ) Ø Õ #(j) Ø This defines C j h œ j parial auocorrelaion For AR(1), parials are j œ 1 3 C œ j j j Moving Average MA(1) Y œ. e ) e 1 E(Y ) œ. Var (Y ) œ 5 (1 ) ) Auocovariances j #( ) 5 (1 ) ) 5 ) j

5 Applied Time Series Noes Example: j œ #(j) œ ( 18 ) 5 (1 ) ) œ 10 ake raio 4(1 ) ) œ -10 ) )5 œ 4 ) +.5) 1 œ 0 () +.5)( ) +) œ 0 Forecas MA(1) 1 ) œ - or ) œ - Y œ. + e ) e n 1 n 1 n e has already occurred bu wha is i? I wan n Y s œ. ) e n 1 n so I need e. Use backward subsiuion: n e œ Y. ) e n n n 1 n n 1 n 3 n 3 œ (Y. ) ) (Y. ) ) (Y. ) ) (Y.) If ) 1, runcaion (i.e. no knowing Y 0, Y 1, ec.) won' hur oo much. If ) 1, major problems Moral: In our example, choose ) œ - 1 so we can inver" he process, i.e. wrie i as long AR. Y 90 œ e.5e 1 Daa œ Ys œ 90.5e s (how o sar?) 1 One way: recursion wih se 0 œ 0 Y Y s (0) AR(p) Ys 8 œ 90.5(1) œ 90.5 error e8 Ys œ Ys œ Ys œ œ 90 œ.. error e +.5 e n n-1 (Y. ) œ! (Y. )! (Y. )! (Y. ) e 1 1 p p

6 Applied Time Series Noes ( 19 ) MA(q) Y œ. e ) e ) e 1 1 q q Covariance, MA(q) Y. œe ) e ) e ) e ) e Y. œ e ) e 1 1 j j j 1 j 1 q q j j 1 j 1 Covariance œ[ ) ) ) ) ) ] 5 (0 if j q) j 1 j 1 q j q Example j = #(j) = MA() 1 5 [1 ) ) ] œ 85 5 œ [ ) ) ) ] œ 18 ) œ [ ) ] œ 40 ) œ.4 Y œ e 1.3e.4e. 1 Can we wrie e œ(y. ) C 1(y 1. ) C (Y. )? Will C j die off exponenially? i.e. is his inverible? Backshif: Y œ. (1 1.3B.4B )e where 1 1 B(e) œ e, B (e) œ B(e ) œ e, ec. e œ (Y ) Formally 1 (1 1.3B.4B ) (1.5B)(1.8B) œ 1.5B 1.8B and 1 X œ1 X X X if X 1 so (1.5B.5B.15B, 8 3 (1.8B.64B.51B, 3 œ1 1.3B 1.9B œ1 C B Obviously C 's die off exponenially. j 1

7 Applied Time Series Noes AR() (Y. ).9(Y. ).(Y. ) œ e 1 ( 0 ) (1.5B)(1.4B)(Y. ) œ e (1.5B)(1.4B)(Y. ) œ e Righ away, we see ha (1.5X)(1.4X) œ 0 has all roos exceeding 1 in magniude so as we did wih MA(), we can wrie (Y ) œe Ce Ce. 1 1 wih C j dying off exponenially. Pas shocks" e j are no so imporan in deermining Y. p 1 p AR(p) (1! B! B! B )(Y. ) œe p If all roos of (1! 1m! m! pm ) œ0 have m 1, series is saionary (shocks emporary) 1 MA() Y. œ (1 ) B ) B )e. If all he roos of (1 ) 1m ) m ) œ 0 have m 1, series is inverible (can exrac e from Y's). Alernaive version of characerisic equaion (I prefer his) p p-1 p- 1 p m! m! m! = 0 saionary <=> roos <1. Mixed Models ARMA(p, q) Example: (Y. ).5(Y. ) œ e.8e 1 1 Y. œ [(1.8B)/(1.5B)]e œ(1.8b)(1.5b.5b.15b )e 3 œe 1.3e.65e.35e Yule-Walker equaions 1 3 (Y. )[(Y. ).5(Y. )] œ (Y. )(e.8e ) j 1 j 1 Take expeced value j œ 0 #(0).5 #(1) œ 5 (1 1.04) #(0)!# (1) = 5 Š 1 ) (! ) ) j œ 1 #(1).5 #(0) œ 5 (.8) #(1)!# (0) = -)5 j 1 #(j).5 #(j 1) œ 0

8 Applied Time Series Noes ( 1 ) Œ 1 #(0) œ #(1).5 1 Œ œ Œ j (j) ec. Define #( j) œ #(j), 3( j) œ 3(j). In general Yule-Walker relaes covariances o parameers. Two uses: (1) Given model, ge #(j) and 3(j) () Given esimaes of #(j) ge rough esimaes of parameers. Idenificaion - Par II Inverse Auocorrelaion IACF For he model (Y. )! (y. )! (Y. ) œe ) e ) e 1 1 p p 1 1 q q define IACF as ACF of he dual model: (Y. ) ) (Y. ) ) (Y. ) œe! e! e 1 1 q q 1 1 p p IACF of AR(p) is ACF of a MA(p) IACF of MA(q) is ACF of an AR(q) How do we esimae ACF, IACF, PACF from daa? n j Auocovariances s# (j) œ D (Y Y )(Y j Y ) În œ 1 ACF s3 (j) œs# (j) Îs# (0) PACF plug s# (j) ino formal defining formula and solve for C. IACF: Approximae by fiing long auoregression (Y. ) œ! s (Y. )! s (Y. ) e 1 1 k k Compue ACF of dual model Y. œe! ^ e! ^ e. 1 1 k k To fi he long auoregressive plug #s (j) ino Yule-Walker equaions for AR(k), or jus regress Y on Y, Y, â,y. j 1 k s j

9 Applied Time Series Noes ( ) All 3 funcions IACF, PACF, ACF compued in PROC ARIMA. How do we inerpre hem? Compare o caalog of heoreical IACF, PACF, ACF for AR, MA, and ARMA models. See SAS Sysem for Forecasing Time Series book for several examples - secion Variance for IACF, PACF approximaely 1 n For ACF, SAS uses Barle's formula. For 3s (j) his is j 1 1 n D Š s3(i) i= j 1 (Fuller gives Barle's formula as afer firs deriving a more accurae esimae of he variance of s3(i). The sum here is infinie so in SAS he hypohesis being esed is H 0: 3(j)=0 > assuming 3 (i)=0 for i>j. Assuming a MA of order no more han j, is he j auocorrelaion 0?) Synax PROC ARIMA; IDENTIFY VAR=Y (NOPRINT NLAG=10 CENTER); ESTIMATE P= Q=1 (NOCONSTANT NOPRINT ML PLOT); FORECAST LEAD=7 OUT=OUT1 ID=DATE INTERVAL=MONTH; (1) I, E, F will work. () Mus have I preceding E, E preceding F (3) CENTER subracs Y (4) NOCONSTANT is like NOINT in PROC REG (5) ML (maximum likelihood) akes more ime bu has slighly beer accuracy han he defaul leas squares. (6) PLOT gives ACF, PACF, IACF of residuals. Diagnosics: Box-Ljung chi-square on daa Y or residuals se. (1) Compue esimaed ACF 3s (j) () Tes is Q œ n(n ) D k Š s3(j) (n j) j œ 1 (3) Compare o ; disribuion wih k p q d.f. Š ARMA(p, q) ñ SAS (PROC ARIMA) will give Q es on original daa and on residuals from fied models.

10 Applied Time Series Noes ( 3 ) ñ Q saisics given in ses of 6, i.e. for j=1 o 6, for j=1 o 1, for j=1 o 18, ec. Noe ha hese are cumulaive ñ For original series H : Series is whie noise o sar wih. 0 ñ For residuals H : Residual series is whie noise. 0 Suppose residuals auocorrelaed - wha does i mean? Can predic fuure residuals from pas - hen why no do i? Model predics using correlaion. Auocorrelaed residuals => model has no capured all he predicabiliy in he daa. So... H 0: Model is sufficien vs. H 1: Needs more work <=> "lack of fi" es Le's ry some examples. All have his kind of header, all have 1500 obs. ARIMA Procedure Name of variable = Y1. Mean of working series = Sandard deviaion = Number of observaions = 1500

11 Y1 Applied Time Series Noes Auocorrelaions ( 4 ) Lag Covar Corr ******************** **************** ************* ********** ******** ******* ****** ***** **** "." marks wo sandard errors Inverse Auocorrelaions Lag Correlaion ********** * * Parial Auocorrelaions Lag Correlaion **************** * * * * Auocorrelaion Check for Whie Noise To Chi Auocorrelaions Lag Square DF Prob

12 Y Applied Time Series Noes Auocorrelaions ( 5 ) Lag Covar Corr ******************** **************** ************* ********** ******** ****** ***** **** *** "." marks wo sandard errors Inverse Auocorrelaions Lag Correlaion ********* * * Parial Auocorrelaions Lag Correlaion **************** * * Auocorrelaion Check for Whie Noise To Chi Auocorrelaions Lag Square DF Prob

13 Y3 Applied Time Series Noes Auocorrelaions ( 6 ) Lag Covar Corr ******************** ********** *********** ******* ******* ***** ***** *** *** "." marks wo sandard errors Inverse Auocorrelaions Lag Correlaion *** ****** * * * * Parial Auocorrelaions Lag Correlaion ********** ******** * * Auocorrelaion Check for Whie Noise To Chi Auocorrelaions Lag Square DF Prob

14 Y4 Applied Time Series Noes Auocorrelaions ( 7 ) Lag Covar Corr ******************** ********** *** ******** **** * *** * * "." marks wo sandard errors Inverse Auocorrelaions Lag Correlaion ************ ***** * * *. Parial Auocorrelaions Lag Correlaion ********** ********** * Auocorrelaion Check for Whie Noise To Chi Auocorrelaions Lag Square DF Prob

15 Y5 Applied Time Series Noes Auocorrelaions ( 8 ) Lag Covar Corr ******************** ********** * * * "." marks wo sandard errors Inverse Auocorrelaions Lag Correlaion **************** ************ ********* ****** **** ** * Parial Auocorrelaions Lag Correlaion ********** ******* **** *** *** ** ** Auocorrelaion Check for Whie Noise To Chi Auocorrelaions Lag Square DF Prob

16 Y6 Applied Time Series Noes Auocorrelaions ( 9 ) Lag Covar Corr ******************** ** ****** * * * * * "." marks wo sandard errors Inverse Auocorrelaions Lag Correlaion ********* ********** ******* ****** *** ** ** Parial Auocorrelaions Lag Correlaion ** ******* *** **** ** ** ** *. Auocorrelaion Check for Whie Noise To Chi Auocorrelaions Lag Square DF Prob

17 Y7 Applied Time Series Noes Auocorrelaions ( 30 ) Lag Covar Corr ******************** * * * * "." marks wo sandard errors Inverse Auocorrelaions Lag Correlaion * * * *. Parial Auocorrelaions Lag Correlaion * * * * Auocorrelaion Check for Whie Noise To Chi Auocorrelaions Lag Square DF Prob

18 Y8 Applied Time Series Noes Auocorrelaions ( 31 ) Lag Covar Corr ******************** *************** ********* ***** *** ** ** * *. "." marks wo sandard errors Inverse Auocorrelaions Lag Correlaion ************** ***** ** * * * Parial Auocorrelaions Lag Correlaion *************** ****** ** * * Auocorrelaion Check for Whie Noise To Chi Auocorrelaions Lag Square DF Prob

19 Applied Time Series Noes Back o Naional League example: ( 3 ) Winning Percenage for Naional League Pennan Winner Name of variable = WINPCT. Mean of working series = Sandard deviaion = Number of observaions = 73 Auocorrelaions Lag Covariance Correlaion ******************** ********* ********* **** *** *** ** ** ***. "." marks wo sandard errors Inverse Auocorrelaions Lag Correlaion **** ******* ** ** ** * * *. Parial Auocorrelaions Lag Correlaion ********* ****** ** * ** * **.

20 Applied Time Series Noes ( 33 ) Auocorrelaion Check for Whie Noise To Chi Auocorrelaions Lag Square DF Prob Looks like AR() or MA() may fi well. How o fi an MA? Daa: ( Mean = 10 ) Sum of squares for ) = -.5 Y = e - ) e-1 Y y = Y y ^ =.5 e e = y-y ^ sum of squared errors = = ) SS(err) so ) ^ -.5 A beer way: ` ^ Make SSq( ) = e ( ^ ` )! )) e () ^ ) = 0 ` ) ` ) How? If e ( ^ ` )) is a residual from a regression on e () ^ ) hen derivaive is 0 by orhogonaliy of residuals o regressors. Taylor's Series: e ( ) = e ( ^ ` ) ) ) + e () ^ ) ( ) - ) ^ ) + remainder ` ) Ignore remainder and evaluae a e (rue ) ) = whie noise 0 e ( ^ ` ) ) - e () ^ ) ( ) - ) ^ ) + e ( ) ) ` ) 0 0 Can calculae e ( ^ ` ) ) and - e () ^ ), error erm is whie noise! ` ) Esimae ( ) - ) ^ ) by regression and ierae o convergence. 0 ` )

21 Applied Time Series Noes ( 34 ) Also: Can show regression sandard errors jusified in large samples. 1. e ( ) ^ ) = Y + ) ^ e ( ) ^ ) for iniial ) ^ -1 `. e ( ^ ) = e ( ^ ) + ( ^ ` ) ) )) e () ^ ) ` ) -1 ` ) Regress sequence (1) on sequence (). Daa MA; * begin Harley modificaion ; hea = *.44376; call sympu('h', pu(hea,8.5)); ile "Using hea = &h " ; if _n_ = 1 hen do; e1=0; w1=0; end; inpu e = y + hea*e1; w = -e1 + hea*w1; oupu; reain; e1=e; w1=w; cards; ; proc prin noobs; var y e e1 w w1; proc reg; model e = w / noin; run; Using hea = Y E E1 W W Parameer Esimaes Parameer Sandard T for H0: Variable DF Esimae Error Parameer=0 Prob > T W

22 Applied Time Series Noes ( 35 ) Anoher way o esimae ARMA models is EXACT MAXIMUM LIKELIHOOD Gonzalez-Farias' disseraion uses his mehodology for nonsaionary series. AR(1): ( Y -. ) = 3 ( Y ) + e Y -. ~ N( 0, 5 /(1-3 ) ) 1 ( Y -. ) - 3 ( Y -1 -.) ~ N( 0, 5 ), =,3,,...,n Likelihood: _ = n-1 "! - # [(Y -. ) - 3 (Y ) ] / 5 1 e # Š e = È1-3 " - (Y -. ) (1-3 )/ 5 1 5È1 5È1 n Posiive in (-1,1) and 0 a +1, -1 => easy o maximize. Logarihms: " n " ln( _) = # ln (1-3 ) - ln [ 1 s ( 3) ] - # n n 1-1 = n-1 (Y 1+Y n)+(1-3)! Y = where s ( 3 ) = SSq / n and SSq = (Y -. ) (1-3 ) +![(Y -. ) - 3 (Y -. ) ]. =.( 3 ) = +(n-)(1-3) If 3 <1 hen choosing 3 o maximize ln( _) does no differ in he limi from choosing 3 n o minimize! [(Y -. ) - 3 (Y -. ) ]. = (leas squares and maximum likelihood are abou he same for large samples OLS MLE). Gonzalez-Farias shows MLE and OLS differ in a nonrivial way, even in he limi, when 3=1.

23 Applied Time Series Noes Example of MLE for Iron and Seel Expors daa: ( 36 ) DATA STEEL STEEL; ARRAY Y(44); n=44; pi = 4*aan(1); do =1 o n; inpu STEEL; Y()=EXPORT; end; Do RHO =.44 o.51 by.01; MU = (Y(1) + Y(n) + (1-rho)*sum(of Y-Y43) )/(+(1-rho)*4); SSq = (1-rho**)*(Y(1)-mu)**; Do = o n; SSq = SSq + (Y()-mu - rho*(y(-1)-mu) )**; end; lnl =.5*log(1-rho*rho) - (n/)*log(*pi*ssq/n) - n/; oupu Seel; end; drop y1-y44; CARDS; ; proc arima daa=seel; I var=expor noprin; e p=1 ml; proc plo daa=seel; plo lnl*rho/vpos=0 hpos=40; ile "Log likelihood for Iron Expors daa"; proc prin daa=seel; run; Log likelihood for Iron Expors daa ARIMA Procedure Maximum Likelihood Esimaion Approx. Parameer Esimae Sd Error T Raio Lag MU AR1,

24 Applied Time Series Noes ( 37 ) Plo of lnl*rho. Legend: A = 1 obs, B = obs, ec. lnl ˆ A A A A ˆ A A -81. ˆ A ˆ A ˆ Šƒˆƒƒƒƒˆƒƒƒƒˆƒƒƒƒˆƒƒƒƒˆƒƒƒƒˆƒƒƒƒˆƒƒƒƒˆƒƒƒ RHO IRON AND STEEL EXPORTS EXCLUDING SCRAPS WEIGHT IN MILLION TONS OBS N PI T EXPORT RHO MU SSQ LNL