3-Jul-3 Time Series Analysis Assoc. Prof. Dr. Sevap Kesel July 03 Saionary Time Series Sricly saionary process: If he oin dis. of is he same as he oin dis. of ( X,... X n) ( X h,... X nh) Weakly Saionary :{ X } is weakly saionary if i. ) E( X ) is independen of. x ( Cov( X, X ) ( r, s) E[( X ( r))( X ( s))] ii. r s x does no depend on. r x s x
3-Jul-3 Auocorrelaion Auocovariance Funcion (ACVF) ( h) Cov( X, X ) x h Auocorrelaion Funcion Properies:. ACF is an even funcion of he lag. ACF lies beween -, ( h) ( h) (0) ( h) ( h) ( h) Sample Auocovariance Funcion: Le X,X,..,X n be observaions of a series. Given he sample mean is X and n The sample auocovariance funcion h ˆ( h) ( X X )( X h n and he sample auocorrelaion funcion ˆ( h) ˆ( h) ˆ(0) he following is rue: Lemma: If he series is random, hen for large n, he sample ACF is approximaely normally disribued wih mean zero and variance (/n). ˆ( h) ~ N(0, ) n X )
3-Jul-3 Correlogram Correlogram is an aid o inerpre a se of ACF where, sample auocorrelaionsare ploed agains lag h. Remarks: For daa conaining rend will exhibi slow decay as h increases. For daa wih a periodic componen will exhibi similar behavior wih he same periodiciy. ˆ ~ N(0, ) k 3. Random Series: n 3.. If a ime series compleely random, for large n, 3.. For a random ime series If 95% of ˆ( h) he values lie wihin ime series is random. n ˆ( h) 0 Correlogram Shor-erm correlaion: Fairly large value of ˆ() is followed by or more coefficiens which is significanly smaller han zero, end o ge successively smaller. ˆ( h) ges o zero for large h. Alernaing series: Correlogram also ends o alernae. Non-saionary series: If he series conains a rend, ˆ( h) values will no come down o zero excep very large h. Trend should be removed firs. Seasonal flucuaions: Correlogram exhibi an oscillaion a he same frequency. If follows a sinusoidal paern, hen so does ˆ( h) 3
3-Jul-3 4 9 8 7 6 5 4 3.0 0.8 0.6 0.4 0. 0.0-0. -0.4-0.6-0.8 -.0 Auocorrelaion Auocorrelaion Funcion for reacion im 0 5 0 5.0 0.8 0.6 0.4 0. 0.0-0. -0.4-0.6-0.8 -.0 Auocorrelaion LBQ T Corr Lag LBQ T Corr Lag LBQ T Corr Lag 536.79 536.43 535.78 535.0 53.46 57.88 50. 506.5 486.83 465.0 445.04 46.95 405.75 38.39 354.3 33.5 88.78 44.93 89.8 3.79 0.0 0.7 0.30 0.55 0.74 0.98.3.6.73.69.64.8.0.8.4.70 3.4 4.0 5.5 0.57 0.04 0.06 0.07 0. 0.6 0. 0.8 0.34 0.35 0.34 0.33 0.35 0.38 0.40 0.43 0.46 0.5 0.58 0.68 0.84 0 9 8 7 6 5 4 3 0 9 8 7 6 5 4 3 Auocorrelaion Funcion for sales Examples o correlograms for differen series 45 35 5 5 5.0 0.8 0.6 0.4 0. 0.0-0. -0.4-0.6-0.8 -.0 Auocorrelaion LBQ T Corr Lag LBQ T Corr Lag LBQ T Corr Lag LBQ T Corr Lag 6.88 6.87 53.58 3.0 0.85 083.7 078.9 078.07 06. 0.48 966.83 93.60 97.95 97.94 9.58 893.0 864.96 848.7 845.93 843.67 85.88 759.0 684.7 63.97 60.39 608.74 606.60 590.8 565.70 550.90 549.0 545.08 5.36 44.37 334.40 56.8 8.80.53.40 96.09 70.47 57. 55.7 47.8 98.68-0.0-0.7 -.6 -.40 -. -0.54 0.4.04.70.0.63.08 0.03-0.7 -.3 -.55 -. -0.45 0.45.60.36.83.46.60 0.45-0.5 -.43 -.78 -.4-0.48 0.75. 3.39 4.67 4.4 3.3.34-0.57 -.7 -.94 -.8-0.7.7 4.8 9.85-0.0-0.8-0.30-0.36-0.8-0.4 0.06 0.6 0.4 0.49 0.39 0.5 0.0-0.7-0.9-0.36-0.8-0.0 0.0 0.36 0.5 0.59 0.50 0.3 0.09-0.0-0.8-0.35-0.7-0.09 0.4 0.4 0.59 0.74 0.63 0.44 0.8-0.08-0.8-0.37-0.7-0.09 0. 0.5 0.73 45 44 43 4 4 40 39 38 37 36 35 34 33 3 3 30 9 8 7 6 5 4 3 0 9 8 7 6 5 4 3 0 9 8 7 6 5 4 3 Auocorrelaion Funcion for raffic faa
3-Jul-3 Tesing Serial Correlaion DW -ρ When here is no serial correlaion, ρ=0 and DW saisic akes a value close o. Posiive serial correlaion produces a DW<, while negaive serial correlaion produces a DW>. DW n ( ) n 5
3-Jul-3 Example Tesing Serial Correlaion Pormaneau Tes: An imporan source of informaion in deecing he presence and form of serial correlaion is he correlogram. Qualiaive examinaion of he correlogram is an imporan diagnosic ool bu i does no consiue a formal saisical es. The Box-Pierce and is relaed es he Lung-Box es are boh pormaneu ess which allow us o es he hypohesis ha he firs h poins in he correlogram are random wih a rue value of zero. Box-Pierce es is defined as h Q n ˆ i i A beer sample saisics is Lung-Box saisics is h * Q n( n ) ˆ i i i ( ni) Q and Q* are disribued Chi-square wih degrees of freedom of h. 6
3-Jul-3 MA(q) Model Linear Process: X Whie Noise Process (WN): {X } is a sequence of i.i.d random variables wih zero mean and finie variance σ. The series is saionary wih γ(+h,)= σ for h=0. Whie Noise process is a purely random process where all auocorrelaion funcions for every h are close o zero. X i0 i, i 0... k k... 3 3 0 - - -3 5 50 75 00 5 50 75 X The plo of index numbers having Whie Noise model 4 7
3-Jul-3 Random Walk Random Walk Model: Le {S, =,..,n} be a process wih S =Σ where is WN. Then, E[S ]=0, Var[S ]=.σ and γ(+h,)=.σ Since γ(+h,)=.σ depends on, he series is no saionary. However, is saionary. X 5 6 8
3-Jul-3 Backward shif operaor Backward Shif Operaor, B, is anoher form of expressing he series BX X B( BX ) BX X B X B X X B 0 MA(q) Model Le { }~WN(0,σ ) MA(q) is X q 0 q For q= MA() E[ ]=0 V[ ]= σ (+θ ) X ( h, ) x x ( h) 0 ( ) 0 h 0 h h h 0 h h 8 9
3-Jul-3 9 AR(p) Model and is auocorrelaion Le { }~WN(0,σ ) AR(p) is For p= AR() E[ ]=0 V[ ]= σ /(-Φ ) X X X X... p p X X h ( h x ) h ( h ) x 0 0
3-Jul-3 AR() Model From he oupu, he behavior of ACF for AR() is ails off; he behavior of PACF for AR() is cus off afer lag.
3-Jul-3 ARMA(p,q) Model Le { }~WN(0,σ ) ARMA(p,q) is X X X... X.. p p q q ( B)( X ) ( B) where B B B B ( )... ( B) B.. B q q p p ARMA(,) X ( X ) ( B)( X ) ( B) where ( B) B ( B) B 3 Parial Auocorrelaion PACF For an AR(p) process PACF, Φ hh is he correlaion beween X and X -h conrolling he effec of X -h- AR(): Φ = Φ=ρ() AR(): Φ = Φ=ρ() hh () () () 0 h 4
3-Jul-3 PACF AR(p) (); h h, () () ; (), hh h, h,,.., h hh ( h) h h h, h, ( ) ( ), h 3,4,... Yule Walker Equaions ( h) ( h ) ( h ) ( h p) ( h) ( h ) ( h ) p p ( h p) h 0 h 0 5 h ( ) for h 0 hh ( h) Parial Auocorrelaion for MA() process 4 6 33 8 3 ( ) ( ) ( ) Asympoic disribuion of Parial Auocorrelaions For a causal AR(p) process, he asympoic disribuion n ˆ d N(0,) kk 6 3
X 3-Jul-3 Properies of he ACF and PACF for various ARMA Models Model ACF PACF AR() Exponenial or =0 for h> kk oscillaory decay AR() Exponenial or sine kk =0 for h> wave decay AR(p) Exponenial or sine kk =0 for h>p wave decay MA() h 0 =0 for h> Dominaed by damped exponenial MA() =0 for h> h 0 Dominaed by damped exponenial or sine wave MA(q) =0 for h>q h 0 Dominaed by linear combinaion of damped exponenial and/or sine waves ARMA(,) Tails off. Exponenial Tails off. Dominaed by exponenial decay from lag decay from lag ARMA(p,q) Tails off afer (q-p) Tails off afer (p-q) lags. Dominaed by lags. Exponenial damped exponenials and or sine waves and/or sine wave decay afer (p-q) lags afer (q-p) lags 7 Example: Yield Daa Yield Morgages-Yield on Gov. Loan Monhly) ACF.00.50.00 0.50 0.00 year 3 6 39 5 65 78 9 monhs 04 7 30 43 56 0.8 0.6 0.4 0. 0 3 5 7 9 3 5 h 7 9 lag Parial Auocorrelaion Funcion for lag h=,,3,..,0 h 3 4 5 6 7 8 9 0 0.84-0.083 0. 0.036 0.08 0.09 0.05 0.035 0.003-0.044 kk h 3 4 5 6 7 8 9 0 0.68 0.00 0.07-0.0-0.08-0.057 0.007-0.5 0. -0.07 kk PACF,00 0,80 0,60 0,40 0,0 0,00-0,0-0,40 3 4 5 6 7 8 9 0 3 4 5 6 7 8 9 0 8 4
3-Jul-3 Table 6: Oupu of he daa se Dependen Variable: YIELD_DATA Mehod: Leas Squares Dae: 0//08 Time: 00:4 Variable Coefficien Sd. Error -Saisic Prob. C.00303 0.07904.6940 0.0000 AR() 0.84074 0.04868 9.609 0.0000 R-squared 0.7937 Mean dependen var 0.994586 Adused R-squared 0.7085 S.D. dependen var 0.9556 S.E. of regression 0.575 Akaike info crierion -0.84987 Sum squared resid 3.8383 Schwarz crierion -0.80354 Log likelihood 68.66904 F-saisic 384.957 Durbin-Wason sa.860085 Prob(F-saisic) 0.000000 Yˆ.003 0.84Y 9 Saionary Condiions: Causaliy The esimaed values of parameers have cerain condiions. X X where X ( X ) afer subsiuions X ( B B...) X ( ) ( B) B In general, AR(p) is causal if he roos of Φ(B) lie in uni circle p p (.. p ) 0 X B B B B X ( ) ( B) B B 30 5
3-Jul-3 Inveribiliy The esimaed values of parameers have cerain condiions. X B ( B) X ( ) X ( B) B B In general, MA(q) is inverible if he roos of θ(b) lie in uni circle X... q q X B B.. B q q q q (.. q ) 0 X B B B B X ( ) ( B) B B 3 Exercise Dependen Variable: SERIES0 Variable Coefficien Sd. Error -Saisic Prob. AR() 0.854304 0.086835 9.838303 0.0000 MA() -0.9478 0.35943 -.63984 0.039 MA() -0.0486 0.44-0.39664 0.695 R-squared 0.49460 Mean dependen var -0.740 Adused R-squared 0.4836 S.D. dependen var.63834 S.E. of regression.7754 Akaike info crierion 3.939 Sum squared resid 33.064 Schwarz crierion 3.755 Log likelihood -55.0986 Durbin-Wason sa.9979 Invered AR Roos.85 Invered MA Roos.4 -. 3 6
3-Jul-3 Dependen Variable: SERIES0 Variable Coefficien Sd. Error -Saisic Prob. AR() 0.9679 0.43683.00340 0.030 AR() -0.090880 0.39854-0.7555 0.7835 MA() -0.4044 0.44466-0.99455 0.37 R-squared 0.49543 Mean dependen var -0.75980 Adused R-squared 0.48466 S.D. dependen var.646460 S.E. of regression.8997 Akaike info crierion 3.043 Sum squared resid 3.76 Schwarz crierion 3.8554 Log likelihood -53.987 Durbin-Wason sa.965834 Invered AR Roos.85. Invered MA Roos.4 33 7