MULTIVARIATE TIME SERIES ANALYSIS AND FORECASTING Manfred Deisler E O S Economerics and Sysems Theory Insiue for Mahemaical Mehods in Economics Universiy of Technology Vienna Singapore, May 2004
Inroducion Time Series: Observaions ordered in ime, informaion conained in ordering: Resuls are no permuaion-invarian in general. Discree ime, equally spaced daa: x, = 1 K T; x R n Main quesions in TSA: Daa driven modelling (sysem idenificaion) Signal and feaure exracion (e.g. seasonal adusmen)
Theory and mehods concern: Model classes Esimaion and inference Model selecion Model evaluaion Areas of applicaion: Signal processing e.g. speech, sonar and radar signals Daa driven modelling for simulaion and conrol of echnical sysems and processes; monioring Time series economerics: Macroeconomerics, finance economerics, applicaions o markeing and firm daa Medicine and biology: Geneics, EEG daa, monioring...
Tuorial Lecure 1: Saionary Processes Saionary processes are an imporan model class for ime series Def: A sochasic process ( for all cons for all n x)( ), x : Ω, is called (wide sense saionary if (i) ' Exx < (ii) E x = m = (iii) n ' γ ( s) = E( x+ s m)( x m) does no depend on Shif invariance of firs and second momens γ : Z R x nxn covariance funcion of ( x ) conains all linear dependence relaions beween process variables
γ (0) K γ ( T + 1) Γ = T K K K γ γ ( T 1) K (0) A funcion γ : Z nxn is called nonnegaive definie if Γ T 0 T Mahemaical characerizaion of covariance funcions of saionary processes: γ is a covariance funcion if and only if γ is nonnegaive definie
Examples for saionary processes (1) Whie noise ε = 0 E E ε ε = δ Σ, Σ 0 no (linear) memory ' s s (2) Moving average (MA) process q x bε = = Σ ; nxm 0 b finie memory (3) Infinie moving average process x = Σ b ε large class of saionary processes =
(4) (Saionary) Auoregressive (AR) process Seady sae soluion of sable VDE of he form p Σ ax = ε ; = 0 nxn a de az ( ) 0 z 1, a( z) a z p =Σ = 0 (5) (Saionary) ARMA process Seady sae soluion of sable VDE of he form azx ( ) = bz ( ) ε z K backwardshif ; bz ( ) =Σ bz q
(6) Harmonic process h ( h 1)/2 i λ + = Σ = Σ cos ϒ + sin ϒ = 0 = 1 x e z a b where λ [ ππ, ] (angular) frequencies, ϒ = λ = 1 ( h 1)/2 /2 [ 0, T + h K + ] z : Ω C, a, b : Ω n n π : Nyquis frequency; ( x ) is a weighed sum of harmonic oscillaions wih random weighs (ampliudes and phases) Saionariy condiions: 0 for : λ 0 * * Ez z <, Ez =, Ez z 1 = 0 1 Ex for : λ = 0
Specral disribuion funcion for a harmonic process [ ] nxn F : π, π : F( λ) = F ; F = Ez z : λ λ * F γ has he same informaion as γ abou he process, however displayed in a differen way
Specral represenaion of saionary process Every saionary process is he (poinwise in ) limi of a sequence of harmonic processes: Theorem: Every saionary process ( ) iλ x e dz( λ ) where [ ] = [ ππ, ] n ( z( λ ) λ π, π ), z( λ): Ω saisfies ( ) x can be represened as z z x Ez z z z * ( π ) = 0, ( π) = 0, ( λ) λ <, lim ( λ + ε) = ( λ), ε 0 ( ( * 4) ( 3))( ( 2) ( 1)) = 0 for λ 1 < λ2 λ3 < λ4 { } E z λ z λ z λ z λ and is unique for given ( x )
Specral disribuion of a general saionary process * [ ] nxn = F : π, π : F( λ) Ez( λ) z ( λ) Specral densiy If γ ( ) < hen here exiss a funcion f :[ π, π ] nxn λ s.. F( λ ) = f( ω ) dw, called he specral densiy π We have iλ γ () = e f( λ ) dλ 1 iλ ( λ ) = (2 π). γ( ) = f e
f is characerized by f 0 λ a.e., f( λ) dλ < and f( λ ) = f( λ )' π In paricular we have γ (0) = f( λ) dλ: Variance decomposiion π The diagonal elemens of f show he conribuions of he frequency bands o he variance of he respecive componen process and he off-diagonal elemens show he frequency band specific covariances and expeced phase shifs beween componen processes.
Parameric Esimaion e.g. AR esimaion for given p Seminonparameric Esimaion e.g. AR esimaor where in addiion p is esimaed Nonparameric Esimaion e.g. Windowed specral esimaion The curse of dimensionaliy : E.g. for AR esimaion (wih given p) he dimension of he parameer space is np ² (for he a ) plus nn+ ( 1) 2 for Σ
Linear ransformaions of saionary processes 2 y = a x ; a nxm, a <, ( x ) = linear, dynamic, ime invarian, sable sysem n saionary ( x ) ( y ) Weighing funcion ( a Z ) Causaliy: a = 0 < 0
iλ iλ( ) iλ iλ = y( ) = x( ) = ( ) x( ) = y e dz λ a e dz λ e a e dz λ 14243 frequency-specific gain and phase shif kz ( ) = az ( a Z ) = ransfer funcion Transformaion of second momens i λ f ( λ ) = k( e ) f ( λ ) yx x iλ * iλ y λ = x λ f ( ) k( e ) f ( ) k ( e ) 0
Soluion of linear vecor difference equaions (VDE s) ay o nxn + ay 1 1+ K+ ay p p = bx 0 + K + bx q q ; a b nxm, or: azy ( ) = bzx ( ) where p q = 0 = 0 = = az ( ) az, bz ( ) bz z: z as well as backward-shif: zy ( ) = ( y 1 )
Seady sae soluion: z ransform If de az ( ) 0 z 1 hen here exiss a causal sable soluion y = k x = 0 1 1 kz = kz ( ) = a ( zbz ) ( ) = (de az ( )) adaz ( ( )) bz ( ); z 0 where
Forecasing for saionary processes Problem: Approximaion of a fuure value x+ h, h > 0 from he pas x, s s Linear leas squares forecasing: * min Ex ( ax ) ( x ax ) a nxn + h + h 0 Proecion inerpreaion Predicion from a finie pas; 1 x, x,, x K r r ' ( + h ) s = 0, = 0, K, = 0 Ex ax x s r
leads o γ (0) K γ ( r ) ( ao, K, ar) O = ( γ( h), K, γ( h + r)) γ( r ) γ(0) xˆ, h = a x Predicor Predicion from an infinie pas; x, x 1, K A saionary process is called (linearly) singular if xˆ, h = x + h for some and hence for all h>, 0 Here x ˆ, h denoes he bes linear leas squares predicor from he infinie pas.
A saionary process is called (linearly) regular if lim x ˆ h, = 0 h for one and hence for all. Theorem (Wold) (i) Every saionary process ( ) x can be represened in a unique way as x = y + z where ( y ) is regular, ( z ) is singular, ' Ey z s = 0 and ( x ) (ii) Every regular process ( ) y and z are causal linear ransformaions of y = k ε ; k ² <,( ε ) = 0 y can be represened as: whie noise and where causal linear ransformaion of ( y ) ε is a
Consequences for forecasing: (i) ( ) (ii) for he regular process ( ) y and ( z ) can be forecased separaely predicor y y we have: = = + y+ h kε + h kε + h kε + h = 0 = h = 0 14243 14243 ˆ, h h 1 predicion error Noe: Every regular process can be forecased wih arbiray accuracy by an ( AR ) MA process How do we obain he Wold represenaion: Specral facorizaion 1 * ' (2 ) ( i λ f k e ) k( e i λ = π Σ ), Σ = Eεε y