Structural Break Detection in Time Series Models

Transcription

1 Srucural Break Deecion in Time Series Models Richard A. Davis Thomas Lee Gabriel Rodriguez-Yam Colorado Sae Universiy (hp:// This research suppored in par by: Naional Science Foundaion IBM faculy award EPA funded proec eniled: Space-Time Aquaic Resources Modeling and Analysis Program (STARMAP) The work repored here was developed un STAR Research Assisance Agreemens CR awarded by he U.S. Environmenal Proecion Agency (EPA) o Colorado Sae Universiy. This presenaion has no been formally reviewed by EPA. EPA does no endorse any producs or commercial services menioned in his presenaion.

2 Illusraive Example How many segmens do you see? τ = 5 τ = 5 τ 3 = 5 ime

3 Illusraive Example Auo-PARM=Auo-Piecewise AuoRegressive Modeling 4 pieces.58 seconds τ = 5 τ = 57 τ 3 = 59 ime 3

4 Inroducion Seup Examples AR GARCH Sochasic volailiy Sae space model Model Selecion Using Minimum Descripion Lengh (MDL) General principles Applicaion o AR models wih breaks Opimizaion using Geneic Algorihms Basics Implemenaion for srucural break esimaion Simulaion resuls Applicaions Simulaion resuls for GARCH and SSM 4

5 Inroducion The Premise (in a general framework): Base model: P θ family or probabiliy models for a saionary ime series. Observaions: y... y n Segmened model: here exis an ineger m 0 and locaions τ 0 = < τ <... < τ m- < τ m = n + such ha Y = X if τ - < τ where {X } is a saionary ime series wih probabiliy disr and θ θ +. Obecive: esimae P θ m = number of segmens τ = locaion of h break poin θ = parameer vecor in h epoch 5

6 . Piecewise AR model: Y Examples = γ + φ Y + L+ φp Y p + σ ε if τ - < τ where τ 0 = < τ <... < τ m- < τ m = n + and {ε } is IID(0). Goal: Esimae m = number of segmens τ = locaion of h break poin γ = level in h epoch p = order of AR process in h epoch ( φ = AR coefficiens in h K φ p ) epoch σ = scale in h epoch 6

7 Piecewise AR models (con) Srucural breaks: Kiagawa and Akaike (978) fiing locally saionary auoregressive models using AIC compuaions faciliaed by he use of he Householder ransformaion Davis Huang and Yao (995) likelihood raio es for esing a change in he parameers and/or order of an AR process. Kiagawa Takanami and Masumoo (00) signal exracion in seismology-esimae he arrival ime of a seismic signal. Ombao Raz von Sachs and Malow (00) orhogonal complex-valued ransforms ha are localized in ime and frequency- smooh localized complex exponenial (SLEX) ransform. applicaions o EEG ime series and speech daa. 7

8 Moivaion for using piecewise AR models: Piecewise AR is a special case of a piecewise saionary process (see Adak 998) m ~ Y = Y I ( / n) n [ τ τ ) = where { } =... m is a sequence of saionary processes. I is Y argued in Ombao e al. (00) ha if {Y n } is a locally saionary process (in he sense of Dahlhaus) hen here exiss a piecewise ~ saionary process } wih { Y n m n wih m n / n 0 as n ha approximaes {Y n } (in average mean square). Roughly speaking: {Y n } is a locally saionary process if i has a imevarying specrum ha is approximaely A(/nω) where A(uω) is a coninuous funcion in u. 8

9 . Segmened GARCH model: Examples (con) Y σ = σ ε = ω + α Y + L+ α p Y p +β σ + L+β q σ q if τ - < τ where τ 0 = < τ <... < τ m- < τ m = n + and {ε } is IID(0). 3. Segmened sochasic volailiy model: Y = σ ε logσ = γ + φ logσ + L+ φ p logσ p + ν η if τ - < τ. 4. Segmened sae-space model (SVM a special case): p( y α = γ α... α + φ α y... y + L+ φ ) = p p( y α p α ) is + σ η specified if τ - < τ. 9

10 Model Selecion Using Minimum Descripion Lengh Basics of MDL: Choose he model which maximizes he compression of he daa or equivalenly selec he model ha minimizes he code lengh of he daa (i.e. amoun of memory required o encode he daa). M = class of operaing models for y = (y... y n ) L F (y) = code lengh of y relaive o F M Typically his erm can be decomposed ino wo pieces (wo-par code) where L ( y) = L( Fˆ y) L(ˆ e Fˆ ) F + L( Fˆ y) L(ˆ e Fˆ ) = code lengh of he fied model for F = code lengh of he residuals based on he fied model 0

11 Model Selecion Using Minimum Descripion Lengh (con) Applied o he segmened AR model: Y Firs erm L( Fˆ y) : L = γ + φ Y + + φp Y p + σ ε if τ - < τ L( Fˆ y) = L(m) + L( τ K τ ) + L( p K p ) + L( ψˆ y) + L+ L( ψˆ = log m + mlog m n + m = log p + m m = p + log n m y) Encoding: ineger I : log I bis (if I unbounded) log I U bis (if I bounded by I U ) MLE θˆ : ½ log N bis (where N = number of observaions used o compue θˆ ; Rissanen (989))

12 Second erm L(ˆ e Fˆ ) : Using Shannon s classical resuls on informaion heory Rissanen demonsraes ha he code lengh of can be approximaed by he negaive of he log-likelihood of he fied model i.e. by L( eˆ Fˆ ) m = log L( ψˆ y) For fixed values of m (τ p )... (τ m p m ) we define he MDL as MDL( m( τ p ) K( τ p )) = log m m m + mlog n + m = log p + m m log n = = p + log L( ψˆ y) The sraegy is o find he bes segmenaion ha minimizes MDL(mτ p τ m p m ). To speed hings up for AR processes we use Y-W esimaes of AR parameers and we replace log L( ψˆ y) wih log (π σˆ ) + n

13 Opimizaion Using Geneic Algorihms Basics of GA: Class of opimizaion algorihms ha mimic naural evoluion. Sar wih an iniial se of chromosomes or populaion of possible soluions o he opimizaion problem. Paren chromosomes are randomly seleced (proporional o he rank of heir obecive funcion values) and produce offspring using crossover or muaion operaions. Afer a sufficien number of offspring are produced o form a second generaion he process hen resars o produce a hird generaion. Based on Darwin s heory of naural selecion he process should produce fuure generaions ha give a smaller (or larger) obecive funcion. 3

14 Applicaion o Srucural Breaks (con) Geneic Algorihm: Chromosome consiss of n genes each aking he value of (no break) or p (order of AR process). Use naural selecion o find a near opimal soluion. Map he break poins wih a chromosome c via ( m ( τ p ) K ( τ m p m)) c = ( δ K δn) ( m m δn where if nobreak poin a δ = p if break poin a ime = τ For example c = ( ) : 6 5 would correspond o a process as follows: andar order is p. AR() =:5; AR(0) =6:0; AR(0) =:4; AR(3) =5:0 4

15 Simulaion Examples-based on Ombao e al. (00) es cases. Piecewise saionary wih dyadic srucure: Consider a ime series following he model.9 Y if + ε if < 53 Y if =.69Y.8Y + ε if 53 < Y + ε if.8y if where {ε } ~ IID N(0) Time 7

16 . Piecewise sa (con) GA resuls: 3 pieces breaks a τ =53; τ =769. Toal run ime 6.3 secs Fied model: φ φ σ - 5: : : True Model Fied Model Time Time 8

17 Simulaion Examples (con). Slowly varying AR() model: Y = ay. 8 Y + ε if 04 where =. 8[ 0.5cos( π /04)] and {ε } ~ IID N(0). a a_ Time ime 9

18 . Slowly varying AR() (con) GA resuls: 3 pieces breaks a τ =93 τ =65. Toal run ime 7.45 secs Fied model: φ φ σ - 9: : : True Model Fied Model Time Time 0

19 . Slowly varying AR() (con) In he graph below righ we average he specogram over he GA fied models generaed from each of he 00 simulaed realizaions. True Model Average Model Frequency Time Time 4

20 Examples Speech signal: GREASY G R EA S Y Time 7

21 Speech signal: GREASY n = 576 observaions m = 5 break poins Run ime = 8.0 secs G R EA S Y Time Time 8

22 Applicaion o GARCH (con) Garch() model: Y σ = σ ε = ω + α Y { ε }~ IID(0) +β σ if τ - < τ σ.4+. Y =.4 +. Y # of CPs σ +.6 σ GA % if if < < 000. AG % CP esimae = 506 Time AG = Andreou and Ghysels (00)

23 Coun Daa Example Model: Y α Pois(exp{β+ α }) α = φα - + ε {ε }~IID N(0 σ ) y MDL True model: ime Breaking Poin Y α ~ Pois(exp{.7 + α }) α =.5α - + ε {ε }~IID N(0.3) < 50 Y α ~ Pois(exp{.7 + α }) α = -.5α - + ε {ε }~IID N(0.3) > 50. GA esimae 5 ime 67secs 30

24 SV Process Example Model: Y α N(0exp{α }) α = γ + φ α - + ε {ε }~IID N(0 σ ) y MDL ime Breaking Poin True model: Y α ~ N(0 exp{α }) α = α - + ε {ε }~IID N(0.05) 750 Y α ~ N(0 exp{α }) α = α - + ε {ε }~IID N(0.5) > 750. GA esimae 754 ime 053 secs 3

25 SV Process Example Model: Y α N(0exp{α }) α = γ + φ α - + ε {ε }~IID N(0 σ ) y MDL ime Breaking Poin True model: Y α ~ N(0 exp{α }) α = α - + ε {ε }~IID N(0.80) 50 Y α ~ N(0 exp{α }) α = α - + ε {ε }~IID N(0.0089) > 50. GA esimae 5 ime 69s 3

26 SV Process Example-(con) True model: Y α ~ N(0 exp{a }) α = α - + e {ε }~IID N(0.80) 50 Y α N(0 exp{α }) α = α - + ε {ε }~IID N(0.0089) > 50. Fied model based on no srucural break: Y α N(0 exp{α }) α = α - + ε {ε }~IID N(0.0935) y original series y simulaed series ime ime 33

27 SV Process Example-(con) Fied model based on no srucural break: Y α N(0 exp{α }) α = α - + ε {ε }~IID N(0.0935) y simulaed series MDL ime Breaking Poin 34

28 Summary Remarks. MDL appears o be a good crierion for deecing srucural breaks.. Opimizaion using a geneic algorihm is well suied o find a near opimal value of MDL. 3. This procedure exends easily o mulivariae problems. 4. While esimaing srucural breaks for nonlinear ime series models is more challenging his paradigm of using MDL ogeher GA holds promise for break deecion in parameer-driven models and oher nonlinear models. 35