Structural Break Detection for a Class of Nonlinear Time Series Models

Transcription

1 Srucural Break Deecion for a Class of Nonlinear Time Series Models Richard A. Davis Thomas Lee Gabriel Rodriguez-Yam Colorado Sae Universiy (hp:// This research suppored in par by an IBM faculy award. SCRA 006 FIM XIII-PORTUGAL

2 Illusraive Example How many segmens do you see? SCRA 006 FIM XIII-PORTUGAL τ = 5 τ = 5 τ 3 = 5 ime

3 Illusraive Example Auo-PARM=Auo-Piecewise AuoRegressive Modeling 4 pieces.58 seconds SCRA 006 FIM XIII-PORTUGAL τ = 5 τ = 57 τ 3 = 59 ime 3

4 A Second Example Any breaks in his series? -4-0 SCRA 006 FIM XIII-PORTUGAL Time 4

5 Inroducion Examples AR GARCH Sochasic volailiy Sae space models Model selecion using Minimum Descripion Lengh (MDL) General principles Applicaion o AR models wih breaks Opimizaion using a Geneic Algorihm Basics Implemenaion for srucural break esimaion Simulaion resuls Applicaions Simulaion resuls for GARCH and SV models SCRA 006 FIM XIII-PORTUGAL 5

6 . Piecewise AR model: Y Examples = γ + φ Y + L+ φp Y p + σ ε if τ - < τ where τ 0 = < τ <... < τ m- < τ m = n + and {ε } is IID(0). Goal: Esimae m = number of segmens τ = locaion of h break poin γ = level in h epoch p = order of AR process in h epoch ( φ = AR coefficiens in h K φ p ) epoch σ = scale in h epoch SCRA 006 FIM XIII-PORTUGAL 6

7 . Segmened GARCH model: Examples (con) Y σ = σ ε = ω + α Y + L+ α p Y p +β σ + L+β q σ q if τ - < τ where τ 0 = < τ <... < τ m- < τ m = n + and {ε } is IID(0). 3. Segmened sochasic volailiy model: Y = σ ε logσ = γ + φ logσ + L+ φ p logσ p + ν η if τ - < τ. 4. Segmened sae-space model (SVM a special case): p( y α = γ α... α + φ α y... y + L+ φ ) = p p( y α p α ) is + σ η specified if τ - < τ. SCRA 006 FIM XIII-PORTUGAL 7

8 Model Selecion Using Minimum Descripion Lengh Basics of MDL: Choose he model which maximizes he compression of he daa or equivalenly selec he model ha minimizes he code lengh of he daa (i.e. amoun of memory required o encode he daa). M = class of operaing models for y = (y... y n ) L F (y) = code lengh of y relaive o F M Typically his erm can be decomposed ino wo pieces (wo-par code) where L ( y) = L( Fˆ y) L(ˆ e Fˆ ) F + L( Fˆ y) L(ˆ e Fˆ ) = code lengh of he fied model for F = code lengh of he residuals based on he fied model SCRA 006 FIM XIII-PORTUGAL 8

9 Model Selecion Using Minimum Descripion Lengh (con) Applied o he segmened AR model: Y Firs erm L( Fˆ y) : = γ + φ Y + L+ φp Y p + σ ε if τ - < τ L( Fˆ y) = L(m) + L( τ K τ ) + L( p K p ) + L( ψˆ y) + L+ L( ψˆ = log Second erm L(ˆ e Fˆ ) : m + mlog m n + m = log p + m m = p + log n m y) L( eˆ Fˆ ) MDL( m( τ p ) K( τ = log m = m p m + mlog log L( ψˆ y) m )) n + m = log p + m m log n + = = p + (log L( ( πσ ψˆ ˆ ) y+ ) n ) SCRA 006 FIM XIII-PORTUGAL 9

10 Opimizaion Using Geneic Algorihm Basics of GA: Class of opimizaion algorihms ha mimic naural evoluion. Sar wih an iniial se of chromosomes or populaion of possible soluions o he opimizaion problem. Paren chromosomes are randomly seleced (proporional o he rank of heir obecive funcion values) and produce offspring using crossover or muaion operaions. Afer a sufficien number of offspring are produced o form a second generaion he process hen resars o produce a hird generaion. Based on Darwin s heory of naural selecion he process should produce fuure generaions ha give a smaller (or larger) obecive funcion. SCRA 006 FIM XIII-PORTUGAL 0

11 Opimizaion Using Geneic Algorihm Geneic Algorihm: Chromosome consiss of n genes each aking he value of (no break) or p (order of AR process). Use naural selecion o find a near opimal soluion. Map he break poins wih a chromosome c via ( m ( τ p ) K ( τ m p m)) c = ( δ K δn) ( m m δn where if nobreak poin a δ = p if break poin a ime = τ For example c = ( ) : 6 5 would correspond o a process as follows: andar order is p. AR() =:5; AR(0) =6:0; AR(0) =:4; AR(3) =5:0 SCRA 006 FIM XIII-PORTUGAL

12 Implemenaion of Geneic Algorihm (con) Generaion 0: Sar wih L (00) randomly generaed chromosomes c... c L wih associaed MDL values MDL(c )... MDL(c L ). Generaion : A new child in he nex generaion is formed from he chromosomes c... c L of he previous generaion as follows: wih probabiliy π c crossover occurs. wo paren chromosomes c i and c are seleced a random wih probabiliies proporional o he ranks of MDL(c i ). k h gene of child is δ k = δ ik w.p. ½ and δ k w.p. ½ wih probabiliy π c muaion occurs. a paren chromosome c i is seleced k h gene of child is δ k = δ ik w.p. π ; w.p. π ;and p w.p. π π. SCRA 006 FIM XIII-PORTUGAL

13 Implemenaion of Geneic Algorihm (con) Execuion of GA: Run GA unil convergence or unil a maximum number of generaions has been reached.. Various Sraegies: include he op en chromosomes from las generaion in nex generaion. use muliple islands in which populaions run independenly and hen allow migraion afer a fixed number of generaions. This implemenaion is amenable o parallel compuing. SCRA 006 FIM XIII-PORTUGAL 3

14 Simulaion Examples-based on Ombao e al. (00) es cases. Piecewise saionary wih dyadic srucure: Consider a ime series following he model.9 Y if + ε if < 53 Y if =.69Y.8Y + ε if 53 < Y + ε if.8y if where {ε } ~ IID N(0) SCRA 006 FIM XIII-PORTUGAL Time 4

15 . Piecewise sa (con) Implemenaion: Sar wih NI = 50 islands each wih populaion size L = 00. Afer every Mi = 5 generaions allow migraion. 4 Replace wors in Island 34 wih bes from Island Sopping rule: Sop when he max MDL does no change for 0 consecuive migraions or afer 00 migraions. Span configuraion for model selecion: Max AR order K = 0 p m p π / p / / / / / / / / SCRA 006 FIM XIII-PORTUGAL 5

16 . Piecewise sa (con) GA resuls: 3 pieces breaks a τ =53; τ =769. Toal run ime 6.3 secs Fied model: φ φ σ - 5: : : True Model Fied Model Time Time SCRA 006 FIM XIII-PORTUGAL 6

17 Simulaion Examples (con). Slowly varying AR() model: Y = ay. 8 Y + ε if 04 where =. 8[ 0.5cos( π /04)] and {ε } ~ IID N(0). a a_ SCRA 006 FIM XIII-PORTUGAL Time ime 7

18 . Slowly varying AR() (con) GA resuls: 3 pieces breaks a τ =93 τ =65. Toal run ime 7.45 secs Fied model: φ φ σ - 9: : : True Model Fied Model SCRA 006 FIM XIII-PORTUGAL Time Time 8

19 . Slowly varying AR() (con) In he graph below righ we average he specogram over he GA fied models generaed from each of he 00 simulaed realizaions. True Model Average Model Frequency Time Time SCRA 006 FIM XIII-PORTUGAL 9

20 Simulaion Examples (con) 3. Simulaed daa from Fearnhead (005): True model has 9 changepoins black=rue red=aparm green=map SCRA 006 FIM XIII-PORTUGAL Time MAP es of m=9 while MAP of m and changepoin locaions gives m= 8 changeps. Plo is condiional on 9 changepoins. 0

21 4. Fearnhead example True Model Fied APARM Model Time Time SCRA 006 FIM XIII-PORTUGAL

22 Theory Consisency. Suppose he number of change poins m is known and le λ =τ /n... λ m =τ m /n be he relaive (rue) changepoins. Then λˆ λ a.s. where λ ˆ = ˆ /n and τ = Auo-PARM esimae of τ. τ ˆ Consisency of he esimae of m? For n large Auo-PARM esimae is m. Have no proved equaliy. SCRA 006 FIM XIII-PORTUGAL

23 Examples Mine explosion seismic race in Scandinavia: (Shumway and Soffer 000 Soffer e al. 005) Two waves: P (primary) compression wave and S (shear) wave Time SCRA 006 FIM XIII-PORTUGAL 5

24 Examples AR orders: SCRA 006 FIM XIII-PORTUGAL Time 6

25 Example: EEG Time series Daa: Bivariae EEG ime series a channels T3 (lef emporal) and P3 (lef parieal). Female subec was diagnosed wih lef emporal lobe epilepsy. Daa colleced by Dr. Beh Malow and analyzed in Ombao e al (00). (n=3768; sampling rae of 00H). Seizure sared a abou.85 seconds. GA GA bivariae univariae resuls: resuls: pieces 4 breakpoins wih AR orders for T3; 7 breakpoins for 5 9 P35 4 T3 Channel P3 Channel EEG T3 channel EEG P3 channel SCRA 006 FIM XIII-PORTUGAL Time in seconds Time in seconds 9

26 Remarks: Example: EEG Time series (con) he general conclusions of his analysis are similar o hose reached in Ombao e al. prior o seizure power concenraed a lower frequencies and hen spread o high frequencies. power reurned o he lower frequencies a conclusion of seizure. T3 Channel P3 Channel Frequency (Herz) Frequency (Herz) SCRA 006 FIM XIII-PORTUGAL Time in seconds Time in seconds 30

27 Example: EEG Time series (con) Remarks (con): T3 and P3 srongly coheren a 9- Hz prior o seizure. srong coherence a low frequencies us afer onse of seizure. srong coherence shifed o high frequencies during he seizure. T3/P3 Coherency Frequency (Herz) SCRA 006 FIM XIII-PORTUGAL Time in seconds 3

28 Applicaion o GARCH Garch() model: Y σ = σ ε = ω + α Y { ε }~ IID(0) +β σ if τ - < τ σ.4+. Y =.4 +. Y # of CPs σ +.6 σ GA % if if < < 000. AG % CP esimae = 506 Time AG = Andreou and Ghysels (00) SCRA 006 FIM XIII-PORTUGAL 3

29 Applicaion o GARCH (con) Garch() model: Y σ = σ ε = ω + α Y { ε }~ IID(0) +β σ if τ - < τ σ.4 +. Y =.4 +. Y # of CPs σ +.8 σ GA % if if < < 000. AG % CP esimae = 50 Time AG = Andreou and Ghysels (00) SCRA 006 FIM XIII-PORTUGAL 33

30 Applicaion o GARCH (con) More simulaion resuls for Garch() : Y = σ ε { ε }~ IID(0) σ.05+.4y =.00+.3Y +.3σ +.σ if < τ if τ < 000. τ Mean SE Med Freq 50 GA Berkes GA Berkes GA Berkes Berkes = Berkes Gombay Horvah and Kokoszka (004). SCRA 006 FIM XIII-PORTUGAL 34

31 Applicaion o Parameer-Driven SS Models Sae Space Model Seup: Observaion equaion: p(y α ) = exp{α y b(α ) + c(y )}. Sae equaion: {α } follows he piecewise AR() model given by α = γ k +φ k α - +σ k ε if τ k- < τ k where = τ 0 < τ < < τ m < n and {ε } ~ IID N(0). Parameers: m = number of break poins τ k = locaion of break poins γ k = level in k h epoch φ k = AR coefficiens k h epoch σ k = scale in k h epoch SCRA 006 FIM XIII-PORTUGAL 35

32 Applicaion o Srucural Breaks (con) Esimaion: For (m τ... τ m ) fixed calculae he approximae likelihood evaluaed a he MLE i.e. / Gn T * T * * T * La ( ψˆ;y n ) = exp{y nα { b( α ) c(yn)} ( α µ ) Gn ( α µ )/} ( K + G ) n / where ˆ ˆ ψˆ = (ˆ γ K γˆ φ K φ σˆ K σˆ ) is he MLE. m m m Remark: The exac likelihood is given by he following formula L( ψ;yn ) = La ( ψ;yn ) Era ( ψ) where Er ( ψ) = exp{ R( α ; α*)} p ( α y ; ψ) dα. a n I urns ou ha log( Er a ( ψ)) is nearly linear and can be approximaed by a linear funcion via imporance sampling e ψ ) ~ e( ψˆ ) + e& ( ψˆ a n )( ψ ψˆ ( AL AL AL ) n n SCRA 006 FIM XIII-PORTUGAL 36

33 Coun Daa Example Model: Y α Pois(exp{β+ α }) α = φα - + ε {ε }~IID N(0 σ ) y MDL True model: ime Breaking Poin Y α ~ Pois(exp{.7 + α }) α =.5α - + ε {ε }~IID N(0.3) < 50 Y α ~ Pois(exp{.7 + α }) α = -.5α - + ε {ε }~IID N(0.3) > 50. GA esimae 5 ime 67secs SCRA 006 FIM XIII-PORTUGAL 37

34 SV Process Example Model: Y α N(0exp{α }) α = γ + φ α - + ε {ε }~IID N(0 σ ) y MDL ime Breaking Poin True model: Y α ~ N(0 exp{α }) α = α - + ε {ε }~IID N(0.05) 750 Y α ~ N(0 exp{α }) α = α - + ε {ε }~IID N(0.5) > 750. GA esimae 754 ime 053 secs SCRA 006 FIM XIII-PORTUGAL 38

35 SV Process Example Model: Y α N(0exp{α }) α = γ + φ α - + ε {ε }~IID N(0 σ ) y MDL ime Breaking Poin True model: Y α ~ N(0 exp{α }) α = α - + ε {ε }~IID N(0.80) 50 Y α ~ N(0 exp{α }) α = α - + ε {ε }~IID N(0.0089) > 50. GA esimae 5 ime 69s SCRA 006 FIM XIII-PORTUGAL 39

36 SV Process Example-(con) True model: Y α ~ N(0 exp{a }) α = α - + e {ε }~IID N(0.80) 50 Y α N(0 exp{α }) α = α - + ε {ε }~IID N(0.0089) > 50. Fied model based on no srucural break: Y α N(0 exp{α }) α = α - + ε {ε }~IID N(0.0935) y original series y simulaed series ime ime SCRA 006 FIM XIII-PORTUGAL 40

37 SV Process Example-(con) Fied model based on no srucural break: Y α N(0 exp{α }) α = α - + ε {ε }~IID N(0.0935) y simulaed series MDL ime Breaking Poin SCRA 006 FIM XIII-PORTUGAL 4