Structural Break Detection in Time Series Models

Transcription

1 Srucural Break Deecion in Time Series Models Richard A. Davis Thomas Lee Gabriel Rodriguez-Yam Colorado Sae Universiy (hp:// This research suppored in par by an IBM faculy award.

2 Illusraive Example How many segmens do you see? τ = 5 τ = 5 τ 3 = 5 ime

3 Illusraive Example Auo-PARM=Auo-Piecewise AuoRegressive Modeling 4 pieces,.58 seconds τ = 5 τ = 57 τ 3 = 59 ime 3

4 A Second Example Any breaks in his series? Time 4

5 Inroducion Examples AR GARCH Sochasic volailiy Sae space models Model selecion using Minimum Descripion Lengh (MDL) General principles Applicaion o AR models wih breaks Opimizaion using a Geneic Algorihm Basics Implemenaion for srucural break esimaion Simulaion resuls Applicaions Simulaion resuls for GARCH and SV models 5

6 . Piecewise AR model: Y Examples = γ + φ Y + L+ φp Y p + σ ε, if τ - < τ where τ 0 = < τ <... < τ m- < τ m = n +, and {ε } is IID(0,). Goal: Esimae m = number of segmens τ = locaion of h break poin γ = level in h epoch p = order of AR process in h epoch ( φ = AR coefficiens in h, K, φ p ) epoch σ = scale in h epoch, 6

7 . Segmened GARCH model: Examples (con) Y σ = σ ε = ω, + α Y + L+ α p Y p +β σ + L+β q σ q, if τ - < τ, where τ 0 = < τ <... < τ m- < τ m = n +, and {ε } is IID(0,). 3. Segmened sochasic volailiy model: Y = σ ε logσ, = γ + φ logσ + L+ φ p logσ p + ν η, if τ - < τ. 4. Segmened sae-space model (SVM a special case): p( y α = γ α,..., α + φ α, y,..., y + L+ φ ) = p p( y α p α ) is + σ η, specified if τ - < τ. 7

8 Model Selecion Using Minimum Descripion Lengh Basics of MDL: Choose he model which maximizes he compression of he daa or, equivalenly, selec he model ha minimizes he code lengh of he daa (i.e., amoun of memory required o encode he daa). M = class of operaing models for y = (y,..., y n ) L F (y) = code lengh of y relaive o F M Typically, his erm can be decomposed ino wo pieces (wo-par code), where L ( y) = L( Fˆ y) L(ˆ e Fˆ ), F + L( Fˆ y) L(ˆ e Fˆ ) = code lengh of he fied model for F = code lengh of he residuals based on he fied model 8

9 Model Selecion Using Minimum Descripion Lengh (con) Applied o he segmened AR model: Y Firs erm L( Fˆ y) : = γ + φ Y + L+ φp Y p + σ ε, if τ - < τ, L( Fˆ y) = L(m) + L( τ, K, τ ) + L( p, K, p ) + L( ψˆ y) + L+ L( ψˆ = log Second erm L(ˆ e Fˆ ) : m + mlog m n + m = log p + m m = p + log n m y) L( eˆ Fˆ ) MDL( m,( τ, p ), K,( τ = log m = m, p m + mlog log L( ψˆ y) m )) n + m = log p + m m log n + = = p + (log L( ( πσ ψˆ ˆ ) y+ ) n ) 9

10 Opimizaion Using Geneic Algorihm Geneic Algorihm: Chromosome consiss of n genes, each aking he value of (no break) or p (order of AR process). Use naural selecion o find a near opimal soluion. Map he break poins wih a chromosome c via ( m,( τ, p ) K,( τ m, p m)) c = ( δ, K, δn), ( m m δn where, if nobreak poin a, δ = p, if break poin a ime = τ For example, c = (, -, -, -, -, 0, -, -, -, -, 0, -, -, -, 3, -, -, -, -,-) : 6 5 would correspond o a process as follows: andar order is p. AR(), =:5; AR(0), =6:0; AR(0), =:4; AR(3), =5:0

11 Implemenaion of Geneic Algorihm (con) Generaion 0: Sar wih L (00) randomly generaed chromosomes, c,...,c L wih associaed MDL values, MDL(c ),..., MDL(c L ). Generaion : A new child in he nex generaion is formed from he chromosomes c,..., c L of he previous generaion as follows: wih probabiliy π c, crossover occurs. wo paren chromosomes c i and c are seleced a random wih probabiliies proporional o he ranks of MDL(c i ). k h gene of child is δ k = δ i,k w.p. ½ and δ,k w.p. ½ wih probabiliy π c, muaion occurs. a paren chromosome c i is seleced k h gene of child is δ k = δ i,k w.p. π ; w.p. π ;and p w.p. π π.

12 Simulaion Examples-based on Ombao e al. (00) es cases. Piecewise saionary wih dyadic srucure: Consider a ime series following he model,.9 Y, if + ε, if < 53, Y, if =.69Y.8Y + ε, if 53 < 769,.3 Y + ε, if.8y, if , where {ε } ~ IID N(0,) Time 4

13 . Piecewise sa (con) GA resuls: 3 pieces breaks a τ =53; τ =769. Toal run ime 6.3 secs Fied model: φ φ σ - 5: : : True Model Fied Model Time Time 6

14 Simulaion Examples (con). Slowly varying AR() model: Y = ay. 8 Y + ε if 04 where =. 8[ 0.5cos( π /04)], and {ε } ~ IID N(0,). a a_ Time ime 7

15 . Slowly varying AR() (con) GA resuls: 3 pieces, breaks a τ =93, τ =65. Toal run ime 7.45 secs Fied model: φ φ σ - 9: : : True Model Fied Model Time Time 8

16 . Slowly varying AR() (con) In he graph below righ, we average he specogram over he GA fied models generaed from each of he 00 simulaed realizaions. True Model Average Model Frequency Time Time 9

17 Simulaion Examples (con) 3. Simulaed daa from Fearnhead (005): True model has 9 changepoins black=rue red=aparm green=map Time MAP es of m=9 while MAP of m and changepoin locaions gives m= 8 changeps. Plo is condiional on 9 changepoins. 0

18 4. Fearnhead example True Model Fied APARM Model Time Time

19 Examples Mine explosion seismic race in Scandinavia: (Shumway and Soffer 000, Soffer e al. 005) Two waves: P (primary) compression wave and S (shear) wave Time 5

20 Examples AR orders: Time 6

21 Example: EEG Time series Daa: Bivariae EEG ime series a channels T3 (lef emporal) and P3 (lef parieal). Female subec was diagnosed wih lef emporal lobe epilepsy. Daa colleced by Dr. Beh Malow and analyzed in Ombao e al (00). (n=3,768; sampling rae of 00H). Seizure sared a abou.85 seconds. GA GA bivariae univariae resuls: resuls: pieces 4 breakpoins wih AR orders, for T3; 7,, breakpoins 6 5,, 3, for 5, 9, P35, 4, T3 Channel P3 Channel EEG T3 channel EEG P3 channel Time in seconds Time in seconds 9

22 Remarks: Example: EEG Time series (con) he general conclusions of his analysis are similar o hose reached in Ombao e al. prior o seizure, power concenraed a lower frequencies and hen spread o high frequencies. power reurned o he lower frequencies a conclusion of seizure. T3 Channel P3 Channel Frequency (Herz) Frequency (Herz) Time in seconds Time in seconds 30

23 Example: EEG Time series (con) Remarks (con): T3 and P3 srongly coheren a 9- Hz prior o seizure. srong coherence a low frequencies us afer onse of seizure. srong coherence shifed o high frequencies during he seizure. T3/P3 Coherency Frequency (Herz) Time in seconds 3

24 Applicaion o Parameer-Driven SS Models Sae Space Model Seup: Observaion equaion: p(y α ) = exp{α y b(α ) + c(y )}. Sae equaion: {α } follows he piecewise AR() model given by α = γ k +φ k α - +σ k ε, if τ k- < τ k, where = τ 0 < τ < < τ m < n, and {ε } ~ IID N(0,). Parameers: m = number of break poins τ k = locaion of break poins γ k = level in k h epoch φ k = AR coefficiens k h epoch σ k = scale in k h epoch 35

25 Applicaion o Srucural Breaks (con) Esimaion: For (m, τ,..., τ m ) fixed, calculae he approximae likelihood evaluaed a he MLE, i.e., / Gn T * T * * T * La ( ψˆ;y n ) = exp{y nα { b( α ) c(yn)} ( α µ ) Gn ( α µ )/}, ( K + G ) n / where ˆ ˆ ψˆ = (ˆ γ, K, γˆ, φ, K, φ, σˆ, K, σˆ ) is he MLE. m m m Remark: The exac likelihood is given by he following formula L( ψ;yn ) = La ( ψ;yn ) Era ( ψ), where Er ( ψ) = exp{ R( α ; α*)} p ( α y ; ψ) dα. a n I urns ou ha log( Er a ( ψ)) is nearly linear and can be approximaed by a linear funcion via imporance sampling, e ψ ) ~ e( ψˆ ) + e& ( ψˆ a n )( ψ ψˆ ( AL AL AL ) n n 36

26 Coun Daa Example Model: Y α Pois(exp{β+ α }), α = φα - + ε, {ε }~IID N(0, σ ) y MDL ime Breaking Poin True model: Y α ~ Pois(exp{.7 + α }), α =.5α - + ε, {ε }~IID N(0,.3), < 50 Y α ~ Pois(exp{.7 + α }), α = -.5α - + ε, {ε }~IID N(0,.3), > 50. GA esimae 5, ime 67secs 37

27 SV Process Example Model: Y α N(0,exp{α }), α = γ + φ α - + ε, {ε }~IID N(0, σ ) y MDL ime Breaking Poin True model: Y α ~ N(0, exp{α }), α = α - + ε, {ε }~IID N(0,.05), 750 Y α ~ N(0, exp{α }), α = α - + ε, {ε }~IID N(0,.5), > 750. GA esimae 754, ime 053 secs 38

28 SV Process Example Model: Y α N(0,exp{α }), α = γ + φ α - + ε, {ε }~IID N(0, σ ) y MDL ime Breaking Poin True model: Y α ~ N(0, exp{α }), α = α - + ε, {ε }~IID N(0,.80), 50 Y α ~ N(0, exp{α }), α = α - + ε, {ε }~IID N(0,.0089), > 50. GA esimae 5, ime 69s 39

29 SV Process Example-(con) True model: Y α ~ N(0, exp{a }), α = α - + e, {ε }~IID N(0,.80), 50 Y α N(0, exp{α }), α = α - + ε, {ε }~IID N(0,.0089), > 50. Fied model based on no srucural break: Y α N(0, exp{α }), α = α - + ε, {ε }~IID N(0,.0935) y original series y simulaed series ime ime 40

30 SV Process Example-(con) Fied model based on no srucural break: Y α N(0, exp{α }), α = α - + ε, {ε }~IID N(0,.0935) y simulaed series MDL ime Breaking Poin 4

31 Summary Remarks. MDL appears o be a good crierion for deecing srucural breaks.. Opimizaion using a geneic algorihm is well suied o find a near opimal value of MDL. 3. This procedure exends easily o mulivariae problems. 4. While esimaing srucural breaks for nonlinear ime series models is more challenging, his paradigm of using MDL ogeher GA holds promise for break deecion in parameer-driven models and oher nonlinear models. 4