Structural Break Detection in Time Series Models

Transcription

1 Srucural Break Deecion in Time Series Models Richard A. Davis Thomas Lee Gabriel Rodriguez-Yam Colorado Sae Universiy (hp:// This research suppored in par by an IBM faculy award. SCMA-Auburn /05

2 Illusraive Example How many segmens do you see? SCMA-Auburn / τ = 5 τ = 5 τ 3 = 5 ime

3 Illusraive Example Auo-PARM=Auo-Piecewise AuoRegressive Modeling 4 pieces,.58 seconds SCMA-Auburn / τ = 5 τ = 57 τ 3 = 59 ime 3

4 Inroducion Examples AR GARCH Sochasic volailiy Sae space models Model selecion using Minimum Descripion Lengh (MDL) General principles Applicaion o AR models wih breaks Opimizaion using a Geneic Algorihm Basics Implemenaion for srucural break esimaion Simulaion resuls Applicaions Simulaion resuls for GARCH and SSM SCMA-Auburn /05 4

5 . Piecewise AR model: Y Examples = γ + φ Y + L+ φp Y p + σ ε, if τ - < τ where τ 0 = < τ <... < τ m- < τ m = n +, and {ε } is IID(0,). Goal: Esimae m = number of segmens τ = locaion of h break poin γ = level in h epoch p = order of AR process in h epoch ( φ = AR coefficiens in h, K, φ p ) epoch σ = scale in h epoch, SCMA-Auburn /05 6

6 Piecewise AR models (con) Srucural breaks: SCMA-Auburn /05 Kiagawa and Akaike (978) fiing locally saionary auoregressive models using AIC compuaions faciliaed by he use of he Householder ransformaion Davis, Huang, and Yao (995) likelihood raio es for esing a change in he parameers and/or order of an AR process. Kiagawa, Takanami, and Masumoo (00) signal exracion in seismology-esimae he arrival ime of a seismic signal. Ombao, Raz, von Sachs, and Malow (00) orhogonal complex-valued ransforms ha are localized in ime and frequency- smooh localized complex exponenial (SLEX) ransform. applicaions o EEG ime series and speech daa. 7

7 . Segmened GARCH model: Examples (con) Y σ = σ ε = ω, + α Y + L+ α p Y p +β σ + L+β q σ q, if τ - < τ, where τ 0 = < τ <... < τ m- < τ m = n +, and {ε } is IID(0,). 3. Segmened sochasic volailiy model: Y = σ ε logσ, = γ + φ logσ + L+ φ p logσ p + ν η, if τ - < τ. 4. Segmened sae-space model (SVM a special case): p( y α = γ α,..., α + φ α, y,..., y + L+ φ ) = p p( y α p α ) is + σ η, specified if τ - < τ. SCMA-Auburn /05 9

8 Model Selecion Using Minimum Descripion Lengh Basics of MDL: Choose he model which maximizes he compression of he daa or, equivalenly, selec he model ha minimizes he code lengh of he daa (i.e., amoun of memory required o encode he daa). M = class of operaing models for y = (y,..., y n ) L F (y) = code lengh of y relaive o F M Typically, his erm can be decomposed ino wo pieces (wo-par code), where L ( y) = L( Fˆ y) L(ˆ e Fˆ ), F + L( Fˆ y) L(ˆ e Fˆ ) = code lengh of he fied model for F = code lengh of he residuals based on he fied model SCMA-Auburn /05 0

9 Model Selecion Using Minimum Descripion Lengh (con) Applied o he segmened AR model: Y Firs erm L( Fˆ y) : = γ + φ Y + L+ φp Y p + σ ε, if τ - < τ, L( Fˆ y) = L(m) + L( τ, K, τ ) + L( p, K, p ) + L( ψˆ y) + L+ L( ψˆ = log m + mlog m n + m = log p + m m = p + log n m y) Encoding: ineger I : log I bis (if I unbounded) log I U bis (if I bounded by I U ) MLE θˆ : ½ log N bis (where N = number of observaions used o compue θˆ ; Rissanen (989)) SCMA-Auburn /05

10 Second erm L(ˆ e Fˆ ) : Using Shannon s classical resuls on informaion heory, Rissanen demonsraes ha he code lengh of can be approximaed by he negaive of he log-likelihood of he fied model, i.e., by L( eˆ Fˆ ) m = log L( ψˆ y) For fixed values of m, (τ,p ),..., (τ m,p m ), we define he MDL as MDL( m,( τ, p ), K,( τ = log m, p m m + mlog )) n + m = log p + m m log n = = p + log L( ψˆ y) The sraegy is o find he bes segmenaion ha minimizes MDL(m,τ,p,, τ m,p m ). To speed hings up for AR processes, we use Y-W esimaes of AR parameers and we replace log L( ψˆ y) wih log (π σˆ ) + n SCMA-Auburn /05

11 Opimizaion Using Geneic Algorihms Basics of GA: Class of opimizaion algorihms ha mimic naural evoluion. Sar wih an iniial se of chromosomes, or populaion, of possible soluions o he opimizaion problem. Paren chromosomes are randomly seleced (proporional o he rank of heir obecive funcion values), and produce offspring using crossover or muaion operaions. Afer a sufficien number of offspring are produced o form a second generaion, he process hen resars o produce a hird generaion. SCMA-Auburn /05 Based on Darwin s heory of naural selecion, he process should produce fuure generaions ha give a smaller (or larger) obecive funcion. 3

12 Applicaion o Srucural Breaks (con) Geneic Algorihm: Chromosome consiss of n genes, each aking he value of (no break) or p (order of AR process). Use naural selecion o find a near opimal soluion. Map he break poins wih a chromosome c via ( m,( τ, p ) K,( τ m, p m)) c = ( δ, K, δn), ( m m δn where, if nobreak poin a, δ = p, if break poin a ime = τ For example, c = (, -, -, -, -, 0, -, -, -, -, 0, -, -, -, 3, -, -, -, -,-) : 6 5 would correspond o a process as follows: andar order is p. AR(), =:5; AR(0), =6:0; AR(0), =:4; AR(3), =5:0 SCMA-Auburn /05 4

13 Implemenaion of Geneic Algorihm (con) Generaion 0: Sar wih L (00) randomly generaed chromosomes, c,...,c L wih associaed MDL values, MDL(c ),..., MDL(c L ). Generaion : A new child in he nex generaion is formed from he chromosomes c,..., c L of he previous generaion as follows: wih probabiliy π c, crossover occurs. wo paren chromosomes c i and c are seleced a random wih probabiliies proporional o he ranks of MDL(c i ). k h gene of child is δ k = δ i,k w.p. ½ and δ,k w.p. ½ wih probabiliy π c, muaion occurs. a paren chromosome c i is seleced k h gene of child is δ k = δ i,k w.p. π ; w.p. π ;and p w.p. π π. SCMA-Auburn /05 5

14 Implemenaion of Geneic Algorihm (con) Execuion of GA: Run GA unil convergence or unil a maximum number of generaions has been reached.. Various Sraegies: include he op en chromosomes from las generaion in nex generaion. use muliple islands, in which populaions run independenly, and hen allow migraion afer a fixed number of generaions. This implemenaion is amenable o parallel compuing. SCMA-Auburn /05 6

15 Simulaion Examples-based on Ombao e al. (00) es cases. Piecewise saionary wih dyadic srucure: Consider a ime series following he model,.9 Y, if + ε, if < 53, Y, if =.69Y.8Y + ε, if 53 < 769,.3 Y + ε, if.8y, if , where {ε } ~ IID N(0,) SCMA-Auburn /05 Time 7

16 . Piecewise sa (con) GA resuls: 3 pieces breaks a τ =53; τ =769. Toal run ime 6.3 secs Fied model: φ φ σ - 5: : : True Model Fied Model Time Time SCMA-Auburn /05 9

17 Simulaion Examples (con). Slowly varying AR() model: Y = ay. 8 Y + ε if 04 where =. 8[ 0.5cos( π /04)], and {ε } ~ IID N(0,). a a_ SCMA-Auburn /05 Time ime 0

18 . Slowly varying AR() (con) GA resuls: 3 pieces, breaks a τ =93, τ =65. Toal run ime 7.45 secs Fied model: φ φ σ - 9: : : True Model Fied Model SCMA-Auburn /05 Time Time

19 . Slowly varying AR() (con) In he graph below righ, we average he specogram over he GA fied models generaed from each of he 00 simulaed realizaions. True Model Average Model Frequency Time Time SCMA-Auburn /05 5

20 Theory Consisency. Suppose he number of change poins m is known and le λ =τ /n,..., λ m =τ m /n be he relaive (rue) changepoins. Then λˆ λ a.s. where λ ˆ = ˆ /n and τ = Auo-PARM esimae of τ. τ ˆ SCMA-Auburn /05 8

21 Example: Monhly Deahs & Serious Inuries, UK Daa: Y = number of monhly deahs and serious inuries in UK, Jan `75 Dec `84, ( =,, 0) Remark: Sea bel legislaion inroduced in Feb `83 ( = 99). Couns Year SCMA-Auburn /05 9

22 Example: Monhly Deahs & Serious Inuries, UK Daa: Y = number of monhly deahs and serious inuries in UK, Jan `75 Dec `84, ( =,, 0) Remark: Sea bel legislaion inroduced in Feb `83 ( = 99). Differenced Couns Year Resuls from GA: 3 pieces; ime = 4.4secs Piece : (=,, 98) IID; Piece : (=99, 08) IID; Piece 3: =09,,0 AR() SCMA-Auburn /05 30

23 Examples Speech signal: GREASY G R EA S Y Time SCMA-Auburn /05 3

24 Speech signal: GREASY n = 576 observaions m = 5 break poins Run ime = 8.0 secs G R EA S Y Time SCMA-Auburn / Time 3

25 Applicaion o GARCH (con) Garch(,) model: Y σ = σ ε, = ω + α Y { ε }~ IID(0,) +β σ, if τ - < τ σ.4+. Y =.4 +. Y # of CPs σ +.6 σ,, GA % if if < 50, 50 < 000. AG % CP esimae = 506 Time AG = Andreou and Ghysels (00) SCMA-Auburn /05 33

26 Applicaion o GARCH (con) More simulaion resuls for Garch(,) : Y = σ ε, { ε }~ IID(0,) σ.05+.4y =.00+.3Y +.3σ +.σ,, if < τ, if τ < 000. τ Mean SE Med Freq 50 GA Berkes GA Berkes GA Berkes Berkes = Berkes, Gombay, Horvah, and Kokoszka (004). SCMA-Auburn /05 35

27 Applicaion o Parameer-Driven SS Models Sae Space Model Seup: Observaion equaion: p(y α ) = exp{α y b(α ) + c(y )}. Sae equaion: {α } follows he piecewise AR() model given by α = γ k +φ k α - +σ k ε, if τ k- < τ k, where = τ 0 < τ < < τ m < n, and {ε } ~ IID N(0,). Parameers: m = number of break poins τ k = locaion of break poins γ k = level in k h epoch φ k = AR coefficiens k h epoch σ k = scale in k h epoch SCMA-Auburn /05 36

28 Applicaion o Srucural Breaks (con) Esimaion: For (m, τ,..., τ m ) fixed, calculae he approximae likelihood evaluaed a he MLE, i.e., / Gn T * T * * T * La ( ψˆ;y n ) = exp{y nα { b( α ) c(yn)} ( α µ ) Gn ( α µ )/}, ( K + G ) n / where ˆ ˆ ψˆ = (ˆ γ, K, γˆ, φ, K, φ, σˆ, K, σˆ ) is he MLE. m m m Goal: Opimize an obecive funcion over (m, τ,..., τ m ). use minimum descripion lengh (MDL) as an obecive funcion use geneic algorihm for opimizaion SCMA-Auburn /05 37

29 Coun Daa Example Model: Y α Pois(exp{β+ α }), α = φα - + ε, {ε }~IID N(0, σ ) y MDL True model: ime Breaking Poin Y α ~ Pois(exp{.7 + α }), α =.5α - + ε, {ε }~IID N(0,.3), < 50 Y α ~ Pois(exp{.7 + α }), α = -.5α - + ε, {ε }~IID N(0,.3), > 50. SCMA-Auburn /05 GA esimae 5, ime 67secs 38

30 SV Process Example Model: Y α N(0,exp{α }), α = γ + φ α - + ε, {ε }~IID N(0, σ ) y MDL ime Breaking Poin True model: Y α ~ N(0, exp{α }), α = α - + ε, {ε }~IID N(0,.05), 750 Y α ~ N(0, exp{α }), α = α - + ε, {ε }~IID N(0,.5), > 750. GA esimae 754, ime 053 secs SCMA-Auburn /05 39

31 SV Process Example Model: Y α N(0,exp{α }), α = γ + φ α - + ε, {ε }~IID N(0, σ ) y MDL ime Breaking Poin True model: Y α ~ N(0, exp{α }), α = α - + ε, {ε }~IID N(0,.80), 50 Y α ~ N(0, exp{α }), α = α - + ε, {ε }~IID N(0,.0089), > 50. GA esimae 5, ime 69s SCMA-Auburn /05 40

32 SV Process Example-(con) True model: Y α ~ N(0, exp{a }), α = α - + e, {ε }~IID N(0,.80), 50 Y α N(0, exp{α }), α = α - + ε, {ε }~IID N(0,.0089), > 50. Fied model based on no srucural break: Y α N(0, exp{α }), α = α - + ε, {ε }~IID N(0,.0935) y original series y simulaed series ime ime SCMA-Auburn /05 4

33 SV Process Example-(con) Fied model based on no srucural break: Y α N(0, exp{α }), α = α - + ε, {ε }~IID N(0,.0935) y simulaed series MDL ime Breaking Poin SCMA-Auburn /05 4

34 Summary Remarks. MDL appears o be a good crierion for deecing srucural breaks.. Opimizaion using a geneic algorihm is well suied o find a near opimal value of MDL. 3. This procedure exends easily o mulivariae problems. 4. While esimaing srucural breaks for nonlinear ime series models is more challenging, his paradigm of using MDL ogeher GA holds promise for break deecion in parameer-driven models and oher nonlinear models. SCMA-Auburn /05 43