Structural Break Detection in Time Series

Transcription

1 Srucural Break Deecion in Time Series Richard A. Davis Thomas ee Gabriel Rodriguez-Yam Colorado Sae Universiy (hp:// This research suppored in par by an IBM faculy award. Monreal 5/05

2 Inroducion Seup Examples AR GARCH Sochasic volailiy Sae space models Model Selecion Using Minimum Descripion engh (MD General principles Applicaion o AR models wih breaks Opimizaion using Geneic Algorihms Basics Implemenaion for srucural break esimaion Simulaion resuls Applicaions Simulaion resuls for GARCH and SSM Monreal 5/05

3 Inroducion The Premise (in a general framework: Base model: P θ family or probabiliy models for a saionary ime series. Observaions: y... y n Segmened model: here exis an ineger m 0 and locaions τ 0 = < τ <... < τ m- < τ m = n such ha Y = X if τ - < τ where {X } is a saionary ime series wih probabiliy disr and θ θ. Obecive: esimae P θ m = number of segmens τ = locaion of h break poin θ = parameer vecor in h epoch Monreal 5/05 3

4 . Piecewise AR model: Y Examples = γ φ Y φp Y p σ ε if τ - < τ where τ 0 = < τ <... < τ m- < τ m = n and {ε } is IID(0. Goal: Esimae m = number of segmens τ = locaion of h break poin γ = level in h epoch p = order of AR process in h epoch ( φ = AR coefficiens in h K φ p epoch σ = scale in h epoch Monreal 5/05 4

5 Piecewise AR models (con Srucural breaks: Monreal 5/05 Kiagawa and Akaike (978 fiing locally saionary auoregressive models using AIC compuaions faciliaed by he use of he Householder ransformaion Davis Huang and Yao (995 likelihood raio es for esing a change in he parameers and/or order of an AR process. Kiagawa Takanami and Masumoo (00 signal exracion in seismology-esimae he arrival ime of a seismic signal. Ombao Raz von Sachs and Malow (00 orhogonal complex-valued ransforms ha are localized in ime and frequency- smooh localized complex exponenial (SEX ransform. applicaions o EEG ime series and speech daa. 5

6 Moivaion for using piecewise AR models: Piecewise AR is a special case of a piecewise saionary process (see Adak 998 m ~ Y = Y I ( / n n [ τ τ = where { } =... m is a sequence of saionary processes. I is Y argued in Ombao e al. (00 ha if {Y n } is a locally saionary process (in he sense of Dahlhaus hen here exiss a piecewise ~ saionary process } wih { Y n m n wih m n / n 0 as n ha approximaes {Y n } (in average mean square. Roughly speaking: {Y n } is a locally saionary process if i has a imevarying specrum ha is approximaely A(/nω where A(uω is a coninuous funcion in u. Monreal 5/05 6

7 . Segmened GARCH model: Examples (con Y σ = σ ε = ω α Y α p Y p β σ β q σ q if τ - < τ where τ 0 = < τ <... < τ m- < τ m = n and {ε } is IID(0. 3. Segmened sochasic volailiy model: Y = σ ε logσ = γ φ logσ φ p logσ p ν η if τ - < τ. 4. Segmened sae-space model (SVM a special case: p( y α = γ α... α φ α y... y φ = p p( y α p α is σ η specified if τ - < τ. Monreal 5/05 7

8 Model Selecion Using Minimum Descripion engh Basics of MD: Choose he model which maximizes he compression of he daa or equivalenly selec he model ha minimizes he code lengh of he daa (i.e. amoun of memory required o encode he daa. M = class of operaing models for y = (y... y n F (y = code lengh of y relaive o F M Typically his erm can be decomposed ino wo pieces (wo-par code where ( y = ( Fˆ y (ˆ e Fˆ F ( Fˆ y (ˆ e Fˆ = code lengh of he fied model for F = code lengh of he residuals based on he fied model Monreal 5/05 8

9 9 Monreal 5/05 Illusraion Using a Simple Regression Model (see T. ee `0 Encoding he daa: (x y... (x n y n. Naïve case ( ( ( ( ( ( " (" n n n n y y x x y y x x naive = = K K. inear model; suppose y i = a 0 a x i i =... n. Then ( ( ( ( ( ( " (" 0 0 a a x x a a x x p n n = = = K 3. inear model wih noise; suppose y i = a 0 a x i ε i i =... n where {ε i }~IID N(0σ. Then If A < (y... (y n hen p= encoding scheme dominaes he naïve scheme K ˆ ˆ ˆ ˆ (ˆ (ˆ ( ˆ ( ˆ ( ( " (" 0 0 σ ε ε σ = = a a a a x x p n n A

10 Model Selecion Using Minimum Descripion engh (con Applied o he segmened AR model: Y = γ φ Y φp Y p σ ε if τ - < τ Firs erm ( Fˆ y : e n = τ τ - and ψ = ( γ φ K φp σ denoe he lengh of he h segmen and he parameer vecor of he h AR process respecively. Then ( Fˆ y = (m ( τ K τ = (m ( n K n m m ( p K p ( p K p m m ( ψˆ ( ψˆ y ( ψˆ m y ( ψˆ m y y Encoding: ineger I : log I bis (if I unbounded log I U bis (if I bounded by I U ME θˆ : ½ log N bis (where N = number of observaions used o compue θˆ ; Rissanen (989 Monreal 5/05 0

11 So m m p (Fˆ y = log m mlog n log p log = Second erm (ˆ e Fˆ : Using Shannon s classical resuls on informaion heory Rissanen demonsraes ha he code lengh of ê can be approximaed by he negaive of he log-likelihood of he fied model i.e. by (ˆ e Fˆ m = n = (log (πσˆ For fixed values of m (τ p... (τ m p m we define he MD as MD( m( τ p = log K( τ m p m m mlog n m = log p p log n m m n = = n n log (πσˆ The sraegy is o find he bes segmenaion ha minimizes MD(mτ p τ m p m. To speed hings up we use Y-W esimaes of AR parameers. Monreal 5/05

12 Opimizaion Using Geneic Algorihms Basics of GA: Class of opimizaion algorihms ha mimic naural evoluion. Sar wih an iniial se of chromosomes or populaion of possible soluions o he opimizaion problem. Paren chromosomes are randomly seleced (proporional o he rank of heir obecive funcion values and produce offspring using crossover or muaion operaions. Afer a sufficien number of offspring are produced o form a second generaion he process hen resars o produce a hird generaion. Monreal 5/05 Based on Darwin s heory of naural selecion he process should produce fuure generaions ha give a smaller (or larger obecive funcion.

13 Applicaion o Srucural Breaks (con Geneic Algorihm: Chromosome consiss of n genes each aking he value of (no break or p (order of AR process. Use naural selecion o find a near opimal soluion. Map he break poins wih a chromosome c via ( m ( τ p K ( τ m p m c = ( δ K δn ( m m δn where if nobreak poin a δ = p if break poin a ime = τ For example c = ( : 6 5 would correspond o a process as follows: andar order is p. AR( =:5; AR(0 =6:0; AR(0 =:4; AR(3 =5:0 Monreal 5/05 3

14 Implemenaion of Geneic Algorihm (con Generaion 0: Sar wih (00 randomly generaed chromosomes c... c wih associaed MD values MD(c... MD(c. Generaion : A new child in he nex generaion is formed from he chromosomes c... c of he previous generaion as follows: wih probabiliy π c crossover occurs. wo paren chromosomes c i and c are seleced a random wih probabiliies proporional o he ranks of MD(c i. k h gene of child is δ k = δ ik w.p. ½ and δ k w.p. ½ wih probabiliy π c muaion occurs. a paren chromosome c i is seleced k h gene of child is δ k = δ ik w.p. π ; w.p. π ;and p w.p. π π. Monreal 5/05 4

15 Implemenaion of Geneic Algorihm (con Execuion of GA: Run GA unil convergence or unil a maximum number of generaions has been reached.. Various Sraegies: include he op en chromosomes from las generaion in nex generaion. use muliple islands in which populaions run independenly and hen allow migraion afer a fixed number of generaions. This implemenaion is amenable o parallel compuing. Monreal 5/05 5

16 Simulaion Examples-based on Ombao e al. (00 es cases. Piecewise saionary wih dyadic srucure: Consider a ime series following he model.9 Y if ε if < 53 Y if =.69Y.8Y ε if 53 < Y ε if.8y if where {ε } ~ IID N( Monreal 5/05 Time 6

17 . Piecewise sa (con Implemenaion: Sar wih NI = 50 islands each wih populaion size = 00. Afer every Mi = 5 generaions allow migraion. 4 Replace wors in Island 34 wih bes from Island Sopping rule: Sop when he max MD does no change for 0 consecuive migraions or afer 00 migraions. Span configuraion for model selecion: Max AR order K = 0 p m p π / p / / / / / / / / Monreal 5/05 7

18 . Piecewise sa (con GA resuls: 3 pieces breaks a τ =53; τ =769. Toal run ime 6.3 secs Fied model: φ φ σ - 5: : : True Model Fied Model Time Time Monreal 5/05 8

19 Simulaion Examples (con 3. Slowly varying AR( model: Y = ay. 8 Y ε if 04 where =. 8[ 0.5cos( π /04] and {ε } ~ IID N(0. a a_ Monreal 5/05 Time ime 9

20 3. Slowly varying AR( (con GA resuls: 3 pieces breaks a τ =93 τ =65. Toal run ime 7.45 secs Fied model: φ φ σ - 9: : : True Model Fied Model Monreal 5/ Time Time 0

21 3. Slowly varying AR( (con In he graph below righ we average he specogram over he GA fied models generaed from each of he 00 simulaed realizaions. True Model Average Model Frequency Time Time Monreal 5/05 4

22 Example: Monhly Deahs & Serious Inuries UK Daa: Y = number of monhly deahs and serious inuries in UK Jan `75 Dec `84 ( = 0 Remark: Sea bel legislaion inroduced in Feb `83 ( = 99. Couns Year Monreal 5/05 5

23 Example: Monhly Deahs & Serious Inuries UK Daa: Y = number of monhly deahs and serious inuries in UK Jan `75 Dec `84 ( = 0 Remark: Sea bel legislaion inroduced in Feb `83 ( = 99. Differenced Couns Year Resuls from GA: 3 pieces; ime = 4.4secs Piece : (= 98 IID; Piece : (=99 08 IID; Piece 3: =09 0 AR( Monreal 5/05 6

24 Examples Speech signal: GREASY G R EA S Y Time Monreal 5/05 7

25 Speech signal: GREASY n = 576 observaions m = 5 break poins Run ime = 8.0 secs G R EA S Y Time Monreal 5/ Time 8

26 Applicaion o GARCH (con Garch( model: Y σ = σ ε = ω α Y { ε }~ IID(0 β σ if τ - < τ σ.4. Y =.4. Y # of CPs 0.5 σ.6 σ GA % if if < < 000. AG % CP esimae = 506 Time AG = Andreou and Ghysels ( Monreal 5/05 9

27 Applicaion o GARCH (con Garch( model: Y σ = σ ε = ω α Y { ε }~ IID(0 β σ if τ - < τ σ.4. Y =.4. Y # of CPs 0.5 σ.8 σ GA % if if < < 000. AG % CP esimae = 50 Time AG = Andreou and Ghysels ( Monreal 5/05 30

28 Applicaion o GARCH (con More simulaion resuls for Garch( : Y = σ ε { ε }~ IID(0 σ.05.4y =.00.3Y.3σ.σ if < τ if τ < 000. τ Mean SE Med Freq 50 GA Berkes GA Berkes GA Berkes Berkes = Berkes Gombay Horvah and Kokoszka (004. Monreal 5/05 3

29 Applicaion o Parameer-Driven SS Models Sae Space Model Seup: Observaion equaion: p(y α = exp{α y b(α c(y }. Sae equaion: {α } follows he piecewise AR( model given by α = γ k φ k α - σ k ε if τ k- < τ k where = τ 0 < τ < < τ m < n and {ε } ~ IID N(0. Parameers: m = number of break poins τ k = locaion of break poins γ k = level in k h epoch φ k = AR coefficiens k h epoch σ k = scale in k h epoch Monreal 5/05 3

30 Applicaion o Srucural Breaks (con Esimaion: For (m τ... τ m fixed calculae he approximae likelihood evaluaed a he ME i.e. / Gn T * T * * T * a ( ψˆ;y n = exp{y nα { b( α c(yn} ( α µ Gn ( α µ /} ( K G n / where ˆ ˆ ψˆ = (ˆ γ K γˆ φ K φ σˆ K σˆ is he ME. m m m Goal: Opimize an obecive funcion over (m τ... τ m. use minimum descripion lengh (MD as an obecive funcion use geneic algorihm for opimizaion Monreal 5/05 33

31 Coun Daa Example Model: Y α Pois(exp{β α } α = φα - ε {ε }~IID N(0 σ y MD True model: ime Breaking Poin Y α ~ Pois(exp{.7 α } α =.5α - ε {ε }~IID N(0.3 < 50 Y α ~ Pois(exp{.7 α } α = -.5α - ε {ε }~IID N(0.3 > 50. Monreal 5/05 GA esimae 5 ime 67secs 34

32 SV Process Example Model: Y α N(0exp{α } α = γ φ α - ε {ε }~IID N(0 σ y MD ime Breaking Poin True model: Y α ~ N(0 exp{α } α = α - ε {ε }~IID N( Y α ~ N(0 exp{α } α = α - ε {ε }~IID N(0.5 > 750. GA esimae 754 ime 053 secs Monreal 5/05 35

33 SV Process Example Model: Y α N(0exp{α } α = γ φ α - ε {ε }~IID N(0 σ y MD ime Breaking Poin True model: Y α ~ N(0 exp{α } α = α - ε {ε }~IID N( Y α ~ N(0 exp{α } α = α - ε {ε }~IID N( > 50. GA esimae 5 ime 69s Monreal 5/05 36

34 SV Process Example-(con True model: Y α ~ N(0 exp{a } α = α - e {ε }~IID N( Y α N(0 exp{α } α = α - ε {ε }~IID N( > 50. Fied model based on no srucural break: Y α N(0 exp{α } α = α - ε {ε }~IID N( y original series y simulaed series ime ime Monreal 5/05 37

35 SV Process Example-(con Fied model based on no srucural break: Y α N(0 exp{α } α = α - ε {ε }~IID N( y simulaed series MD ime Breaking Poin Monreal 5/05 38

36 Summary Remarks. MD appears o be a good crierion for deecing srucural breaks.. Opimizaion using a geneic algorihm is well suied o find a near opimal value of MD. 3. This procedure exends easily o mulivariae problems. 4. While esimaing srucural breaks for nonlinear ime series models is more challenging his paradigm of using MD ogeher GA holds promise for break deecion in parameer-driven models and oher nonlinear models. Monreal 5/05 39