Structural Break Detection in Time Series

Transcription

1 Srucural Break Deecion in Tie Series Richard A. Davis Thoas Lee Gabriel Rodriguez-Ya Colorado Sae Universiy (hp:// This research suppored in par by an IBM faculy award.

2 Inroducion Background Seup for AR odels Model Selecion Using Miniu Descripion Lengh (MDL) General principles Applicaion o AR odels wih breaks Opiizaion using Geneic Algorihs Basics Ipleenaion for srucural break esiaion Siulaion exaples Piecewise auoregressions Slowly varying auoregressions Applicaions Mulivariae ie series EEG exaple Applicaion o nonlinear odels (paraeer-driven sae-space odels) Poisson odel Sochasic volailiy odel 2

3 Inroducion Srucural breaks: Kiagawa and Akaike (978) fiing locally saionary auoregressive odels using AIC copuaions faciliaed by he use of he Householder ransforaion Davis Huang and Yao (995) likelihood raio es for esing a change in he paraeers and/or order of an AR process. Kiagawa Takanai and Masuoo (200) signal exracion in seisology-esiae he arrival ie of a seisic signal. Obao Raz von Sachs and Malow (200) orhogonal coplex-valued ransfors ha are localized in ie and frequency- sooh localized coplex exponenial (SLEX) ransfor. applicaions o EEG ie series and speech daa. 3

4 Locally saionary: Inroducion (con) Dahlhaus ( ) locally saionary processes esiaion Adak (998) piecewise saionary applicaions o seisology and bioedical signal processing MDL and coding heory: Lee ( ) esiaion of disconinuous regression funcions Hansen and Yu (200) odel selecion 4

5 Inroducion (con) Tie Series: y... y n Piecewise AR odel: Y = γ φ Y L φp Y p σ ε if τ - < τ where τ 0 = < τ <... < τ - < τ = n and {ε } is IID(0). Goal: Esiae = nuber of segens τ = locaion of h break poin γ = level in h epoch p = order of AR process in h epoch ( φ = AR coefficiens in h K φ p ) epoch σ = scale in h epoch 5

6 Moivaion for using piecewise AR odels: Piecewise AR is a special case of a piecewise saionary process (see Adak 998) ~ Y = Y I ( / n) n [ τ τ ) = where { } =... is a sequence of saionary processes. I is Y argued in Obao e al. (200) ha if {Y n } is a locally saionary process (in he sense of Dahlhaus) hen here exiss a piecewise ~ saionary process } wih { Y n n wih n / n 0 as n ha approxiaes {Y n } (in average ean square). Roughly speaking: {Y n } is a locally saionary process if i has a ievarying specru ha is approxiaely A(/nω) 2 where A(uω) is a coninuous funcion in u. 6

7 Exaple--Monhly Deahs & Serious Inuries UK Daa: y = nuber of onhly deahs and serious inuries in UK Jan `75 Dec `84 ( = 20) Reark: Sea bel legislaion inroduced in Feb `83 ( = 99). Couns Year 7

8 Differenced Couns Exaple -- Monhly Deahs & Serious Inuries UK (con) Daa: x = nuber of onhly deahs and serious inuries in UK differenced a lag 2; Jan `75 Dec `84 ( = 3 20) Reark: Sea bel legislaion inroduced in Feb `83 ( = 99) Year Model: b= {N }~AR(3). Tradiional regression analysis: Y = a bf ( ) W 0 if 98 f() = if 98 < 20. X = Y Y 2 = bg ( ) N g() = 0 if 99 0 oherwise.. 8

9 Model Selecion Using Miniu Descripion Lengh Basics of MDL: Choose he odel which axiizes he copression of he daa or equivalenly selec he odel ha iniizes he code lengh of he daa (i.e. aoun of eory required o encode he daa). M = class of operaing odels for y = (y... y n ) L F (y) = code lengh of y relaive o F M Typically his er can be decoposed ino wo pieces (wo-par code) where L ( y) = L( Fˆ y) L(ˆ e Fˆ ) F L( Fˆ y) L(ˆ e Fˆ ) = code lengh of he fied odel for F = code lengh of he residuals based on he fied odel 9

10 0 Illusraion Using a Siple Regression Model (see T. Lee `0) Encoding he daa: (x y )... (x n y n ). Naïve case ) ( ) ( ) ( ) ( ) ( ) ( ") (" n n n n y L y L x L x L y y L x x L naive L = = L L K K 2. Linear odel; suppose y i = a 0 a x i i =... n. Then ) ( ) ( ) ( ) ( ) ( ) ( ") (" 0 0 a L a L x L x L a a L x x L p L n n = = = L K 3. Linear odel wih noise; suppose y i = a 0 a x i ε i i =... n where {ε i }~IID N(0σ 2 ). Then If A < L(y )... L(y n ) hen p= encoding schee doinaes he naïve schee K L ) ˆ ˆ ˆ ˆ (ˆ ) (ˆ ) ( ˆ ) ( ˆ ) ( ) ( ") (" σ ε ε σ = = a a L L a L a L x L x L p L n n A

11 Model Selecion Using Miniu Descripion Lengh (con) Applied o he segened AR odel: Y = γ φ Y L φp Y p σ ε if τ - < τ Firs er L( Fˆ y) : Le n = τ τ - and ψ = ( γ φ K φp σ ) denoe he lengh of he h segen and he paraeer vecor of he h AR process respecively. Then L( Fˆ y) = L() L( τ K τ = L() L( n K n ) L( p K p ) L( p K p ) L( ψˆ ) L( ψˆ y) L L( ψˆ y) L L( ψˆ y) y) Encoding: ineger I : log 2 I bis (if I unbounded) log 2 I U bis (if I bounded by I U ) MLE θˆ : ½ log 2 N bis (where N = nuber of observaions used o copue θˆ ; Rissanen (989))

12 So p 2 L(Fˆ y) = log2 log2 n log2 p log 2 = Second er L(ˆ e Fˆ ) : Using Shannon s classical resuls on inforaion heory Rissanen deonsraes ha he code lengh of ê can be approxiaed by he negaive of he log-likelihood of he fied odel i.e. by L(ˆ e Fˆ ) = n = 2 (log2 (2πσˆ ) ) 2 For fixed values of (τ p )... (τ p ) we define he MDL as MDL( ( τ p = log ) K( τ 2 p log 2 )) n = log 2 p p 2 log 2 2 n 2 n = = n 2 n log2 (2πσˆ ) 2 2 The sraegy is o find he bes segenaion ha iniizes MDL(τ p τ p ). To speed hings up we use Y-W esiaes of AR paraeers. 2

13 Opiizaion Using Geneic Algorihs Basics of GA: Class of opiizaion algorihs ha iic naural evoluion. Sar wih an iniial se of chroosoes or populaion of possible soluions o he opiizaion proble. Paren chroosoes are randoly seleced (proporional o he rank of heir obecive funcion values) and produce offspring using crossover or uaion operaions. Afer a sufficien nuber of offspring are produced o for a second generaion he process hen resars o produce a hird generaion. Based on Darwin s heory of naural selecion he process should produce fuure generaions ha give a saller (or larger) obecive funcion. 3

14 Applicaion o Srucural Breaks (con) Geneic Algorih: Chroosoe consiss of n genes each aking he value of (no break) or p (order of AR process). Use naural selecion o find a near opial soluion. Map he break poins wih a chroosoe c via ( ( τ p ) K ( τ p )) c = ( δ K δn) ( δn where if nobreak poin a δ = p if break poin a ie = τ For exaple c = ( ) : 6 5 would correspond o a process as follows: andar order is p. AR(2) =:5; AR(0) =6:0; AR(0) =:4; AR(3) =5:20 4

15 Ipleenaion of Geneic Algorih (con) Generaion 0: Sar wih L (200) randoly generaed chroosoes c... c L wih associaed MDL values MDL(c )... MDL(c L ). Generaion : A new child in he nex generaion is fored fro he chroosoes c... c L of he previous generaion as follows: wih probabiliy π c crossover occurs. wo paren chroosoes c i and c are seleced a rando wih probabiliies proporional o he ranks of MDL(c i ). k h gene of child is δ k = δ ik w.p. ½ and δ k w.p. ½ wih probabiliy π c uaion occurs. a paren chroosoe c i is seleced k h gene of child is δ k = δ ik w.p. π ; w.p. π 2 ;and p w.p. π π 2. 5

16 Ipleenaion of Geneic Algorih (con) Execuion of GA: Run GA unil convergence or unil a axiu nuber of generaions has been reached.. Various Sraegies: include he op en chroosoes fro las generaion in nex generaion. use uliple islands in which populaions run independenly and hen allow igraion afer a fixed nuber of generaions. This ipleenaion is aenable o parallel copuing. 6

17 Siulaion Exaples-based on Obao e al. (200) es cases. Piecewise saionary wih dyadic srucure: Consider a ie series following he odel.9y ε if < 53 Y =.69Y.8Y 2 ε if 53 < Y.8Y 2 ε if where {ε } ~ IID N(0) Tie 7

18 . Piecewise sa (con) Ipleenaion: Sar wih NI = 50 islands each wih populaion size L = 200. Afer every Mi = 5 generaions allow igraion. 4 Replace wors 2 in Island 34 2 wih bes 2 fro Island Sopping rule: Sop when he ax MDL does no change for 0 consecuive igraions or afer 00 igraions. 2 Span configuraion for odel selecion: Max AR order K = 0 p p π /2 p /2 /2 /2 /2 /2 /2 /2 /2 8

19 . Piecewise sa (con) GA resuls: 3 pieces breaks a τ =53; τ 2 =769. Toal run ie 6.3 secs Fied odel: φ φ 2 σ 2-52: : : True Model Fied Model Tie Tie 9

20 . Piecewise sa (con) Siulaion: 200 replicaes of ie series of lengh 024 were generaed. (SLEX resuls fro Obao e al.) # of segens ASE = n (/ 33) {log fˆ( / n ω ) log f ( / n ω )} Auo-SLEX % Change Poins ASE n 32 = = = 0 /2 /4 3/4 /4 2/4 3/4 2/8 4/8 5/8 6/8 7/ (4.56) (6.74) (5.20) 50.0 (6.25) GA % ean sd ASE ω = 2 π / (.3) 3.73 (.3)

21 . Piecewise sa (con) Siulaion (con): True odel: Y =.9Y.69Y.32Y ε.8y.8y 2 2 ε ε if if if < < AR orders seleced (percen): Order p p p

22 2. Piecewise saionary: Siulaion Exaples (con).9y Y =.9Y where {ε } ~ IID N(0). ε ε if < 98 if Tie 22

23 2. Piecewise saionary (con) GA resuls: 2 pieces wih break a τ =96. Toal run ie.96 secs Fied odel: φ φ 2 σ 2-52: : : True Model Fied Model Tie Tie 23

24 2. Piecewise saionary (con) Siulaion: 200 replicaes of ie series of lengh 024 were generaed. AR() odels wih break a 96/024 =.9. # of segens Auo-SLEX % ASE (.039) (.056) (.047) (.096) # of segn change poins % ean sd ASE (.03).060 (.044) (.00) 24

25 Siulaion Exaples (con) 3. Piecewise saionary wih shor segens: Y.9 Y =.25 Y ε ε if where {ε } ~ IID N(0). GA resuls: 2 pieces wih break a τ =47 if < Tie 25

26 3. Piecewise saionary (con) Siulaion resuls: Change occurred a ie τ = 5; 5/500=. # of segens change poins % ean sd

27 Siulaion Exaples (con) 4. Slowly varying AR(2) odel: Y = ay. 8 Y 2 ε if 024 where =. 8[ 0.5cos( π /024)] and {ε } ~ IID N(0). a a_ Tie ie 27

28 4. Slowly varying AR(2) (con) GA resuls: 3 pieces breaks a τ =293 τ 2 =65. Toal run ie secs Fied odel: φ φ 2 σ 2-292: : : True Model Fied Model Tie Tie 28

29 4. Slowly varying AR(2) (con) Siulaion: 200 replicaes of ie series of lengh 024 were generaed. # of sege ns 4 Auo-SLEX % ASE (.030) # of sege ns change poins % ean sd ASE (.025) (.04) (.029) (.06) (.033).269 (.040)

30 4. Slowly varying AR(2) (con) Siulaion: 200 replicaes of ie series of lengh 024 were generaed. # of segens Auo-SLEX % ASE # of segens change poins % ean sd ASE (.030) (.025) (.04) (.029) (.06) (.033).269 (.040)

31 Siulaion (con): 4. Slowly varying AR(2) (con) True odel: Y = ay. 8Y 2 ε if 024 AR orders seleced (percen): (2 segen realizaions) Order p p AR orders seleced (percen): (3 segen realizaions) Order p p p

32 4. Slowly varying AR(2) (con) In he graph below righ we average he specogra over he GA fied odels generaed fro each of he 200 siulaed realizaions. True Model Average Model Frequency Tie Tie 32

33 Siulaion Exaples (con) 5. Piecewise ARMA: Y =.9Y.9Y where {ε } ~ IID N(0). ε ε ε.7ε.7ε if if if < < Tie 33

34 5. Piecewise ARMA (con) GA resuls: 3 pieces breaks a τ =53 τ 2 =769. Toal run ie.53 secs Fied odel: AR orders 4 2 True Model Fied Model Tie Tie 34

35 Exaple: Monhly Deahs & Serious Inuries UK Daa: Y = nuber of onhly deahs and serious inuries in UK Jan `75 Dec `84 ( = 20) Reark: Sea bel legislaion inroduced in Feb `83 ( = 99). Couns Year 35

36 Exaple: Monhly Deahs & Serious Inuries UK Daa: Y = nuber of onhly deahs and serious inuries in UK Jan `75 Dec `84 ( = 20) Reark: Sea bel legislaion inroduced in Feb `83 ( = 99). Differenced Couns Year Resuls fro GA: 3 pieces; ie = 4.4secs Piece : (= 98) IID; Piece 2: (=99 08) IID; Piece 3: =09 20 AR() 36

37 Exaples Speech signal: GREASY G R EA S Y Tie 37

38 Speech signal: GREASY n = 5762 observaions = 5 break poins Run ie = 8.02 secs G R EA S Y Tie Tie 38

39 Exaples Large brown ba echolocaion: 400 daa poins aken a 7icrosecond inervals (oal duraion of.0028 seconds). Daa and ideas abou M- saionariy described here are fro Buddy Gray Wayne Woodward and heir group a SMU. hp://faculy.su.edu/hgray/research.h ba echolocaion Tie 39

40 Feaures of daa: ie varying frequency exaples of which are chirps and doppler signals found in radar sonar and counicaion heory. daa appears o be ade up of wo signals. each signal has a frequency ha is changing linearly in ie. i.e. ha is he cycle is lenghening in ie. an AR(20) odel is he bes fiing AR odel. Residuals are uncorrelaed bu no independen Tie 40

41 Exaples (ba daa con) M-Saionariy (Gray e al): Cov(Y()Y(τ)) = R(τ). This noion corresponds o a ie-deforaion (logarihic in his case) o ake he ransfored process saionary in he ordinary sense. The Euler process (Gray and Zhang `98) is an exaple of an M- saionary process Tie 4

42 Exaples (ba daa con) GA resuls: 6 pieces breaks a τ =6 τ 2 =98 τ 3 =205 τ 4 = 265 τ 5 = 353. Fied odel: AR orders ; Toal run ie 4.7 secs Tie 42

43 Exaples (ba daa specrogras) Euler(2) Gray e al Auo-PARM Tie 43

44 ie ie ie ie ie ie

45 ie ie log(ie) log(ie) log(ie)

46 Applicaion o Mulivariae Tie Series Mulivariae ie series (d-diensional): y... y n Piecewise AR odel: where τ 0 = < τ <... < τ - < τ = n and {Ζ } is IID(0 I d ). In his case MDL( ( τ p Y /2 = γ Φ Y L Φ p Y p Σ Z if τ - < τ ) K( τ p d d( d )) = log logn )/2 logn 2 = 2 = p d = 2 logp τ =τ ( ˆ ˆ T ˆ log( V ) ( Y Y ) V ( Y Yˆ )) 2 where Yˆ = E( Y Y KY ) and Vˆ = E( Y Yˆ and he AR paraeers ) are esiaed by he ulivariae Y-W equaions based on While s generalizaion of he Durbin-Levinson algorih. 46

47 Exaple: Bivariae Tie Series {Y } sae as he series in Exaple 2 (3 segens: AR() AR(3) AR(2)) {Y 2 } sae as he series in Exaple (2 segens: AR() AR()) GA resuls: TS : 3 pieces wih breaks a τ =53 and τ 2 =769. Toal run ie 6.3 secs TS 2: 2 pieces wih break a τ =96. Toal run ie.96 secs Bivariae: 4 pieces wih breaks a τ =97 τ 2 =59 τ 3 =769: AR() AR() AR(2) AR(2): Toal run ie 26 secs Tie Tie 47

48 Exaple: EEG Tie series Daa: Bivariae EEG ie series a channels T3 (lef eporal) and P3 (lef parieal). Feale subec was diagnosed wih lef eporal lobe epilepsy. Daa colleced by Dr. Beh Malow and analyzed in Obao e al (200). (n=32768; sapling rae of 00H). Seizure sared a abou.85 seconds. GA GA bivariae univariae resuls: resuls: pieces 4 breakpoins wih AR orders for T3; 7 2 breakpoins for 5 9 P35 4 T3 Channel P3 Channel EEG T3 channel EEG P3 channel Tie in seconds Tie in seconds 48

49 Rearks: Exaple: EEG Tie series (con) he general conclusions of his analysis are siilar o hose reached in Obao e al. prior o seizure power concenraed a lower frequencies and hen spread o high frequencies. power reurned o he lower frequencies a conclusion of seizure. T3 Channel P3 Channel Frequency (Herz) Frequency (Herz) Tie in seconds Tie in seconds 49

50 Exaple: EEG Tie series (con) Rearks (con): T3 and P3 srongly coheren a 9-2 Hz prior o seizure. srong coherence a low frequencies us afer onse of seizure. srong coherence shifed o high frequencies during he seizure. T3/P3 Coherency Frequency (Herz) Tie in seconds 50

51 Applicaion o Paraeer-Driven SS Models Sae Space Model Seup: Observaion equaion: p(y α ) = exp{α y b(α ) c(y )}. Sae equaion: {α } follows he piecewise AR() odel given by α = γ k φ k α - σ k ε if τ k- < τ k where = τ 0 < τ < < τ < n and {ε } ~ IID N(0). Paraeers: = nuber of break poins τ k = locaion of break poins γ k = level in k h epoch φ k = AR coefficiens k h epoch σ k = scale in k h epoch 5

52 Applicaion o Srucural Breaks (con) Esiaion: For ( τ... τ ) fixed calculae he approxiae likelihood evaluaed a he MLE i.e. /2 Gn T * T * * T * La ( ψˆ;y n ) = exp{y nα { b( α ) c(yn)} ( α µ ) Gn ( α µ )/2} ( K G ) n /2 where ˆ ˆ 2 2 ψˆ = (ˆ γ K γˆ φ K φ σˆ K σˆ ) is he MLE. Goal: Opiize an obecive funcion over ( τ... τ ). use iniu descripion lengh (MDL) as an obecive funcion use geneic algorih for opiizaion 52

53 Applicaion o Srucural Breaks (con) Miniu Descripion Lengh (MDL): Choose he odel which axiizes he copression of he daa or equivalenly selec he odel ha iniizes he code lengh of he daa (i.e. aoun of eory required o sore he daa). Code Lengh( daa ) = CL( fied odel ) CL( daa fied odel ) MDL ( τ K τ ) ~ CL( paraeers ) CL( residuals ) = log( ) log( n ).5 log( τ ˆ τ ) log( La ( ψ ;y τ : )) : τ = = CL (" Paraeers ") CL (" residuals ") Generalizaion: AR(p) segens can have unknown order. MDL ( ( τ p ) K ( τ p )) = log( ) log( n ) 0.5 = ( p 2)log( τ τ ) = log( L ( ψ ˆ ;y a τ : τ: )) 53

54 Coun Daa Exaple Model: Y α Pois(exp{β α }) α = φα - ε {ε }~IID N(0 σ 2 ) y MDL blue =AL red = IS True odel: ie Breaking Poin Y α ~ Pois(exp{.7 α }) α =.5α - ε {ε }~IID N(0.3) < 250 Y α ~ Pois(exp{.7 α }) α = -.5α - ε {ε }~IID N(0.3) > 250. GA esiae 25 ie 267secs 54

55 SV Process Exaple Model: Y α N(0exp{α }) α = γ φ α - ε {ε }~IID N(0 σ 2 ) y MDL ie Breaking Poin True odel: Y α ~ N(0 exp{α }) α = α - ε {ε }~IID N(0.05) 750 Y α ~ N(0 exp{α }) α = α - ε {ε }~IID N(0.25) > 750. GA esiae 754 ie 053 secs 55

56 SV Process Exaple Model: Y α N(0exp{α }) α = γ φ α - ε {ε }~IID N(0 σ 2 ) y MDL ie Breaking Poin True odel: Y α ~ N(0 exp{α }) α = α - ε {ε }~IID N(0.80) 250 Y α ~ N(0 exp{α }) α = α - ε {ε }~IID N(0.0089) > 250. GA esiae 25 ie 269s 56

57 SV Process Exaple-(con) True odel: Y α ~ N(0 exp{a }) α = α - e {ε }~IID N(0.80) 250 Y α N(0 exp{α }) α = α - ε {ε }~IID N(0.0089) > 250. Fied odel based on no srucural break: Y α N(0 exp{α }) α = α - ε {ε }~IID N(0.0935) y original series y siulaed series ie ie 57

58 SV Process Exaple-(con) Fied odel based on no srucural break: Y α N(0 exp{α }) α = α - ε {ε }~IID N(0.0935) y siulaed series MDL ie Breaking Poin 58

59 Suary Rearks. MDL appears o be a good crierion for deecing srucural breaks. 2. Opiizaion using a geneic algorih is well suied o find a near opial value of MDL. 3. This procedure exends easily o ulivariae probles. 4. While esiaing srucural breaks for nonlinear ie series odels is ore challenging his paradig of using MDL ogeher GA holds proise for break deecion in paraeer-driven odels and oher nonlinear odels. 59