Detection of structural breaks in multivariate time series

Transcription

1 Detection of structural breaks in multivariate time series Holger Dette, Ruhr-Universität Bochum Philip Preuß, Ruhr-Universität Bochum Ruprecht Puchstein, Ruhr-Universität Bochum January 14, 2014

2 Outline 1 Motivation 2 Piecewise stationary processes 3 Testing for stationarity 4 Detecting structural breaks 5 Finite sample properties

3 Switch in variance 1 / 46

4 Switch in variance Figure: 1024 realizations of an independent series, where the variance switches from 1 to 4. 1 / 46

5 AR(1) model with structural break 2 / 46

6 AR(1) model with structural break Figure: realizations of an AR(1) process where the parameter switches from 0.9 to 2 / 46

7 A less obvious example 3 / 46

8 A less obvious example Figure: realizations of an AR(1) process where the parameter switches from 0.7 to 3 / 46

9 The goal of this talk (1) Do there exist structural breaks (in the autocovariance structure)? (2) If yes, how many of them are present? (3) Where are the break points located? (4) In the multivariate setting: in which components do breaks occur? 4 / 46

10 The goal of this talk (1) Do there exist structural breaks (in the autocovariance structure)? (2) If yes, how many of them are present? (3) Where are the break points located? (4) In the multivariate setting: in which components do breaks occur? This talk gives an answer to these problems The number of change points is unknown. Structural breaks can occur in different components of the time series (these will be identified). 4 / 46

11 Some references in the one dimensional case Detecting change points in the second-order structure [like the variance or the parameters in an AR(p) model] has found considerable interest. Inclan and Tiao (1994, JASA), Chen and Gupta (1997, JASA) and Lee and Park (2001, SJoS) are testing for one change point in the variance. Lee et al. (2003, SJoS) trying to detect one change in specific parameters [like the AR(1) parameter]. Binary segmentation is used to detect multiple change points working but far from being optimal. Davis et al. (2006, JASA) propose an algorithm to fit piecewise AR(p) processes. 5 / 46

12 Multivariate case Much less literature in the multivariate case. Aue et al. (2009, AoS) propose a test for detecting one structural break in the variance matrix. Cho and Fryzlewicz (2013) develope an algorithm to segment parts with different second order characteristics. 6 / 46

13 Stationarity Consider a centered R d -valued time series {X t } t Z Question: Does Γ(t, h) := E[X t+h X H t ] = [γ ij (t, h)] i,j=1,...,d change over time for some h Z? 7 / 46

14 Stationarity Consider a centered R d -valued time series {X t } t Z Question: Does change over time for some h Z? Move to the frequency domain: does change over time? Γ(t, h) := E[X t+h X H t ] = [γ ij (t, h)] i,j=1,...,d f(t, λ) = 1 2π Γ(t, h) exp( iλh) h Z 7 / 46

15 Piecewise stationarity Model: triangular scheme X t,t, t = 1,..., T, with a piecewise stationary representation on K + 1 intervals, i.e. where X t,t = l=0 Ψ(1) l=0 Ψ(2). l=0 Ψ(K+1) l Z t l if 0 = b 0T < t b 1T l Z t l if b 1T < t b 2T l Z t l if b K T < t b K+1 T = T, White noise {Z t} t Z N(0, I d ) (normality assumption is not necessary) Ψ (1) l,..., Ψ (K+1) l R d d (l = 0, 1,...) 0 = b 0 < b 1 < < b K < b K+1 = 1. 8 / 46

16 The local spectral density matrix Compact notation where X t,t = Ψ l (t/t )Z t l, l=0 Ψ l : [0, 1] R d d t = 1,..., T are piecewise constant functions (on the (same) K + 1 intervals (b 0, b 1 ], (b 1, b 2 ],... (b K, b K+1 ]). Null hypothesis (no change in the second order characteristics) H 0 : K = 0 9 / 46

17 The local spectral density matrix Compact notation where X t,t = Ψ l (t/t )Z t l, l=0 Ψ l : [0, 1] R d d t = 1,..., T are piecewise constant functions (on the (same) K + 1 intervals (b 0, b 1 ], (b 1, b 2 ],... (b K, b K+1 ]). Null hypothesis (no change in the second order characteristics) H 0 : K = 0 local spectral density matrix at time u [0, 1] f(u, λ) = 1 Ψ l (u)ψ H 2π m(u) exp( iλ(l m)). l,m=0 9 / 46

18 Motivating the procedure If K 1, the spectral density f(u, λ) has points of discontinuity in u direction. For v [0, 1] consider D(v, ω) = lim ɛ 0 ( ωπ 0 f(v ɛ, λ)dλ ωπ 0 f(v + ɛ, λ)dλ) D(v) := sup D(v, ω) := ω [0,1] sup sup ω [0,1] a,b=1,...,d If there is no structural break at time v, then D(v) = 0. D(v, ω) a,b 10 / 46

19 Motivating the procedure If K 1, the spectral density f(u, λ) has points of discontinuity in u direction. For v [0, 1] consider D(v, ω) = lim ɛ 0 ( ωπ 0 f(v ɛ, λ)dλ ωπ 0 f(v + ɛ, λ)dλ) D(v) := sup D(v, ω) := ω [0,1] sup sup ω [0,1] a,b=1,...,d If there is no structural break at time v, then D(v) = 0. For testing the hypothesis H 0 : K = 0, we consider D := sup D(v) = sup D(v, ω). v [0,1] v,ω [0,1] D(v, ω) a,b 10 / 46

20 Next steps In order to test the hypothesis H 0 : K = 0 we 1) construct an empirical version ˆD T (v, ω) of D(v, ω). 2) use as an estimator of D. ˆD T := sup ˆD T (v, ω) v,ω [0,1] 3) use the AR( ) bootstrap to estimate the (1 α)-quantile q 1 α of ˆD T under H 0 and reject H 0 if ˆD T > ˆq 1 α. 11 / 46

21 Estimating D = sup v,ω [0,1] D(v, ω) We estimate f(u, λ) by the local periodogram I N (u, λ) := 1 2πN N 1 r,s=0 X ut N/2+1+r,T X H ut N/2+1+s,T exp( iλ(r s)). Note: I N (u, λ) is not consistent for f(u, λ) (but for D(v, ω) we only need averages)! 12 / 46

22 Estimating D = sup v,ω [0,1] D(v, ω) Estimate D(v, ω) = lim ɛ 0 ( by Riemann sums, that is ωπ 0 f(v ɛ, λ)dλ ωπ 0 f(v + ɛ, λ)dλ). ˆD T (v, ω) := 2π N ωn/2 k=1 ( I N ( v N/(2T ), λk ) IN ( v + N/(2T ), λk ) ), where λ k := 2πk N Note: The estimate is consistent! 13 / 46

23 The statistic v N γ sup ω [0,1] [ ˆD T (v, ω)] a,b (N = T /8, γ = 0.4) / 46

24 The statistic v N γ sup ω [0,1] [ ˆD T (v, ω)] a,b (N = T /8, γ = 0.4) / 46

25 Asymptotic properties of ˆD T = sup v,ω [0,1] ˆD T (v, ω) Theorem Assume N and N/T c 0: a) If K = 0 then N γ ˆDT = o P (1) for any 0 < γ < 1 2. If c > 0, then N1/2 ˆDT converges weakly to a centered Gaussian process. b) If K 1 then there exist constants C R + such that lim P( sup [ ˆD T (b r, ω)] a,b > C) = 1 T ω [0,1] for all (r, a, b) {1,..., K} {1,..., d} 2 where sup [D(b r, ω)] a,b > 0. ω [0,1] 16 / 46

26 Bootstrapping ˆD T Note: If c = 0 then N 1/2 ˆDT does not converge weakly! N 1/2 ˆDT (v 1, ω) and N 1/2 ˆDT (v 1, ω) are asymptotically uncorrelated! We estimate the quantiles of ˆD T under the null hypothesis H 0 : K = 0 using an AR( )-Bootstrap [Berg et al. (2011, JSPI), Kreiss et al. (2011, AoS)]. This procedure exploits the fact that every stationary process can be accurately approximated by AR(p)-models, if p is sufficiently large. 17 / 46

27 Main idea of AR(p) bootstrap (1) We generate bootstrap replicates of a process {X t } t Z with spectral density g (λ) = 1 0 f(u, λ)du 18 / 46

28 Main idea of AR(p) bootstrap (1) We generate bootstrap replicates of a process {X t } t Z with spectral density g (λ) = 1 0 f(u, λ)du (2) Note that g (λ) is the best approximation of f(u, λ) with respect to the L 2 -distance π 1 π 0 tr [( f(u, λ) g(λ) )( f(u, λ) g(λ) ) H] dudλ 18 / 46

29 Main idea of AR(p) bootstrap (1) We generate bootstrap replicates of a process {X t } t Z with spectral density g (λ) = 1 0 f(u, λ)du (2) Note that g (λ) is the best approximation of f(u, λ) with respect to the L 2 -distance π 1 π 0 tr [( f(u, λ) g(λ) )( f(u, λ) g(λ) ) H] dudλ (3) The process {X t } t Z is approximated by an AR(p) process, where p is increasing with the sample size. 18 / 46

30 Main idea of AR(p) bootstrap (1) We generate bootstrap replicates of a process {X t } t Z with spectral density g (λ) = 1 0 f(u, λ)du (2) Note that g (λ) is the best approximation of f(u, λ) with respect to the L 2 -distance π 1 π 0 tr [( f(u, λ) g(λ) )( f(u, λ) g(λ) ) H] dudλ (3) The process {X t } t Z is approximated by an AR(p) process, where p is increasing with the sample size. (4) We can prove consistency! 18 / 46

31 Algorithm for generating replicates ˆD T of ˆD T 1) Choose p N and compute an estimator (â 1,p,..., â p,p) for ( p p ) (a 1,p,..., a p,p) := argmin tr E[(X t,t b j,p X t j,t )(X t,t b j,p X t j,t ) H ], b 1,p,...,b p,p j=1 j=1 2) Set X t,t = X t,t for t = 1,..., p. 3) Calculate p X t,t = â j,p X t j,t + ˆΣ 1/2 p Z j,t for t > p j=1 where Z j,t are independent N(0, I d ) distributed and ˆΣ p = 1 T (ẑ i z T )(ẑ i z T ) H z T := 1 T ẑ j T p T p j=p+1 j=p+1 p ẑ j : = X j,t â i,p X j i,t for j = p + 1,..., T. i=1 4) Define ˆD T (v, ω) as ˆD T (v, ω) but with the X t,t replaced by its bootstrap replicates X t,t. 5) Define ˆD T := sup (v,ω) [0,1] 2 ˆD T (v, ω). 19 / 46

32 Algorithm for testing H 0 1) Calculate the test statistic ˆD T using the observed data {X 1,T,..., X T,T }. 2) Choose p N and determine estimates (â 1,p,..., â p,p, ˆΣ p ) which fit an AR(p) model to the observed data. 3) Generate B N replicates ˆD T,i i = 1,..., B. 4) Estimate the (1 α) quantile of ˆD T by the corresponding empirical quantile ( ˆD T ) T, (1 α)b of the sample { ˆD T,1,..., ˆD T,T }. 5) Reject H 0 if ˆD T > ( ˆD T ) T, (1 α)b. 20 / 46

33 Summary This yields an asymptotic level α test for the null hypothesis of no structural breaks. We have to choose: - The window-length N - The AR-dimension p. 21 / 46

34 Summary This yields an asymptotic level α test for the null hypothesis of no structural breaks. We have to choose: - The window-length N - The AR-dimension p. Take p as the minimizer of the the AIC criterion. How to choose N? Just a few minutes (this problem is partially open)! Finite sample properties? At the end of this talk! 21 / 46

35 Follow-Up questions If H 0 has been rejected, the following questions occur: 1) How many break points? Construction of an estimator ˆK for K. 2) Where are the break points located? Construction of an estimator (ˆb 1,..., ˆb ˆK ). How can the data {X t,t } t=1,...,t be subdivided into stationary segments? 3) In which components do the breaks occur? 22 / 46

36 Main idea Under the null hypothesis of no structural breaks: N γ holds (for 0 < γ < 1/2), while sup ˆD T (v, ω) = o P (1) v,ω [0,1] N γ for all b i i = 1,..., K if K 1. sup ˆD T (b i, ω) ω [0,1] 23 / 46

37 Main idea Under the null hypothesis of no structural breaks: N γ holds (for 0 < γ < 1/2), while sup ˆD T (v, ω) = o P (1) v,ω [0,1] N γ for all b i i = 1,..., K if K 1. sup ˆD T (b i, ω) ω [0,1] Identify a (possible) structural break in the component (a, b) at time point v if N γ sup [ ˆD T (v, ω)] a,b > ɛ T,a,b (v) (1) ω [0,1] for some thresholding sequence ɛ T,a,b (v) = o(n γ ) satisfying lim inf ɛ T,a,b(v) C > 0. T 23 / 46

38 Choice of ɛ T,a,b Identify a (possible) structural break in the component (a, b) at time point v if (too many!!) N γ sup [ ˆD T (v, ω)] a,b > ɛ T,a,b (v). ω [0,1] We choose where ( d(d + 1)T ɛ T,a,b (v) = 2M T,a,b (v, 1) log 2N ), M T,a,b (v, ω) = 1 N ωn [I 2N (v, λ k,2n )] aa [I 2N (v, λ k,2n )] bb k=1 24 / 46

39 Example / 46

40 v N γ sup [ ˆD T (v, ω)] a,b with γ = 0.4, N = T /8 ω [0,1] / 46

41 Localization of structural breaks (1) ˆB P = {ˆb 1,..., ˆb ˆK } : potential break points in { N T, N+1 T N T,..., ˆB D = : detected break points T } (2) Add the element b ˆB P to the set ˆB D for which sup ( sup N γ [ ˆD T ( b, ω)] a,b ) (a,b) {1,...,d} 2 ω [0,1] is maximal and replace the set ˆB P by ˆB P \[ b N T, b + N T ]. (3) Repeat step (2) until B P = ˆK = ˆB D (ˆb 1,..., ˆb ˆK ) are the different elements of ˆB D 27 / 46

42 Asymptotic properties (informal) Theorem Assume lim inf T ɛ T,a,b(v) C > 0, ɛ T,a,b (v) = o(n γ ), then (a) The probability that the decision rule indicates a structural break although there is no one vanishes asymptotically. (b) The probability that the procedure detects all structural breaks (and the corresponding components) converges to / 46

43 Regularization In the detection algorithm we require the choice of γ and N. We choose γ = 0.4 How to choose N (this work is not finished)? - If there are only a few structural breaks, N should be rather large. - If there exist many break points with small distances, N should be small. 29 / 46

44 Regularization In the detection algorithm we require the choice of γ and N. We choose γ = 0.4 How to choose N (this work is not finished)? - If there are only a few structural breaks, N should be rather large. - If there exist many break points with small distances, N should be small. Good choice for the test: N = T /2 29 / 46

45 Regularization One more example: 30 / 46

46 Regularization One more example: Good choice for the test: N = T /4 30 / 46

47 Regularization One more example: Good choice for the test: N = T /4 Good choice for the detection procedure: N = T /8 30 / 46

48 Data-driven choice of N 1) Use a set of even integers satisfying T N1 < N 2 <... < N n T 5/6 2) Estimate for each N i the number ˆK T (N i ) of break points 3) Define i := sup{i {2,..., n} ˆK T (N i 1 ) ˆK T (N i )} (here sup = ) and { N Ni if i = n if i = 4) Choose N = 2N for the test for structural breaks N = N for the estimation of the number and location of breakpoints. N n 31 / 46

49 Data-driven choice of N 1) Use a set of even integers satisfying T N1 < N 2 <... < N n T 5/6 2) Estimate for each N i the number ˆK T (N i ) of break points 3) Define i := sup{i {2,..., n} ˆK T (N i 1 ) ˆK T (N i )} (here sup = ) and { N Ni if i = n if i = 4) Choose N = 2N for the test for structural breaks N = N for the estimation of the number and location of breakpoints. 5) In the applications (FFT) N n N i = 2 log 2 ( T ) 1+i 31 / 46

50 Size of the test (univariate) X t = Z t + 0.5Z t 1 (2) X t = 0.5X t 1 + Z t (3) Model (2) Model (3) T 5 % 10 % 5 % 10 % Table: Empirical rejection frequencies of the bootstrap test in model (2) and (3) with different T. 32 / 46

51 Size of the test (multivariate) ( ) θ1 θ X t = Z t + 2 Z θ 2 θ t 1 (4) 1 ( ) φ1 φ X t = 2 X φ 2 φ t 1 + Z t. (5) 1 H 0 : Model (4) H 0 : Model (5) θ = (0.3, 0.1) θ = ( 0.5, 0.1) φ = (0.3, 0.1) φ = ( 0.5, 0.1) T 5% 10% 5% 10% 5% 10% 5% 10% Table: Empirical rejection frequencies of the bootstrap test in model (4) and (5) with different choices of θ = (θ 1, θ 2), φ = (φ 1, φ 2) and T. 33 / 46

52 Power of the test X t,t = X t,t = X t,t = ( K θl [ bl T +1, b l+1 T ](t) 0.2 θ l ( φl [ bl T +1, b l+1 T ](t) l=0 K l=0 K 1 [ bl T +1, b l+1 T ](t) l=0 ) Z t 1 + Z t (6) 0.2 φ l ) X t 1,T + Z t (7) ( ) σl 0.2 Z 0.2 σ t (8) l (6) (7) (8) T = 128 T = 256 T = 512 b parameter new Aue new Aue new Aue ( 1 4, 2 3, 3 ) (1, 1.5, 1, 1.5) ( 1 ) (1, 1.5) ( 1 4, 2 3, 3 ) (0.5, 0.5, 0.5, 0.5) ( 1 ) (0.5, 0.5) ( 1 4, 2 3, 3 ) (1, 2, 1, 0.5) ( 1 ) (1, 2) / 46

53 Performance of the detection rule - white noise model X t,t = 4 j=1 1 ( j 1 4 T, j 4 T ](t)θ jz t, Θ 1 := ( ) Θ 2 := ( ) Θ 3 := ( ) Θ 4 := ( ) {Z t} t Z is a two dimensional Gaussian white noise process Three changes at T /4; T /2 and 3T /4 35 / 46

54 Performance of the detection rule - white noise model / 46

55 Performance of the detection rule - white noise model Figure: Empirical distribution of ˆb = (ˆb 1,..., ˆb ˆK ) based on 100 simulation runs for sample sizes T {512, 1024, 2048}. Left: new procedure. Right: Davis et al (2006) T= T= T= / 46

56 Performance of the detection rule - white noise model Figure: Empirical distribution of ˆb = (ˆb 1,..., ˆb ˆK ) based on 100 simulation runs for sample sizes T {512, 1024, 2048}. Left: new procedure. Right: Davis et al (2006) T= 512 T= T= T= T= T= / 46

57 Performance of the detection rule - AR(2) model X t,t = ( ) ( ) X t 1,T + X t 2,T if t 1 T 2 ( ) ( ) X t 1,T + X t 2,T if 1 < t 3 2 T 4 ( ) ( ) X t 1,T + X 0 0 t 2,T if 3 < t 1 4 T +Z t {Z t} t Z is a two dimensional centered Gaussian process with covariance ( 1 ) Two changes at T /2 and 3T /4 38 / 46

58 Performance of the detection rule - AR(2) model / 46

59 Performance of the detection rule - AR(2) model Figure: Empirical distribution of ˆb = (ˆb 1,..., ˆb ˆK ) based on 100 simulation runs for sample sizes T {512, 1024, 2048}. Left: new procedure. Right: Davis et al (2006) T= T= T= / 46

60 Performance of the detection rule - AR(2) model Figure: Empirical distribution of ˆb = (ˆb 1,..., ˆb ˆK ) based on 100 simulation runs for sample sizes T {512, 1024, 2048}. Left: new procedure. Right: Davis et al (2006) T= T= T= T= T= T= / 46

61 Two locally stationary MA(1) models / 46

62 Two locally stationary MA(1) models Figure: Empirical distribution of ˆb = (ˆb 1,..., ˆb ˆK ) based on 100 simulation runs for sample sizes T {512, 1024, 2048}. Left: new procedure. Right: Davis et al (2006) T= T= T= / 46

63 Two locally stationary MA(1) models Figure: Empirical distribution of ˆb = (ˆb 1,..., ˆb ˆK ) based on 100 simulation runs for sample sizes T {512, 1024, 2048}. Left: new procedure. Right: Davis et al (2006) T= T= T= T= T= T= / 46

64 Multivariate data example We consider sector ETFs Symbol Name Sector 1 XLB Materials Select Sector SPDR Commodities 2 XLP Consumer Staples Select Sector SPDR Consumer Staples 3 XLU Utilities Select Sector SPDR Utilities 4 XLF Financial Select Sector SPDR Financials 5 XLE Energy Select Sector SPDR Energy and their log returns X t,j := log(y t,j /Y t 1,j ) with Y t,j denoting the adjusted closing price at time t of sector ETF j. 43 / 46

65 Multivariate data example 0e+00 2e 04 4e e+00 2e 04 4e e+00 2e 04 4e e+00 4e 05 8e e+00 4e 05 8e e+00 4e 05 8e e+00 2e 04 4e e+00 2e 04 4e 04 6e e+00 2e 04 4e e+00 2e 04 4e Figure: Plot of functions v N γ e+00 2e 04 4e e+00 2e 04 4e sup ˆD T (v, ω) a,b for a, b = 1,..., / 46 ω [0,1]

66 Multivariate data example Figure: Plot of the log-returns for the sector ETFs. 45 / 46

67 Summary Summary A test for the presence of structural breaks in multivariate time series Estimation of the number of structural breaks occur Estimation of the locations and components, where structural breaks occur Outperforms common binary segmentation approach if more than one break point is present The methodology is based on the assumption of a piecewise stationary multivariate process but it can be generalized to processes with different locally stationary behavior on different segements 1 ( ωπ D(v, ω) := lim ɛ 0 ɛ 0 v+ɛ v f(u, λ)dudλ ωπ v 0 v ɛ f(u, λ)dudλ ) 46 / 46

68 Summary Summary A test for the presence of structural breaks in multivariate time series Estimation of the number of structural breaks occur Estimation of the locations and components, where structural breaks occur Outperforms common binary segmentation approach if more than one break point is present The methodology is based on the assumption of a piecewise stationary multivariate process but it can be generalized to processes with different locally stationary behavior on different segements Future work: 1 ( ωπ D(v, ω) := lim ɛ 0 ɛ 0 v+ɛ v f(u, λ)dudλ ωπ v Investigate the choice of regularization parameters (in particular N) How to adjust the procedure if the dimension d is large (growing)? Multiscale inference? 0 v ɛ f(u, λ)dudλ ) 46 / 46