Testing for Anomalous Periods in Time Series Data. Graham Elliott

Transcription

1 Testing for Anomalous Periods in Time Series Data Graham Elliott 1

2 Introduction The Motivating Problem There are reasons to expect that for a time series model that an anomalous period might occur where the model is different to the usual model. e.g. (a) Market interference (willful or accidental) for a period of time, for example an attempt to manipulate a financial market. (b) Bubbles. There might be a period where the model has very different dynamics. (c) Collusion. Typical collusion models have periods of collusion or non collusion (for this paper, a single collusion period). Identifying if such an occurrence appears in a model is useful either to directly test the above hypotheses and also to be sure that it is reasonable to average over the data for testing and estimation purposes for any model at hand. Testing for Anomalous Periods in Time Series Data 2

3 Introduction What we do Derive some tests for situations where the parameters of a model shift at some unknown time to a new set of values, for an unknown length of time. 1. Test against a stable model 2. Test against a permanent breaking model (currently in process, still doing numerical and theoretical work). Testing for Anomalous Periods in Time Series Data 3

4 Introduction The literature The list of published work on testing stability is beyond enormous. Ad hoc tests for breaks (null stability). Chow (1960), Quandt (1958,60), Andrews (1993), Bai (1997), Bai and Perron (1998), Perron and coauthors (many), Han, Hall and Boldea (2012). Optimal Tests for breaks (null stability). Nyblom (1989), Andrews and Ploberger (1994), Sowell (1996), Hansen (2000), Forchini (2002), Elliott and Mueller (2006) Tests between different types of breaking processes. Cogley and Sargent (2005). Many papers on other aspects of instability, not related directly to this paper. Testing for Anomalous Periods in Time Series Data 4

5 Outline 1. The limit problem for general breaking processes 2. Theoretical derivation of the test against a stable model 3. Performance of this test. 4. Issues with Interpreting Rejections 5. Theoretical derivation of the test against a breaking model Testing for Anomalous Periods in Time Series Data 5

6 General Limit Result General Breaking Parameters model Consider a model with time varying parameters, that is for all t T Γ T,t = Γ 0 + T 1 2B( t T ; θ) For a given θ Θ, B( ;θ) is a function on the unit interval that describes the shape of the parameter instability through time. This shape is parameterized by the finite dimensional parameter θ e.g. Single break in the mean of a time series. y t = μ + 1 t τ d + ε t ε t iiii(0,1) so Γ T,t = μ + T 1/2 T 1/2 d1 t τ. Testing for Anomalous Periods in Time Series Data 6

7 General Limit Result General Breaking Parameters Model Measured RV's : X T = x T,1,, x T,T R qq with sample size T. Density X T given by T t=1 f T,t (Γ T,t ) (this form of the likelihood arises naturally when f T,t (Γ T,t ) is the density of x T,t conditional on F T,t 1, the σ-field generated by {x T,t } t 1 s=1 ). The model with density T t=1 f T,t (Γ 0 ) is the "stable" model. Define l T,t = llf T,t (Γ), and s T,t = l T,t / Γ and h T,t = s T,t / Γ. Testing for Anomalous Periods in Time Series Data 7

8 General Limit Result General Breaking Parameters Model Testing for Anomalous Periods in Time Series Data 8

9 General Limit Result General Breaking Parameters Model Assume the breaking process B( t ; θ) satisfies the following condition T This rules out random walk breaks but allows for a great many of the breaking processes considered in the literature. Testing for Anomalous Periods in Time Series Data 9

10 General Limit Result General Breaking Parameters Model Then we have for the test of H 0 : Γ T,t = Γ 0 vs H 1 : Γ T,t = Γ 0 + T 1 2B( t T ; θ) then T llll T = T 1/2 B( t T ; θ) s T,t(Γ 0 ) t=1 T T 1 B t T ; θ h T,t Γ 0 B t T ; θ + o p(1) t=1 1 B(s, θ) H 1 2dd(s) B s, θ HB s, θ dd 0 (Li and Mueller 2009) Testing for Anomalous Periods in Time Series Data 10

11 Limiting Problem General Limit Result Consider the k 1 vector continuous time process G( ) on the unit interval G s = W s + H 1/2 s B r, θ dd 0 By Girsanov's Theorem, the log Radon-Nikodym derivative of the measure of G with parameter θ Θ relative to the measure ν of the standard Wiener process W, evaluated at W, is given by llf θ W 1 = B(s, θ) H 1 2dd(s) B s, θ HB s, θ dd 2 0 This is the same limit as for the sample problem Testing for Anomalous Periods in Time Series Data 11

12 Aside: Limits of Experiments General Limit Result Define an experiment as Ɛ = Ω, F, P h : h H so it is the usual probability triple extended to be a family of measures indexed by a (local) parameter h. We can define for any h 0 H the set of LR tests f h (X) f h0 (X) h H For a sample with r.v. X n the experiments Ɛ n = Ω n, F n, P n,h : h H then this experiment is said to converge to the limit experiment Ɛ if for every finite subset I H, every h 0 H f n,h (X n ) f n,h0 (X n ) h I d f h(x) f h0 (X) h I (Le Cam, best reference is Van der Vaart 1998). Testing for Anomalous Periods in Time Series Data 12

13 Aside: Limits of Experiments General Limit Result Further, let T be a statistic defined on Ɛ and T n be a statistic defined on Ɛ n where Ɛ n d Ɛ then for every Tn there exists a T such that T n d T. (Van der Vaart calls this the asymptotic representation theorem) This suggests that if we find a limit experiment that is relevant for a set of problems (it is the limit of interesting experiments) then we can find an asymptotically optimal test by first finding the optimal limit test T then finding a sample analog for the problem with data that converges to this optimal limit test T. The gain in this approach is twofold (a) Can find optimal tests for wide classes of problems (b) It is easier than doing everything in the small sample problem because the limit experiment will typically take a simpler form than the likelihood in the small sample problem. Testing for Anomalous Periods in Time Series Data 13

14 Limiting Problem General Limit Result (Elliott and Mueller 2013). This confirms that the limit experiment in the Le Cam sense is the one described with G(s) for all of the problems that satisfy Conditions 1 and 2. Further, the Asymptotic Representation Theorem (e.g. in van der Vaart (1991)) tells us that any statistic for the sample problem has corresponding statistic in the limit model. An implication is that optimal tests in the limit model cannot be beaten (wrt the notion of optimality) by statistics that do not converge to these optimal limit tests. This allows us to work directly with the limit experiment in constructing statistics Testing for Anomalous Periods in Time Series Data 14

15 Anomalies as a Break The Test We will consider situations where the parameters, stable over most of the data, briefly take on a different value for a relatively small time period. So the anomaly model for G(s) is G s s 0 = W s + ββ + δ 1 ρ 1 < r < ρ 2 dd. We will construct a test that is (a) Invariant to translations G s G s + β s, β R (b) Weighted Average Power over remaining parameters in G(s). (c) Scale will be consistently estimable. Testing for Anomalous Periods in Time Series Data 15

16 Derivation The Test For a given set of parameters, the optimal (invariant) test for the limit problem is llll θ = δ [(1 ρ 1 < s < ρ δ2 ρ 2 ρ 1 ]dg(s) 2 [1 ρ 1 < s < ρ 2 ρ 2 ρ 1 ] 2 dd 0 1 Define ρ = (ρ 2 ρ 1 ) then this solves to llll θ = δ G ρ 2 G ρ 1 ρ G(1) δ2 ρ (1 ρ ) 2 Testing for Anomalous Periods in Time Series Data 16

17 The Weighted Average Power Test Family The Test We have three parameters that determine the direction of power. (1) Starting point of the anomaly ρ 1 We can place weights on these from zero to (1-ρ ) equal to w(ρ 1 ) (2) Length of the anomaly ρ We will fix this to a specific value (3) Size of the departure δ We will weight these as normally distributed with mean zero and variance σ δ 2. This variance will be fixed at a point. Testing for Anomalous Periods in Time Series Data 17

18 Limit Problem Statistic The Test Integrating over δ and w(ρ 1 ) results in the statistic w(ρ 1 ) eee G ρ 2 G ρ 1 ρ G(1) 2 2 ρ 1 ρ + σ δ 2 dρ 1 We will set the weights so that they are equal across all values resulting in the statistic 1 1 ρ eee G ρ 2 G ρ 1 ρ G(1) 2 2 ρ 1 ρ + σ δ 2 dρ 1 This is the maximal power given the choices for the weighted average. Testing for Anomalous Periods in Time Series Data 18

19 Multivariate Limiting Problem The Test For the case where G(s) is a vector (many parameters) llll θ = δ H 1/2 G ρ 2 G ρ 1 ρ G(1) δ HH ρ (1 ρ ) 2 where H is the variance of the score (required to scale the score to obtain convergence to G(s)). Now integrate with respect to δ~n(0, σ δ 2 H 1 ) and we obtain the large sample statistic 1 1 ρ eee G ρ 2 G ρ 1 ρ G(1) G ρ 2 G ρ 1 ρ G(1) dρ ρ 1 ρ + σ δ so we replace the squared term by the inner product. Notice that we do not need to know H to do this, although we will estimate it to scale the scores. Testing for Anomalous Periods in Time Series Data 19

20 Small Sample Tests The Test To implement the test for any application, we need to have the sample analogs of G(s). For the mean of a series, i.e. y t = μ + ε t as the null model where the residuals are such that partial sums suitably scaled satisfy a functional central limit theorem then we have in place of G(s) the sequence G s = T 1/2 ω 1 y t where ω 2 is an estimate of the scaled spectral density of the shocks at frequency zero. If there are other regressors with fixed coefficients in the regression, use y projected on these regressors. Testing for Anomalous Periods in Time Series Data 20 [TT] t=1

21 Small Sample Tests The Test For coefficient on a variable in a linear regression then construct G(s) analog as y t = βx t + γ z t + ε t G s = T 1/2 ω 1 [TT] x t ε t t=1 where ε t are residuals from the stable regression and the scaling parameter is now the spectral density at frequency zero of x t ε t. Notice that since the limit result is the same for each case, the critical values are also the same, they do not depend on dimensions of the additional regressors etc. (assumptions) More generally we have as above the scaled score. Testing for Anomalous Periods in Time Series Data 21

22 Small Sample Tests The Test For coefficient in a logit regression then construct G(s) analog as P[y t = 1 x t = Λ βx t = eee βx t 1+eee βx t G s = T 1/2 ω 1 [TT] x t (y t Λ β x t ) t=1 where β are estimates from the stable regression and the scaling parameter is now the spectral density at frequency zero of x t (y t Λ β x t ). More generally we could construct any moment condition, either from the FOC of the maximum likelihood problem as above or directly as a GMM problem. Testing for Anomalous Periods in Time Series Data 22

23 Small Sample Statistic The Test Given the sequence G ( t T ) t=1 T the statistic to be computed is 1 T [ρ T] T [ρ T] t=1 eee t + [ρ T] G ( T ) G ( t 2 T ) ρ G(1) 2 2 ρ 1 ρ + σ δ This statistic rejects for large values where the critical values can be constructed from the limit distribution derived earlier. We still have choices of the two parameters to consider. Testing for Anomalous Periods in Time Series Data 23

24 Assumptions for Linear Regression Model Additional Material Let Q T,t = [X T,t, Z T,t ], K<, and for the stable model (i) E Q T,t ε T,t = 0 (ii) Q T,t, ε T,t is either uniform mixing size r/(r-1) or strong mixing size -2r/(r-2) for r>2 2r iii E Q T,t Q T,t = Σ Q, E Q T,t ε T,t < K, T 1 Q T,t Q T,t p t=1 sσ Q uniformly in s and T 1 T t=1 Q T,t Q T,t is positive definite almost surely for large enough T, (iv) Q T,t, ε T,t is globally covariance stationary with a nonsingular long run covariance matrix (spectral density at frequency zero) [ss] Testing for Anomalous Periods in Time Series Data 24

25 Performance of the Tests Performance of the test We examine (a) How the test works for various lengths of the anomaly (b) Small sample comparisons with some other tests Testing for Anomalous Periods in Time Series Data 25

26 Asymptotic Power Performance of the Test Testing for Anomalous Periods in Time Series Data 26

27 Asymptotic Power Performance of the Test Testing for Anomalous Periods in Time Series Data 27

28 Limiting Problem Performance of the test Testing for Anomalous Periods in Time Series Data 28

29 Small Sample Size Performance of the Test MC design T=50 T=100 T=200 T=500 X t = 1, Z t = ς t, no het X t = ς t, Z t = 1, no het X t = 1, Z t = ς t, het X t = ς t, Z t = 1, het Regression method as above, where ς t = 0.5ς t + ε t, ε t ~N(0, v) where v is chosen to normalize the variance to 1. Heteroskedastic model has u t = ς t u t where the shocks are independent of each other. Standard errors computed using White (1982) corrections Testing for Anomalous Periods in Time Series Data 29

30 Small Sample Power Comparisons Performance of the Test Green: known breaks, Blue: my test, Magenta: qll, Aqua: new Enders, Red: Quadratic, Yellow: Enders Bootstrap Testing for Anomalous Periods in Time Series Data 30

31 Relevance of the asymptotic theory Performance of the Test Anomoly in mean: sample size increasing as colors go from blue to red. Blue: T=100, Green: T=250, Orange: T=500, Red: T=1000. Testing for Anomalous Periods in Time Series Data 31

32 Do rejections indicate an Anomaly? Interpreting Rejections A problem with interpreting the results of all break tests is that they tend to have good power against not only the breaking process for which the test was defined for but also many other forms of breaks. Elliott and Mueller (2006) show that against a wide set of break models optimal tests have equivalent power envelopes. Outside of these processes it is still numerically clear that the tests are almost interchangeable with little loss in power. This means that it is difficult to interpret a rejection rejections do not indicate well the form of the breaking process. Testing for Anomalous Periods in Time Series Data 32

33 For General Break Test methods Interpreting Rejections Testing for Anomalous Periods in Time Series Data 33

34 Tests against a break in the Center of the Sample Interpreting Rejections Red is the anomaly test (10%), Green is AP and Blue is SupF Testing for Anomalous Periods in Time Series Data 34

35 Power against an anomaly Interpreting Rejections Red is the anomaly test (10%), Green is AP and Blue is SupF Testing for Anomalous Periods in Time Series Data 35

36 Do rejections indicate an Anomaly? Interpreting Rejections Blue line indicates single break, Green is double Break and Red is the Anomaly model. Testing for Anomalous Periods in Time Series Data 36

37 Distinguishing Anomalies from Permanent Breaks Expanding the Null The problem that tests have power against both anomalous periods as well as permanent breaks means that if we really want to distinguish these two then the above tests are helpful but not really all that decisive. This suggests that perhaps a test that controls somewhat for breaks might be of use. The idea would be to expand the null hypothesis to allow for a single break at an unknown point, and test this set of models against the alternative of their being an anomaly. From a testing perspective the issue that arises is that there are now parameters under the null hypothesis that are unknown under the null hypothesis (the size and location of the break) that cannot be handled through invariance. Suggestions include ad hoc tests or to use Elliott, Mueller and Watson (2013) to construct these tests. Testing for Anomalous Periods in Time Series Data 37

38 Distinguishing Anomalies from Permanent Breaks Expanding the Null We have the following processes under the null and alternative hypotheses. Alternative Hypothesis: G s s 0 = W s + ββ + δ 1 ρ 1 < r < ρ 2 dd. Null Hypothesis: G s s 0 = W s + ββ + δ 1 ρ < r dd. Note that the positions of the break in the null and alternative are not imposed to be the same. Testing for Anomalous Periods in Time Series Data 38

39 An Ad Hoc test for linear regressions Expanding the Null Consider the regression y t = βx t + β 1 x t 1 t > τ + β 2 x t 1 t > τ 0.1T + β 3 x t 1 t > τ + 0.1T + γ z t + ε t where τ is the least squares estimate for a break date and test the hypothesis H 0 : β 2 = β 3 = 0 H a : β 2 0 aaa / oo β 3 0 (which really tests for two additional breaks). This test does not control size for all δ under the null, since for small δ the estimate for the break date is poor. However it turns out that this effect is maximized at δ=0, so we can adjust the critical value in order to control size everywhere (at the cost of power). Testing for Anomalous Periods in Time Series Data 39

40 An Ad Hoc test for linear regressions Expanding the Null Size and power: Blue: ρ=0.15, Green: ρ=0.5, Red: ρ=0.85. Results are asymptotic. Testing for Anomalous Periods in Time Series Data 40

41 Distinguishing Anomalies from Permanent Breaks Expanding the Null Dealing with nuisance parameters: 1. Under both the null and alternative scale is through consistent estimation. 2. Under both the null and alternative β is dealt with through invariance. 3. Under the null, we integrate over both (δ,ρ). We need to do this whilst maintaining asymptotic size control, i.e. that for all (δ,ρ) size is controlled, not just for the weighting. Denote this distribution by Λ. 4. Under the alternative we employ a weighted average power approach as above, denote as F the distribution over (δ,ρ 1, ρ ). Testing for Anomalous Periods in Time Series Data 41

42 Distinguishing Anomalies from Permanent Breaks Expanding the Null Let - θ 0 = (δ,ρ) Θ 0 be the null values (δ R, ρ (ρ, ρ)) and - θ 1 = (δ,ρ 1, ρ ) Θ 1 with spaces as above. We want a test of the form where φ = 1 f θ 1 dd f θ 0 dλ where α=size. sss θ0 Θ 0 P[φ = 1] α EMW works by approximating Λ by distributions on Θ 0 and using these two equations along with a power bound to provide a computational procedure that estimates Λ. Testing for Anomalous Periods in Time Series Data 42

43 Distinguishing Anomalies from Permanent Breaks Expanding the Null The problem that tests have power against both anomalous periods as well as permanent breaks means that if we really want to distinguish these two then the above tests are helpful but not really all that decisive. This suggests that perhaps a test that controls somewhat for breaks might be of use. The idea would be to expand the null hypothesis to allow for a single break at an unknown point, and test this set of models against the alternative of their being an anomaly. From a testing perspective the issue that arises is that there are now parameters under the null hypothesis that are unknown under the null hypothesis (the size and location of the break) that cannot be handled through invariance. Suggestion is to use Elliott, Mueller and Watson (2013) to construct these tests. Testing for Anomalous Periods in Time Series Data 43

44 EMW Algorithm Expanding the Null The algorithm steps are essentially (a) Choose points on support of Λ for point masses (b) For any set of weights (that sum to one) over these points, we can compute the NP test (c) The critical value is determined by size given the weights, completing the test (d) The LFD given these points of support is solved using a fixed point algorithm (e) Taking these weights as true (which gives a distribution under the null), compute power. (f) Increase the critical value so that power falls by ε (g) Check size on all of the null parameter space, if there is a violation increase the number of points of support in (a) Testing for Anomalous Periods in Time Series Data 44

45 Form of the likelihoods Expanding the Null To implement, we need (a) Likelihoods for the data we use the limit experiments results above. (b) A chosen alternative point F. Here we use the same choice as in the weighted average power test discussed above, with possibly an increase in σ δ 2 to account for the models being closer together. So the density is as above for the test against the null of stability. (c) Points on support of Λ for point masses. I am following EMW in their tests for a break point and integrating over a partition of the space for δ using (symmetric) uniform distributions and points in ρ. Here the densities are constructed from 1 δ δ ρ(1 ρ 1/2 eee * ϕ δ ρ(1 ρ G ρ ρρ(1) ρ(1 ρ ϕ δ ρ(1 ρ G ρ ρρ(1) 2 ρ(1 ρ G ρ ρρ(1) ρ(1 ρ. Testing for Anomalous Periods in Time Series Data 45

46 Conclusion Conclusion Able to construct an optimal test against stability depends fairly strongly on the length of the anomaly we are looking for. Has good large sample power in the sense that it is much more powerful than using tests for instability not designed for this alternative. Also seems to work better than other tests in small samples (so far) Still there is the issue that anomalies can be confused with permanent breaks. How to construct a test which allows for a break in the null model is ongoing, how it works when breaks are more complicated remains to be seen. Testing for Anomalous Periods in Time Series Data 46

47 Enders test Additional Material The suggestion was to run the following regression M y t = β 0 + β 1j sss 2f jππ + β T 2j ccc 2f jππ T j=1 x t + u t and test the null that H 0 : β 11 = = β 2M = 0 (i.e. all Fourier terms do not enter). They suggested a bootstrap approach (no proof) to compute critical values. In application they set M=1, search over values for f j. Testing for Anomalous Periods in Time Series Data 47