Summer Researh Projet 25-26 The Distributors of Change Points in Long 11ennoryProesses Guan Yu Zheng Department of Mathematis and Statistis University of Canterbury
The distributions of hange points in long memory proesses Guan Yu Zheng Supervisors: Maro Reale and Bill Rea Department of Mathematis and Statistis University of Canterbury New Zealand February 9, 27 Abstrat In this paper we present the properties of empirial distributions of different statistis (e.g. standard deviation and number of breaks) related to the presene of strutural breaks in simulated Frational Gaussian Noise series with various Hurst parameters. Strutural Breaks are deteted with Atheoretial Regression Trees, a strutural break identifiation method. The simulation results were applied to four ase studies to hek whether a Regime Swithing or Frational Gaussian Noise model is more adequate. Keywords: Long-range dependene, strong dependene, global dependene, Hurst phenomena. Introdution The onept of long memory was introdued by Mandelbrot in 1965. It also is known as long-range dependeny, strong dependene or Hurst phenomenon. Both of statisti and eonomi literatures have made great effort to develop this field of study. Up to - 1 -
now, the presene of long memory time series is beoming more and more important in new areas like finane and network ommuniations as well as hydrology, geophysis and limatology. The intention of this paper is to provide a simple approah to test whether a given dataset is better desribed by a long memory model or a regime swithing model The next setions of this paper are organized as follows: setion (2) gives a brief definition of long memory proess and self-similarity. Setion (3) presents overview of Frational Integration (FI), Frational Gaussian Noise (FGN) and Regime Swithing (RS). Setion ( 4) desribes the distributional properties of strutural breaks for different Hurst oeffiients. Setion (5) presents the results of our investigation of self-similarity properties of four real datasets. Setion (6) presents the onlusions. 2 Definition of Long memory and Self-similarity In this setion we introdue the onept of long memory and self-similarity. Then, an illustration is provided of the equivalene of self-similarity and long memory. 2.1 Long memory There are several ways of defining the long memory of a disrete time series formulated both in time and frequeny domain. A simple definition proposed by Baillie [2] was given in eonometri literature and illustrated below. Definition 1 The proess possesses long memory if ll lim 2]PU)I ~ fl---4. ;=-n Equivalently, the spetral density f((jj) of the data will be unbounded at low frequenies. - 2-
tree ring in Campito Mountain simulate anna(1, 1) -3-2 -1 1 2 1 2 3 4 5 Time Time Series am11 Series x 2 4 6 8 1 2 4 6 8 1 Lag Lag Series: x Raw Periodogram Series: x Raw Periodogram. :1.2.3 frequeny bandwidth = 5.13e-5.4.5..1.2.3 frequeny bandwidth = 5.35e-5.4.5 Figure 1: Time series plot, autoorrelation and raw periodogram with tree ring width in Campito Mountain (Left) and a simulated ARMA (1,1) with p =.5 and q =.5. Silverberg and Verspagen [3] pointed out that the deay rate of the Autoorrelation Funtion (ACF) in a short memory proess (ARMA) is about exponential with a zero spetral density at the origin. However, the ACF of a long memory deays at a hyperbolial rate whih is muh slower than the exponential rate found for stationary ARMA proesses. Furthermore, the power of spetrum is ompletely dominated by the low frequeny omponent. And how about trying to first differene the raw dataset, and what hanges an we find? The whole time series is smoothed, and appear more stationary. Even though the ACF deays very quikly, its values still stay at some signifiant level. Furthermore, the power in the origin of periodogram is perished. In other words, there is no power in the origin. There are lear signs that the data is over-differened. - 3-
Series damp LL "1: u <( I - - - - - - I - -~r - I I 1 -- ---- / \ -- -- ------ - -- -- - ~- - ~--- -- ---- I I I I I 2 3 4 5 6 Lag E :J 1-...... u <])...n (",) + <]).,..--- (",) I <]) Series: x Raw Periodogram.,..---..1.2.3.4.5 frequeny bandwidth= 5.13e-5 Figure 2. ACF (above) and Periodogram (Below) of first differened Campito Mountain. The Autoregressive Frational Integrated Moving Average (ARFIMA) model was derived to overome this type of problem. Formula 1 The ARFIMA (p,d,q) model is expressed as t/j(b)t/ % 1 = B(B) 1 Where (e 1 ) is a White Noise with zero mean, and 11d = (1- B)d, where de (,.5), is a differene operator. In this paper, the simplest ARFIMA model, whih is the fational integrated noise (FI) or ARFIMA (O,d,O), is onsidered - 4-
Formula 2 The FI(d) model is expressed as t!..dx, =, Its ounter-part for ontinuous time series is the Frational Gaussian Noise (FGN). We would explain it later. The relationship between d and the Hurst oeffiient His: H =.5 + d. 2.2 Self-similarity The onept of self-similar proess was first introdued by Kolmogorov and its definition as presented by Mandelbrot and van Ness [4] is: Definition 2 a stohasti proess (X (t),t 2 ) is self-similar with Index H >, if have the same finite dimensional distribution. His alled Hurst parameter or self-similar parameter with values between and 1. In other words, the stohasti points have the same statistial properties at different periods of time in time series data. 2.3 Connetion between long memory and self-similarity The auto-orrelation funtion an be expressed as 1 p(k) = 2[(k + 1)2H - 2k2H + (k -1)2H], in term of self-similarity. When H =.5, the proess is a White Noise. Interestingly, the orrelations deay at very slow rate and are not summable for Y2 < H < 1. This = 1 implies LP(k) ~ oo; therefore, a self-similar proess with -:::; H < 1 an be k=-= 2 onsidered a long memory proess. 3 Model This paper onsiders two alternative models the Regime Swithing (RS) vs. Frational Integration (FI) or Frational Gaussian Noise (FGN), and they are applied to four ase studies to help identifying whih model is more appropriate to desribe them. The FGN an be defined from definition 2 and below from Mandelbrot and van Ness [4]. - 5 -
Definition 3: a stohasti proess (X (t), t 2 ) has stationary inrement, if X(t+)-X()=d X(t)-X(O) The time series plots were given below for illustration. Another point we should notie is that the FGN proess is self-similar if the H parameter is onstant through the series. FGN with H=.5 FGN with H=.65 2 4 GOO 8 1 2 4 GOO 8 1 Time Time FGN with H=.8 FGN with H=.95 2 4 GOO 8 1 2 4 GOO 8 1 Time Time Figure 3: Four different Frational Gaussian Noise time series plot. The alternative model is a Regime Swithing or strutural break model presented by Ohanissian, Russell and Tsay[5], Granger and Terasvirta [6]. This model ontains stationary sub-time series (stationary ARMA models) with probabilisti hanges on state levels. Its formulation was presented by Chen and Tiao[7] is as follows. - 6-
Formula 3: a (disrete) time series y 1 if Where (x,,t;::: ) is loal-stationary ARMA model, (;t,,t > O)E N(O,a 2 ), and p 1 is binary variable with Prob (p, = 1) =a and Prob( (p, = ) = 1-a. In Regime Swithing, the mean levels hange over time aording to the probability of p 1 During the hanges or breaks, the series are stationary and have different statistial properties. Hene, Regime Swithing is not self-similar. Some authors, like Russell, Ohanissian and Tsay [5], laimed that RS an be distinguished from a long memory proess. But the two alternative models are almost observationally equivalent. Hene the two models are diffiult to tell apart. In setion 5, we would provide some distribution tests for the four datasets and we shall try to tell whether a FI/FGN or RS is more appropriate. 4 Properties of simulated Frational Gaussian Noises In this part, nine FGN series with different Hurst parameters (H=.55,.6,.65,.7,.75,.8,.85,.9 and.95) were simulatedlooo times with the statistial pakage fseries[8] in R[9], and were broken into regimes by ART (Cappelli and Reale[lO]). The length of eah series onsisted of 545 sequential points whih is the same as example of the Capito Mountain. Then the empirial distributions of some statistis were derived from evaluating the statistial properties on these regimes. These statistis were number of breaks, regime length, mean level, standard deviation, skewness and kurtosis. Another R pakage MASS was used to estimate the distribution parameters by the Maximum Likelihood approah. 4.1 Number of breaks The number of breaks is an integer value, and its distribution is disrete. The Poisson distribution gave the best approximation and was applied. The Normal distribution was used as an alternative. The shape of the distribution was symmetri when H was larger than.9. Then, it turns to be right-skewed ash beame smaller. Hene, we found that the Poisson distribution fitted well the disttibution for large H values (H2..8). - 7 -
H value Expeted number H value Expeted number of breaks of breaks.95 8.29.7.75.9 6.82.65.15.85 5.1.6.15.8 3.19.55.75 1.76 Table 1: Expeted number of breaks in FGNs. The distribution hanged to exponential-like shape if H was less than.8. As the number of breaks tend to zero, but this also meant that for.5 :=:;; H <.8 it was not a problem to distinguish between long memory and RS. Another point we notied is that the expetation of number of breaks was shrunk as H was lessened. That implies that the FGN proesses beame "stationary" as no breaks were deteted. - 8 -
Dist of breaks with H=.95 Dist of breaks with H=.9 ::-. ::-. (/) "" "" (/) >]) >]) :~ i 5 1 15 2 4 6 8 1 12 x95$br x9$br Dist of breaks with H=.85 Dist of breaks with H=.8 ::-. ::-. (~ (/) "" "" >]) >]) 2 4 6 8 1 2 4 6 8 x85$br x8$br Dist of breaks with H=.75 Dist of breaks in H=.65 ::-. >- (/) "" il >]) >]) 2 3 4 5..5 1. 1.5 2. i x75$br x65$br Figure 4: Empirial distribution of the number of breaks with Hurst parameters. (Solid lines are Normal lines, dash lines are Poisson) 4.2 Regime length The distribution of original regime length is highly right-skewed and gamma distributed. It did not present well-behaved features. Natural logarithm transformation was used and gave some interesting results. After log-transformation, distribution is less skewed, and it is quite symmetri at.9:::; H < 1. Gamma and Normal distribution were used. The Gamma distribution probably was the better one to apture the hanges in the empirial distribution. However, distribution was atypial at small H value (.5 :::; H <. 8 ). - 9-
Dist of log regime length with H=.95 Dist of log regime length with H=.9 3 4 5 6 7 8 9 3 4 5 6 7 8 9 lnre95 lnre9 Dist of log regime length with H=.8 Dist of log regime length with H=.75 3 4 5 6 7 8 9 4 5 6 7 8 lnre8 lnre75 Dist of log regime length with H=.7 Dist of log regime length with H=.65 ~~... J >-- n >-- "=' (/) "=' (/) Q) Q) ~ 4 5 6 7 8 5 6 7 8 lnre7 lnre65 Figure 5: Empirial distributions of log regime length. Gamma (solid) and Normal (dash) The expeted value of the regime length beame larger and larger as H beame smaller and smaller. This is onsistent with the findings of the previous results as the number of breaks tends to zero. At.5 s H s.65, the regime length was very likely to be equal to the length of the series. H value Expetation of log H value Expetation of log regime length regime length.95 6.46 (637.61).7 8.23 (3763.93).9 6.65 (771.26).65 8.52 (54).85 6.95 (142.3).6 8.59 (5388.83).8 7.37 (1584.58).55 8.6 (545).75 7.81 (247.29) Table 2: Expetations of log regime length and original regime length. - 1-
4.3 Mean levels Differently from the previous two statistis, the empirial distribution of the mean always appears Normal. The entre was zero, and the sample standard deviation beame smaller ash moved from.95 to.55. The Logisti distribution was used as well. It might be a reasonable alternative. Dist of mean with H=.95 Dist of mean with H=.85-3 -2-1 2 3-2 -1 2 x95$mn x85$mn Dist of mean with H=.8 Dist of mean with H=.7-2 -1-1. -.5..5 1. x8$mn x7$mn Dist of mean with H=.65 Dist of mean with H=.55 -.4 -.2..2.4.6 -.5..5 x65$mn x55$mn Figure 6: Empirial distributions of mean. Normal (solid) and Logisti (dash) 4.4 Skewness In term of the sknewness, the empirial distribution showed symmetry through all H values. Its entre shifts from.8 to 1, and the density of point at 1 beomes larger and larger ash beomes smaller. In other words, the skewness is more stable where His small. When H=.55, the skewness was 1. Additionally, densities along the two-sides deayed very quikly. Beause of these features, Logisti and normal distribution worked well where.8:::; H < 1. The Logisti just did a little better than the Normal for this ase.. 11
The rest might be estimated by double-exponential distributions. Dist of SD with H=.95 Dist of SD with H=.9 f No:l [./\ ~..6.8 1. 1.2 x95$sd ~====~--=~=p==~~~=f~=-h~.4.6.8 1. x9$sd Dist of SD with H=.8 Dist of SD with H=.75.7.8.9 1. 1.1 1.2 x8$sd.7.8.9 1. 1.1 x75$sd Dist of SD with H=.65 Dist of SD with H=.6 :>. ""' (() ID ~~ ~.92.94.96.98 1. 1.2 1.4 1.6 x65$sd.99.995 1. 1.5 1.1 1.15 x6$sd Figure 7: Empirial distributions of standard deviation. Normal (Solid) and Logisti (Dash) 4.5 Kurtosis Similar to the setion 4.4, symmetry and diminished dispersion were apparent features in these distributions through all Hurst oeffiients. The only differene was that the entre of the distribution was approximately zero. 4.6 omments In this setion, the distributions of statistis for FGN's with different H oeffiients were generally identified at large H values between.8 andl whereas further studies are needed for.5~h<.8. - 12-
Dist of Kurtosis with H=.95 Dist of Kurtosis with H=.9-1.5-1. -.5..5 1. 1.5 2. - 1. -.5..5 1. 1.5 2. 2.5 x85$kur x8$kur Dist of Kurtosis with H=.8 Dist of Kurtosis with H=.7-1 2 -.5..5 1. x8$kur x7$kur Dist of Kurtosis with H=.65 Dist of Kurtosis with H=.55 -.4 -.2..2.4 -.3 -.2 -.1..1.2 x65$kur x55$kur Figure 8: Empirial distributions of Kurtosis. Nmmal (Solid) and Logisti (Dash) 5. Disussion In this setion, we applied the results of setion 4 to four ase studies, and drew some useful information to tell whether long memory with stationary inrement or regime swithing is more appropriate. The desription of all datasets was given by Rea et al [11]. In the next paragraphs, we present the distributions of number of breaks and log regime length as they appeared partiular effetive in identifying between the two alternative models. Poisson and gamma distribution were respetively applied. Moreover, Figure 9-12 provided empirial distributions on eah ase, and Figure 13-16 presented time series and its strutural breaks by ART. The results for the other statistis are available on requests from the author. - 13-
5.1 Nile Minima The estimated Hurst parameter for the Nile Minima was.837 given by the Whittle estimator in fseries. ART returned 1 breaks i.e. 11 regimes in this dataset. The expeted number of breaks in the simulated series for the onesponding H is 11.18. The probability of having 1 breaks is 11.7 perent whih was almost the highest through all the densities. Hene, this an be onsidered a long memory proess at.95 onfidene level. In term of regime length, the simulation showed the expeted log regime length was 3.64 (38.9). And, log regime length from Nile Minima was 4.33 (76.25). That suppmted the hypothesis that the Nile River followed a FGN proess. Hist of simulated number of break at FGN H=.837._... ~ 'Ui Q) ~ 5 1 15 2 Number of breaks Hist of simulated regime length at FGN H=.837 "1: ~ 'Ui Q) <'-! ~ 2 4 6 8 1 regime length Figure 9: Empirial distribution of number of breaks and log regime length in Nile yearly minimum water levels 662 to 1284AD. 5.2 Camptio Mountain ART found 12 break points in Campito Mountain. However, the expeted break point was 5.97 given FGN with H=.876. The hane of having 12 was smaller than 1 perent. - 14-
Compared to 6.36 (578) from simulated data, the sample mean of log regime length was slightly smaller (6.2, (491.27)) with a little enor. The Campito Mountain data is onsidered a lass example of a FI proess. However the result based on our simulations suggested the regime swithing was more suited thus supporting the results of Rea et al [ 11]. Hist of number of breaks with FGN H=.876 2,-o '(ii '<'""" (!) q 5 1 15 number of breaks Hist of log regime length of Camp ito with H=.876 "T 2,-o '(ii ("! (!) q 2 4 6 8 1 12 log regime length Figure 1: Empirial distribution of number of breaks and log regime length in tree ring width in Capito Mountain 3435BC to 1969AD. 5.3 Shihua Cave The break number was 14 in this dataset. Whereas the expetation of the number of breaks was only 5.79. The probability of having 14 breaks was.17. That implies the FGN model poorly performed against the RS. Similarly, the distribution of the long regime length is far from the gamma distribution. Sample average of log regime length was 7.25 (1416), and expeted length from - 15 -
gamma was 5.65 (285.8). with the data. Obviously FGN and Self-similarity were inonsistent Hist of simulate Shihua ave with FGN H=.83.2;- 'Ui Q).,.-- 2 4 6 8 1 12 14 number of breaks Hist of log regime length with FGN H=.83.2; 'Ui Q) ---... -- 2 4 6 8 1 log regime length Figure 11: Empirial distribution of number of breaks and log regime length in thikness of annual layers of a stalagmite in Shihua Cave 665BC to 1985AD. 5.4 Elk lake Like the previous two ases, the empirial Poisson and gamma distribution presented an apparent ontrast with the observed results. The data was not FGN. The data was given 8 break points and 9 regimes. For a self-similar long memory proess of this distribution we have approximately 4.15. The probability of having 8 breaks was just 3 perent. In addition, the sampled log regime length was 8.58 (532) whih was muh larger than 7.33 (1526.85) from model. Billet al [11] also introdued infrequent hanges on mean through time sales. That strongly suggested Regime swithing would desribe to data better. - 16-
Hist of simulated elk lake with FGN H=.86 (~ 2;- 'in <D "' q 2 4 6 8 1 12 number of breaks Hist of simulated ekllake with FGN H=.86 ~. 'in ~ <D q 2 4 6 8 1 12 log regime length Figure 12: Empirial distribution of number of breaks and log regime length in tree ring width in Capito Mountain 3435BC to 1969AD. 6. Conlusion We have proposed a data driven parametri proedure to distinguish between long memory model and RS. The tehnique is to generate FGNs, introdue parametri distributions for eah quantity by breaking series using ART, and ompare to atual data. Four data sets are onsidered and the long memory behavior tests. We found that distributions were easy to derive when Hurst parameter is high enough (usually larger than.8). When Hurst was lower than that point, some other tools needed to form theoretial distribution model. But given the low likelihood of having breaks for.5:::; H <.8, this study provides a useful tool to distinguish between long memory and RS. Three of the four examples exhibit regime swithing property and non-self-similarity - 17 -
with lear results. The results here obtained an be exploited to onstrut a test for self-similarity vs Regime Swithing. Yearly minimum water levels in Nile river (f) - (J) = > = - (J) ('() - 2 - CD ~ - = ~~\~J~~~~~~~~~ I I I I I I I 1 2 3 4 5 6 Time a< 18.5 3.5 11 16 123 1154 1118 1292 126 Figure 13: Time series of Nile river yearly minimum water levels and its ART. E..,--- q = = :@ = ~ ) : ""' = Tree ring width in Campito Mountain -3-2 -1 1 2 Time 53.26 47.69 36.31-18 -
Figure 14: Time series of Capito mountain and its ART. Thikness of ShiHua Cave from 665BC to 1994AD U) U) LD (I) _ u LD 5 1 15 2 25 Time 41.7 55.33 Figure 15: Time series of thikness in Shihua Cave and its ART - 19-
= 1 <n <n Q) : _.:o<: = u :E L.O = Time series plot of thikness of elkvarve 2 4 6 8 1 Time b : 7 12.5 4.564 1.914.9576 2.63 1.784 Figure 16: Time series of thikness of Elk lake varve sequene and its ART. Referenes [1] J. Beran. Statistis for long memory proesses. Chapman & Hall /CRC Press, 1994. [2] R.T. Baillie. Long Memory Proesses and Frational Integration in Eonometris. Journal of Eonometris, volume 73: 5-59, 1996. [3] G. Silverberg & B. Verspagen. Long Memory in Time Series of Eonomi Growth and Covergene. Applied Dynami Modeling of the Austrian Fous Sessions of the loth SASE Conferene, Vienna, Austia, 1999. [4] B. Mandelbrot and John W. Van Ness. Frational Brownian Motions, Frational Noises and Appliations. SIAM Review, 1(4):442-437, 1968. [5] A. Ohanissian, J.R. Russell and R.S. Tsay. True or Spurious Long Memory? A New Test. [6] Granger, C. and T. Terasvirta. Oasional strutural breaks and long memory. UCSD Disussion Paper, 1999. [7] C. Chen and G. Tiao. Random Level-Shift Time Series Models, ARIMA - 2-
Approximations, and Level-Shift Detetion. Joumal of Business and Eonomi Statistis 8:83-97,199. [8] Diethelm Wuertz, many others, and see the SOURCE file.fseries: Finanial Software Colletion-/Series, 25. R pakage Version 22.163. [9] R development Core Team. R: A language and environment for statistial omputing. R foundation for Statistial Computing, Vienna, Austria, 25. ISBN 3-951-7-. [1] C. Cappelli and M. Reale. Deteting hanges in mean levels with Atheoretial Regression Trees. Researh Report UCMSD 25/2, Department of Mathematis and Statistis, University of Canterbury, 25. [11] B. Rea, M. Reale and J. Brown. Are Long Memory Time Series Self-Similar? Department of Mathematis and Statistis, University of Canterbury, 26. - 21 -