Long-range dependence

Long-range dependence Kechagias Stefanos University of North Carolina at Chapel Hill May 23, 2013 Kechagias Stefanos (UNC) Long-range dependence May 23, 2013 1 / 45

Outline 1 Introduction to time series Stationarity AR(1), MA(1), ARMA(p,q) time series Spectral density and periodogram 2 Long-Range Dependence (LRD) Definitions of LRD in time and spectral domain Examples of time series with long-memory Application in finance - long-memory volatility model 3 Multivariate extension Definitions Causal and non-causal representations Examples Kechagias Stefanos (UNC) Long-range dependence May 23, 2013 2 / 45

PART A INTRODUCTION TO TIME SERIES Kechagias Stefanos (UNC) Long-range dependence May 23, 2013 3 / 45

Introduction to time series Time series is a set of observations x t, each one being recorded at a specified time t. Discrete time series: The set T 0 of times at which observations are made is discrete. Example (Population in the U.S.) t x t 1790 3,929,214 1800 5,308,483 1810 7,239,881 1820 9,638,453.. 1980 226,545,805 Kechagias Stefanos (UNC) Long-range dependence May 23, 2013 4 / 45

Mathematical model Question: What mathematical model should we use in time series analysis? We can think of each observation x t as a realized value of a certain random variable X t. Then the time series {x t, t T 0 } is the realization of the family of random variables {X t, t T 0 }. Definition (Stochastic Process) A stochastic process is a family of random variables {X t, t T } defined on a probability space (Ω, F, P). We will consider T = Z. Kechagias Stefanos (UNC) Long-range dependence May 23, 2013 5 / 45

Weak Stationarity Definition (Stationarity) A stochastic process {X t } t Z is (weakly or second-order) stationary if for any t, s Z, E X (t) 2 < EX (t) = EX (0) Cov(X (t), X (s)) = Cov(X (t s), X (0)) The time difference t s is called time-lag. Time domain perspective Consider a (weakly) stationary time series {X t } t Z. In the time domain, one focuses on the functions γ X (h) = Cov(X h, X 0 ) = Cov(X t+h, X t ), for all t, h Z ρ X (h) = γ X (h) γ X (0), Kechagias Stefanos (UNC) Long-range dependence May 23, 2013 6 / 45

ACV, ACVF called autocovariance function (ACVF) and autocorrelation function (ACF). ACVF and ACF are measures of dependence in time series. Sample counterparts of ACVF and ACF are the functions γ X (h) = 1 N N h t=1 Basic Properties (X t+ h X )(X t+ h X ), ρ X (h) = γ X (h), h N 1. γ X (0) 1 (Symmetry) ρ X (h) = ρ X ( h), h Z 2 (Range) ρ X (h) 1, h Z 3 (Interpretation) ρ X (h) close to -1, 0 and 1 correspond to strong negative, weak and strong positive correlation, respectively, in time series at lag h. Kechagias Stefanos (UNC) Long-range dependence May 23, 2013 7 / 45

Time series examples Example 1. (White Noise.) A time series X n = Z n, n Z is called White Noise, denoted WN(0, σ 2 ), if EZ n = 0 and γ Z (h) = { σ 2, if h = 0, 0, if h 0, ρ Z (h) = { 1, if h = 0, 0, if h 0. Example 2. (MA(1)). A time series is called a Moving Average of order 1 (MA(1)) if it is given by X n = Z n + θz n 1, n Z, where Z n WN(0, σ 2 ). Observe that, σ 2 (1 + θ 2 ), h = 0, γ X (h) = EX h X 0 = E(Z h +θz h 1 )(Z 0 +θz 1 ) = σ 2 θ, h = 1, 0, h 2, Kechagias Stefanos (UNC) Long-range dependence May 23, 2013 8 / 45

Time series examples and hence 1, h = 0, θ ρ X (h) =, h = 1, 1+θ 2 0, h 2. Example 3. (AR(1)). A (weakly) stationary time series {X n } n Z is called Autoregressive of order 1 (AR(1)) if it satisfies the AR(1) equation X n = φx n 1 + Z n, n Z, where Z n WN(0, σ 2 ). To see that AR(1) time series exists, suppose φ < 1 and notice that, X n = φ 2 X n 2 + φz n 1 + Z n = φ m X n m + φ m 1 Z n (m 1) +... + Z n = φ m Z n m. m=0 The time series is well defined in the L 2 (Ω)-sense because, Kechagias Stefanos (UNC) Long-range dependence May 23, 2013 9 / 45

Time series examples E ( n2 m=n 1 φ m Z n m ) 2 = n 2 for φ < 1. Also we can easily calculate, m=n 1 φ 2m σ 2 0, as n 1, n 2, φ h γ X (h) = σ 2 1 φ 2, ρ X (h) = φ h. Example 4. (ARMA(p,q)) A (weakly) stationary time series {X n } n Z is called Autoregressive moving average of orders p and q (ARMA(p, q)) if it satisfies the equation X n φ 1 X n 1... φ p X n p = Z n + θ 1 Z n 1 +... + θ q Z n q, where Z n WN(0, σ 2 ). It is convenient to express the conditions in terms of the so-called backshift operator B, defined as B k X n = X n k, B 0 = I, n Z. Kechagias Stefanos (UNC) Long-range dependence May 23, 2013 10 / 45

Spectral domain With this notation, the ARMA(p, q) equation becomes φ(b)x n = θ(b)z n, where φ(z) = 1 φ 1 z... φ p z p, θ(z) = 1 + θ 1 z +... + θ q z q are the so-called characteristic polynomials. Spectral domain perspective In the spectral domain the focus is on the function f X (λ) = 1 2π h= e ihλ γ X (h), λ ( π, π], called spectral density. Observe that f X is well defined pointwise when γ X L 1 (Z). Kechagias Stefanos (UNC) Long-range dependence May 23, 2013 11 / 45

Spectral domain Example 1. (White Noise cont d). If Z n WN(0, σ 2 ) then f X (λ) = σ2, λ ( π, π]. 2π Example 2. (AR(1) cont d). If {X n } n Z is AR(1) time series with φ < 1 and γ X (h) = σ 2 φ h /(1 φ 2 ), then f X (λ) = = ( σ 2 2π(1 φ 2 1 + ) σ 2 2π(1 φ 2 ) = σ2 1 2π 1 φe iλ 2. ) (e ihλ + e ihλ )φ h h=1 (1 + φe iλ φeiλ + 1 φe iλ 1 φe iλ ) Kechagias Stefanos (UNC) Long-range dependence May 23, 2013 12 / 45

Spectral domain Basic properties 1 Symmetry f X (λ) = f X ( λ) 2 Non-negative f X (λ) 0. For 3 Inverse discrete Fourier transform γ X (h) = π π e ihλ f X (λ)dλ. A sample counterpart to the spectral density is defined by 1 γ X (h)e ihλ = 1 N 2 X n e inλ =: I X,N(λ), 2π 2π 2π h <N with the first relation holding only at the so-called Fourier frequencies λ = λ k = 2πk [ ] [ ] N 1 N with k =,...,. N 2 2 n=1 Kechagias Stefanos (UNC) Long-range dependence May 23, 2013 13 / 45

Spectral domain I X,N is known as the periodogram and has the following properties: 1. (Computational speed). I X,N (λ k ) can be computed by fast Fourier Transform (FFT) in O(NlogN) steps, supposing N can be factored out in many factors. 2. (Statistical properties). I X,N (λ) is not a consistent estimator for 2πf (λ), but is asymptotically unbiased. the periodogram needs to be smoothed out to become consistent. Kechagias Stefanos (UNC) Long-range dependence May 23, 2013 14 / 45

PART B LONG-RANGE DEPENDENCE Kechagias Stefanos (UNC) Long-range dependence May 23, 2013 15 / 45

Slowly varying functions Definition 1: A function L is slowly varying at infinity if it is positive and, for any a > 0, L(au) lim u L(u) = 1. A function L is slowly varying at zero if the function L(1/u) is slowly varying at infinity. Example: L(u) = log(u) is slowly varying at infinity. We are now ready to introduce time series with long-range dependence. these will involve a parameter and a slowly varying function L. d (0, 1/2), Kechagias Stefanos (UNC) Long-range dependence May 23, 2013 16 / 45

Definitions of long-range dependence d (0, 1/2), L 1, L 2 slowly varying at infinity, L 3 slowly varying at zero. Condition I. X n = µ + k=0 ψ kɛ n k with {ɛ n } n Z WN and Condition II. ψ k = L 1 (k)k d 1, as k. γ(k) = L 2 (k)k 2d 1, as k. Condition III. Condition IV. k= γ(k) =. f (λ) = L 3 (λ)λ 2d, as λ 0. In statistical inference, all slowly varying function are replaced by constants (in the asymptotic sense, e.g. L 2 (u) c 2 ). Kechagias Stefanos (UNC) Long-range dependence May 23, 2013 17 / 45

Definitions of long-range depdendence Definition 2: A second-order stationary time series X = {X n } n Z is called long-range dependent (LRD, in short) if one of the non-equivalent conditions IIV above holds. The parameterd (0, 1/2) is called a long-range dependence (LRD) parameter. Other names: Long memory, strongly dependent, 1/f (1/λ) noise, colored noise, burstiness,... Definition 3: A second-order stationary time series X = {X n } n Z is called short-range dependent (SRD, in short) if k= γ(k) <. Other names: short memory, weakly dependent,... Kechagias Stefanos (UNC) Long-range dependence May 23, 2013 18 / 45

Some real examples of LRD series Kechagias Stefanos (UNC) Long-range dependence May 23, 2013 19 / 45

Some real examples of LRD series Kechagias Stefanos (UNC) Long-range dependence May 23, 2013 20 / 45

Some real examples of LRD series Kechagias Stefanos (UNC) Long-range dependence May 23, 2013 21 / 45

Some real examples of LRD series Kechagias Stefanos (UNC) Long-range dependence May 23, 2013 22 / 45

Cond. IV III As mentioned earlier Conditions I-IV are not equivalent in general. We will examine briefly how they are related. Fact: If k= γ(k) <, then the series X = {X n} n Z has a continuous spectral density f (λ) = 1 2π k= e ikλ γ(k). If cond. III fails, that is, k= γ(k) <, the fact above implies that f (λ) is continuous and f (0) = 1 2π k= γ(k) <. But by cond. IV, lim λ 0 f (λ) = lim λ 0 L 3 (λ)λ 2d =. Hence a contradiction, showing that cond. IV implies III. Definition: A slowly varying function L on [0, ) is called quasi-monotone if it is of bounded variation on any compact interval of [0, ) and if for all δ > 0, x u δ dl(u) = O(x δ L(x)), as x. Kechagias Stefanos (UNC) Long-range dependence May 23, 2013 23 / 45

Comparing cond. II and IV Cond. IV II: Informally, γ(k) = π π e ikλ f (λ)dλ = = k 2d 1 L 3 ( 1 k ) kπ kπ π π e ikλ L 3 (λ)λ 2d dλ e iz z 2d L 3( z k ) L 3 ( 1 k ) dz k 2d 1 L 3 ( 1 k ) e iz z 2d dz = k 2d 1 L 3 ( 1 k 2d) )2 cos(π(1 )Γ(1 2d). 2 and hence L 2 (u) L 3 (1/u)2 cos( π(1 2d) 2 )Γ(1 2d). These calculations can be justified assuming that L 3 is quasi-monotone. Otherwise the result is not in general even in the case L 3 (k) c 3. Kechagias Stefanos (UNC) Long-range dependence May 23, 2013 24 / 45

Comparing cond. II and IV Cond. II IV: Informally, f (λ) = 1 2π k= = λ 2d L 2 ( 1 λ ) 1 2π e ikλ γ(k) = 1 2π k= λ 2d L 2 ( 1 λ ) 1 2π k= e ikλ L 2 (k)k 2d 1 e ikλ L 2(kλ 1 λ ) L 2 ( 1 λ ) (kλ)2d 1 λ e iz z 2d 1 dz = λ 2d L 2 ( 1 λ ) 1 Γ(2d) cos(πd), 2π thus L 3 (λ) L 2 ( 1 λ ) 1 π Γ(2d) cos(πd). These calculations can be justified assuming that L 2 is quasi-monotone. Otherwise the result is not in general even in the case L 2 (k) c 2. Kechagias Stefanos (UNC) Long-range dependence May 23, 2013 25 / 45

Some real examples of LRD series Kechagias Stefanos (UNC) Long-range dependence May 23, 2013 26 / 45

FARIMA(0, d, 0) Example : If d is a non negative integer, then X t is said to be an ARIMA(0, d, 0) process if (I B) d X n = Z n, where Z n WN(0, σ 2 ) and B is the backshift operator. In simple cases of d=1,2,.., this equation is: d = 1 : (I B)X n = X n X n 1 = Z n d = 2 : (I B) 2 X n = (X n X n 1 ) (X n 1 X n 2 ) = Z n and so on when d 3. Remark: In many applications we want to difference the observed time series in order to achieve approximate stationarity. However even though differencing might seem appropriate, taking the first or second difference may be too strong! Kechagias Stefanos (UNC) Long-range dependence May 23, 2013 27 / 45

FARIMA(0, d, 0) Example cont d: For other values of d we would like to interpret the solution to the ARIMA(0, d, 0) as X n = (I B) d Z n = b j B j Z n = j=0 b j Z n j, j=0 where b j s are the coefficients in the Taylor expansion of (1 z) d = 1 + dz + = ( j j=0 k=1 d(d + 1) z 2 + 2! ) k 1 + d z j = k j=0 d(d + 1)(d + 2) z 3 +... 3! Γ(j + d) Γ(j)Γ(d) zd =: b j z j. The time series is well defined when j=0 b2 j < which depends on the behavior of b j as j. Kechagias Stefanos (UNC) Long-range dependence May 23, 2013 28 / 45 j=0

FARIMA(0, d, 0) Example cont d: By Stirling s formula Γ(p) 2πe (p 1) (p 1) p 1/2, as p, we have b j = Γ(j + a) Γ(j + b) j d 1, as j. Γ(d) Then the time series is well defined if 2(d 1) + 1 = 2d 1 < 0 or d < 1/2. Definition: The time series X n = (I B) d Z n = j=0 b jz n j is called FARIMA(0, d, 0) when d < 1/2. Since b j jd 1 Γ(d) a FARIMA(0, d, 0) series is LRD when 0 < d < 1/2, in the sense of condition I and hence II and III. Kechagias Stefanos (UNC) Long-range dependence May 23, 2013 29 / 45

FARIMA(0, d, 0) Example cont d: Fact 1: FARIMA(0, d, 0) series has spectral density ( f (λ) = σ2 2 sin λ ) 2d = σ2 2π 2 2π 1 e iλ 2d σ2 2π λ 2d as λ 0. Basic idea: A series X n = j=0 b jz n j has spectral density f (λ) = σ2 2π Here, j=0 b je ijλ = (1 e iλ ) d. b j e ijλ j=0 2 Kechagias Stefanos (UNC) Long-range dependence May 23, 2013 30 / 45

FARIMA(0, d, 0) Fact 2: FARIMA(0, d, 0) series has autocovariances γ(k) = σ 2 ( 1) k Γ(1 2d) Γ(1 d)γ(1 k d) Basic idea: γ(k) = π π Γ(1 2d)sin(πd) σ2 k 2d 1, as k. π 2π e ikλ f (λ)dλ = σ2 cos(kλ)(2 sin(λ/2)) 2d dλ 2π 0 and the formula for the last integral is known. Kechagias Stefanos (UNC) Long-range dependence May 23, 2013 31 / 45

FARIMA(0, d, 0) Kechagias Stefanos (UNC) Long-range dependence May 23, 2013 32 / 45

Long Memory in Volatility Long memory stochastic volatility model (LMSV). Consider the latent variable model for the return series r t r t = σ t v t, where v t is an independent identically distributed series with mean zero and finite variance and σ 2 t is given by, σ t = exp(h t /2), where h t is a Gaussian long memory series independent of v t. Using the moment generating function of the Gaussian distribution it can be shown for the LMSV model that ρ r 2 t (j) Cj 2d 1, as j. Kechagias Stefanos (UNC) Long-range dependence May 23, 2013 33 / 45

PART C MULTIVARIATE LRD Kechagias Stefanos (UNC) Long-range dependence May 23, 2013 34 / 45

Multivariate LRD We focus here on vector-valued (R p -valued), second order stationary times series X = {X n } n Z. Autocovariance matrix function: γ(h) = (γ jk (h)) j,k=1,...,p = EX 0 X h EX 0EX h, h Z Spectral density matrix function (if it exists) f (λ) = (f jk (λ)) j,k=1,..,p, λ ( π, π]. It satisfies π π e inλ f (λ)dλ = γ(n), n Z. Kechagias Stefanos (UNC) Long-range dependence May 23, 2013 35 / 45

Mutlivariate LRD Remark 1: The cross covariances γ jk (n) may not be equal to γ jk ( n) when j k. Therefore in contrast to the univariate case, it is not true in general that γ(n) = γ( n), n Z. (1) If (1) holds, the time series X is called time reversible. Remark 2: Since (1) may not hold, f (λ) is in general, complex valued. f (λ) is Hermitian symmetric, non negative definite and satisfies f ( λ) = f (λ), λ [ π, π). We are now ready to extend Conditions II and IV to the multivariate case. Let D = diag(d 1,..., d p ) with d j (0, 1/2), j = 1,..., p. Kechagias Stefanos (UNC) Long-range dependence May 23, 2013 36 / 45

Multivariate LRD Condition II-v. The autocovariance matrix function of the time series X = {X n } n Z satisfies: γ(n) = n D (1/2)I L 2 (n)n D (1/2)I, (2) where L 2 is an R p p -valued function satisfying L 2 (u) R, as u +, for some p p matrix R. Equivalently we can write, γ jk (n) = L 2,jk (n)n (d j +d k ) 1 R jk n (d j +d k ) 1, as n. Condition IV-v. The spectral density matrix function satisfies f (λ) = λ D L 3 (λ)λ D, (3) where L 3 is a C p p -valued, Hermitian symmetric, non-negative definite matrix function satisfying L 3 (λ) G, as λ 0, for some p p, Hermitian symmetric, non-negative definite matrix G. Equivalently, f jk (λ) = L 3,jk (λ)λ (d j +d k ) G jk λ (d j +d k ) =: g jk e iφ jk λ (d j +d k ) as λ 0, Kechagias Stefanos (UNC) Long-range dependence May 23, 2013 37 / 45

Multivariate LRD where g jk R and φ jk [ π, π). Remarks: 1 The individual component series {Xn} j n Z, j = 1,..., p, of a multivariate LRD series are LRD with parameters d j, j = 1,..., p. 2 Note from (3) that f (λ) is Hermitian, non negative definite. The entries φ jk are referred to as phase parameters. 3 We supposed for simplicity that all slowly varying functions behave as constants. 4 The squared coherence function Hjk 2 (λ) = f jk(λ) 2 /(f jj (λ)f kk (λ)) satisfies 0 Hjk 2 (λ) 1. As λ 0, this translates into 0 lim λ 0 G jk 2 λ 2(dj +dk) G jj λ 2d j G kk λ 2d k = G jk 2 G jj G kk 1 (4) and also explains why the choice of λ (d j +d k ) is natural for the cross-spectral density f jk (λ). Kechagias Stefanos (UNC) Long-range dependence May 23, 2013 38 / 45

Comparing cond. II-v and IV-v Suppose that L 2,jk are quasi-monotone. Then cond II-v implies cond IV-v with G jk = g jk e iφ jk given by { Rjk R kj φ jk = arctan tan( π } R jk + R kj 2 (d j + d k )), g jk = Γ(d j + d k )(R jk + R kj ) cos( π 2 (d j + d k )). 2π cos(φ jk ) Suppose that RL 3,jk, IL 3,jk are quasi-monotone. Then cond IV-v implies cond II-v with { R jk = 2Γ(1 (d j +d k )) RG jk sin( π 2 (d j +d k )) IG jk cos( π } 2 (d j +d k )) Kechagias Stefanos (UNC) Long-range dependence May 23, 2013 39 / 45

Linear Representation Suppose that the series X n has a linear representation of the form X n = Ψ m Z n m, m= where {Ψ m = (ψ m,jk ) j,k=1,...,p } m Z, a sequence of real matrices such that ψ m,jk = L jk (m) m d j 1, where L(m) = (L jk (m)) j,k=1,...,p is an R p p -valued function satisfying L(m) A + as m, and L(m) A as m, for some p p real matrices A + = (a + jk ) j,k=1,...,p, A = (a jk ) j,k=1,...,p. Then X n is LRD. Kechagias Stefanos (UNC) Long-range dependence May 23, 2013 40 / 45

Example Consider the series X n = Ψ m Z n m, m=0 where {Z n } n Z, and Ψ m as earlier. Then the phase parameters appearing in the (j, k) element of the spectral density matrix of X n have the form φ jk = (d j d k ) π 2. Question: What behavior should I consider for Ψ m to get a causal represenation? Linear combinations of c a,b m = m b cos(2πm a ) and s a,b m = m b sin(2πm a ) (?) Kechagias Stefanos (UNC) Long-range dependence May 23, 2013 41 / 45

FARIMA(0,D,0) M + = (m + ij ), M = (m ij ) Rp p. {Z n } n Z such that EZ n = 0 and EZ n Z m = σ 2 I if n = m, and = 0 if n m. Define a multivariate FARIMA(0, D, 0) series as X n = (I B) D M + Z n + (I B 1 ) D M Z n. For the component wise spectral density I have where c j,k = σ2 2π f jk (λ) c j,k λ (d j +d k ), as λ 0, (e i(d j d k ) π 2 A1 + e i(d j d k ) π 2 A2 + e i(d j d k ) π 2 A3 + e i(d j d k ) π 2 A4 ) and A 1 = p t=1 m+ jt m+ kt, A 2 = p t=1 m+ jt m kt, A 3 = p t=1 m jt m+ kt, A 4 = p t=1 m jt m kt. Kechagias Stefanos (UNC) Long-range dependence May 23, 2013 42 / 45

Lots and Lots of Fun Stuff Self similar processes Issing model in two dimensions Infinite sourse Poisson model with Heavy tails Kechagias Stefanos (UNC) Long-range dependence May 23, 2013 43 / 45

References The results in part c are a joint work with my advisor Vladas Pipiras. 1 Brockwell, P. J. & Davis, R. A. (2009), Time series: Theory and methods, Springer. 2 Pipiras, V. & Taqqu, S. M. (forthcoming 2014), Long-Range Dependence and Self-Similarity. 3 Doukhan, P. Oppenheim, G. Taqqu, S. M. (2003), Theory and applications of Long-Range Dependence Kechagias Stefanos (UNC) Long-range dependence May 23, 2013 44 / 45

Thank you! Kechagias Stefanos (UNC) Long-range dependence May 23, 2013 45 / 45