Modelling and Estimating long memory in non linear time series. I. Introduction. Philippe Soulier

Transcription

1 3. Introduction Modelling and Estimating long memory in non linear time series. Definition of long memory Philippe Soulier Université Paris X soulier 3. Linear long memory processes 4. Non linear long memory processes 5. Estimation of long memory I. Introduction Long memory processes (linear or non linear) are considered mainly in two fields of applications: teletraffic and financial time series. The topics covered in these lectures are related to modelling financial time series. For fundamental results, references and applications, cf. Theory and applications of long-range dependence. Edited by Paul Doukhan, George Oppenheim and Murad Taqqu (3). {P t } financial time series. Returns: Deutsche Mark/ US Dollar Exchange Rate Volatility : R t = (P t P t )/P t. σ t = E [ R t R t, R t,... ] Stylised facts Lecture notes of T. Mikosch (4) Returns are uncorrelated, martingale difference sequence. Non linear transformations of the returns are (strongly) correlated. E.g. log-squared returns Volatility of Deutsche Mark/ US Dollar Exchange Rate 3 Issue: the returns form a stationary process. Exchange rate US $/ DM; 985/9/6 to 998/5/ Not to be discussed here. 3 stationary process! 4

2 Volatility of DM/$ Exchange Rates ACF Lag Two issues: What processes can be used to model long memory in the volatility of financial time series? What are the properties of these processes, what statistical procedures can be used? Sample autocorrelation of log-squared returns. Figures taken from Hurvich and Ray (3). 5 II. Definition of long memory A stationary process {X t } is said to have long memory if the asymptotic behaviour of some usual statistics is very different from that of a weakly dependent (or short memory) process, such as i.i.d. or strongly mixing sequences. Examples: Weak convergence of partial sums with an unusual normalisation and/or limiting distribution. Empirical autocorrelation; Discrete Fourier transforms; Many others, not to be discussed here. Long memory is an asymptotic property, with no precise definition 6 Working definition of long memory A weakly stationary process {X t } exhibits second order long memory if its autocovariance function is regularly varying: H (/, ) is the Hurst index; cov(x, X t ) = L(t) t H. L is slowly varying at infinity, i.e. x >, lim t L(tx)/L(t) =. 7

3 Consequence: n var X k = ˆL(t) n H, k= where ˆL is slowly varying at infinity and equivalent to L, up to a multiplicative constant, depending only on H. Sometimes taken as a definition. Equivalent formulation : the spectral density function: f(x) = π k Z cov(x, X t )e ikx. {X t } has long memory if f is regularly varying at zero: f(x) = L(x)x H. L is slowly varyig at zero, lim t L(t)/ L(/t) = c H. Not exactly equivalent. See Tauberian theorems for regularly varying functions in Bingham, Goldie and Teugels (987). 8 9 III. Gaussian and Linear Long Memory Processes Gaussian process A Gaussian process has long memory if and only if its autocovariance function is regularly varying with index H, / < H <. Linear process X t = j Z a jɛ t j, a j <, {ɛ t } i.i.d. sequence with zero mean and finite variance. If {ɛ t } is Gaussian White Noise, then {X t } is Gaussian. Sufficient condition for long memory: Examples Fractional Gaussian Noise: Gaussian process with autocovariance function: cov(x, X t ) = σ { t + H t H + t H }. Spectral density: f(x) = c H σ sin (x/) k Z kπ + x H. a j = l(j)j d /, d (, /), l slowly varying. Then {X t } has long memory with Hurst index H = d + /.

4 Asymptotic properties ARFIMA(p, d, q) process X t = (I B) d U t = j= U t : ARMA(p, q) stationary and invertible. H = d + /. Spectral density f(x) = e ix d f U (x). f U is smooth and positive around zero. Γ(d + j) j! Γ(d) U t j. Let X t be a Gaussian or linear long memory process with Hurst index H. Convergence of partial sums L [nt] (n)n H X k k= B H : fractional Brownian motion: D = σh B H (t) cov(b H (s), B H (t)) = { t H t s H + s H } The normalisation is the standard deviation of the partial sums. 3 Convergence of empirical variance l(n) n α n (X t ) w t= - α = / and Z Gaussian if H < 3/4, - α = H and Z non gaussian if H 3/4. Z Gaussian white noise Fractional Gaussian noise H = / H =.7 Estimation of the Hurst index by spectral methods next talk 4 Fractional Gaussian noise ARFIMA(,.4, ) H =.9 H =.9 5

5 Brownian motion Fractional Brownian Motion H = / H = Empirical autocorrelations : Gaussian white noise, FBM.7, FBM.9, ARFIMA(,.4,) 6 Fractional Brownian Motion cumulative sum H =.9 ARFIMA(,.4, ) 7 4 We have recalled the main properties of Gaussian and linear long memory processes. They are quantitatively different from those of short memory processes cumulative sums: ARFIMA(,.4,); Brownian motion, FBM.7 and FGN.9 Different normalizations, but as in the short memory case, linked to standard deviation. Asymptotic distributions are Gaussian or related to Gaussian (weighted chi-squared). Invariance principle to Fractional Brownian motion. 8 9

6 Bilinear type models IV Non linear long memory Processes We will next describe two classes of non linear long memory models, present some of their asymptotic properties, and see how they differ from those of Gaussian and linear long memory processes. Statistical issues will be addressed in the second lecture. There are many other models, that we will not have time to discuss. E.g. Random coefficients AR or ARCH processes; Markov switching processes; Stochastic unit root, processes linked to queues; etc. Stochastic volatility models {ɛ t } i.i.d., E[ɛ] =, E[ɛ ]=, R t = σ t ɛ t, σ t = E[R t R s, s t ] is the conditional variance, i.e. the volatility. {R t } is a martingale difference (hence uncorrelated) sequence. Can {σ t }, { R t }, {R t }, {log(r t )} have long memory? ARCH processes σ t = a + a j R t j, a j. j= If j= a j <, {σ t } is strictly stationary and has short memory. If j= a j =, no weakly stationary solution: E[σ t ] =. Strictly stationary solution if a j cρ j, ρ <. Kazakevičius et Leipus (3) The FIGARCH problem : does there exist a strictly stationary solution if a j = Γ( d + j)/(γ( d) j!) d (, /)? or more generally, a j cj d /, j= aj =. Exponential models To avoid the difficulties of ARCH type processes, model directly the log-squared volatility: R t = σ t ɛ t, σ t = e Yt/, Y t = ψ j ζ t j. j= {ɛ t }, {ζ t } i.i.d. sequences; for each t, ζ t is independent of {ɛ s, s t}, ɛ t and ζ t can be either independent or dependent. If ψ j cj d, d (, /), then {log(σ t )} has long memory with Hurst index H = d + /. 3

7 e3 e3 3e3 4e3 5e In both cases, if ψ j cj d, d (, /), {σ t }, {R t }, { R t u } have long memory with same Hurst index H = d + / and FI-EGARCH processes (Nelson 99, Bollerslev and Mikkelsen 996). ζ t = g(ɛ t ), E.g. Nelson (99) suggested g(z) = θz + γ( z E[ ɛ ]), modelizes the so-called leverage effect. LMSV process (Harvey 998, Breidt et al. 998) {ζ t } is independent of {ɛ t }. [nt] n H D U k = σu B H (t) k= with {U t} any of the above processes, under moment conditions, satisfied if {ζ t} is Gaussian. Surgailis and Viano () If P(ɛ t = ) =, log(r t ) = Y t + log(ɛ t ) has long memory with Hurst index H = d + /. 4 Hence it is sufficient to study the memory of the log-squared returns. 5 Simulation of LMSV processes : R t = ɛ te Yt/, {Y t} ARFIMA(,r,), d = {ɛ t} Gaussian {ɛ t} Pareto log squared returns 6 Sample autocorrelation of log-squared returns of LMSV process. 7

8 Duration driven long memory Spurrious long memory: It has been known for a long time that in finite samples, slow decay of the autocovariance function can be an artefact: of polynomial trends (Bhattacharya et al. 983) of structural break (Diebold and Inoue, Mikosch and Starica 4) Cumulative sums of LMSV process with Gaussian shocks: returns, absolute, squared and log-squared returns But polynomial trends can be eliminated by differencing; stationary models with structural change have been introduced, in particular as a model for volatility. 8 9 Stationary models with structural change General shot noise process Renewal-Reward (Taqqu and Levy 986); Long memory shot noise (Giraitis, Molchanov and Surgailis 993, Klüppelberg and Kühn 4); ON-OFF process (Taqqu, Willinger and Sherman 997, Heath, Resnick and Samorodnitsky 998); Infinite source Poisson (Mikosch, Resnick, Rootzen, Stegeman ), with random transmission rate (Maulik, Resnick and Rootzén ); Error duration (Parke 999, Hsieh, Hurvich and Soulier 3). {S n } n Z : points of a stationary renewal process: birth dates; {U n }: i.i.d. independant of {S n }: rewards or pulses or shocks; {η n }: i.i.d. non-negative: durations; g function. When is it defined? X t = n Z U n g({t S n }/η n ). Other examples in Kulik and Szekli (). 3 3

9 Simplification: g(t) = [,) (t). Rules out the shot noise processes of Giraitis et al. (993), Klüppelberg and Kühn (4). Model now X t = n Z U n [Sn,S n+η n) (t). The shock U k created at time S k survives for a duration η k. The process is well defined iff only a finite number of rewards survive at time t. Sufficient condition: E[η] <. The process is then strictly stationary and weakly stationary if Stationary renewal process {τ k } k i.i.d. sequence of nonnegative r.v. with distribution function F and finite mean λ = ( F (s))ds. τ independent of {τ k } k with distribution function t F (t) = P(τ t) = λ ( F (s))ds Let τ be independent of {τ j } j distributed as τ ; {τ j } j< be independent of {τ j } j, distributed as { τ j } j. Point process on the line:..s < S < S < S <..., S k = k j= τ j, k, S k = k j= τ j, k <, N(s, t] = k (s,t] (S k ). E[U ] <. 3 N is translation invariant. EN(s, t] = λ(t s). 33 Duration driven long memory processes Exemples of stationary renewal processes. Deterministic renewal process: S n = µn. Poisson process: F (t) = F (t) = e λt. Renewal process with heavy tailed inter-arrival distribution: F (t) = L(t)( + t) α, F (t) = L(t)( + t) α, < α <. Renewal-reward: τ n+ = η n. Exactly one random shock at a time. ON-OFF: U n, τ n+ = η n + ζ n. Succession of ones and zeros, at most one shock at a time. Infinite source Poisson: {S n } are the points of a homogeneous Poisson process. U n is the transmission rate: constant or random. Error duration: S n = n

10 Autocovariance function.6 Centered shocks. cov(x, X t ) = λ E[U] E[(η t) + ]. If η is regularly varying: P(η > t) = l(t)t β, then: cov(x, X t ) = l(t)t β. Long memory with Hurst index: H = (3 β)/ Sample autocorelation of Error Duration process, H = Asymptotic properties of duration driven long memory processes Centered shocks. [nt] ˆl(n)n /β X k k= fi. di. Λ β (t). β-stable Lévy process (with independent increments). Error duration, β =. ARFIMA(,.4,) H =.9 H =.9 But: n H n k= X k P, because var ( nk= X k ) = L(t) n 3 β = L(t)n H and H = (3 β)/ > /β. cumulative sum cumulative sum 38 39

11 cumsum(d4ns * d4shock) Modelling long memory in volatility: fractional differencing or duration driven long memory? stable Levy process FBM H = cumulative sum of log-squared returns US$-DM Levy or Gauss? cumulative sums: returns and log-squared returns US$/DM FBM H =.9, Levy. stable process. 4 Other problems Non centered shocks: tails of interarrival distribution vs. tails of durations. Convergence in D? Proved for the infinite source Poisson (Resnick and Van den Berg, ) Convergence of empirical process? Finite dimensional convergence obvious for the ON-OFF process. Sample autocorrelations: ˆγ n (p) = n n p k= X kx k+ p. Centered shocks: n /β {ˆγ n (p) γ(p)} converges weakly to a β- stable limit. 4 V. Estimation of long memory Working definition : {X t } is a long memory process if it is weakly stationary with spectral density f regularly varying at : L is a slowly varying function. f(x) = x H L(x), For statistical purpose, it is equivalent and more convenient to write: f(x) = sin(x/) H L(x), 43

12 Given this definition, estimating long memory means estimating the Hurst index H. Frequency domain methods Two types of methods : time domain and frequency domain. Time domain methods: R/S, Agregated variance, DFA... The discrete Fourier transform and the periodogram Drawbacks: asymptotic properties not always established, rates of convergence are often bad and asymptotic variance depends on H. x k = kπ/n, d k = πn nt= X t e itx k, I k = d k, k < n/ The periodogram is an asymptotically unbiased but inconsistent estimator of the spectral density f of a weakly stationary process. I k = n ˆγ n (p)e itx k πn p= n ˆγ n (p) = n n p k= X kx k+ p : empirical autocovariance, estimator of the p-th Fourier coefficient of f. 46 Heuristic approximation : the random variables ξ k := are i.i.d. exponentially distributed. I k f(x k ), k < n/. Always wrong (except for Gaussian white noise), even asymptotically, but useful: the conclusions drawn from this wrong approximation can be justified theoretically in some cases. est-ce clair? à chaque fois que je le dis, on me pose des questions qui montrent que ça ne l est pas. 47

13 More precisely, in the case Gaussian or linear processes, for any fixed integer u and sequences of integers k,..., k u, the vector ) converges weakly to ( dk f(x k ),..., f(x k ) a vector of u independent standard complex Gaussian random variables in the weakly dependent case; a vector of u dependent complex Gaussian random variables with dependent complex and imaginary parts in the long memory case. But this dependency is concentrated on low frequencies, and vanishes for increasing frequencies. d k 48 The GPH estimator Geweke Porter-Hudak (983) Assume that the spectral density can be expanded as f(x) = c sin(x/) H { + o()}. true for ARFIMA(p, d, q) and FGN. Write then: log(i k ) = log(f(x k )) + log(ξ k ) = log(c) + ( H) log( sin(x k /) ) + log(ξ k ) + bias. 49 The GPH estimator Ĥ GP H is the OLS estimator based on m frequencies: m Ĥ GP H = ν k log(i k ) k= m π = H + ν k log(ξ k ) + bias H + N (, ) + bias. 4m k= m {log( sin(xk/) ) m log( sin(xj/) )} j= ν k = m m {log( l= sin(xl/) ) m log( sin(xj/) )}. j= Rate of convergence? The choice of m is linked to the bias term and depends on the (unknown) regularity of x H f(x) around zero. If f is second order regularly varying: f(x) = c sin(x/) H { + O(x β ) }. Then: bias = O((m/n) β ). Optimal choice : m = n β/β+, Optimal rate of convergence : n β/β+. Main problem : choice of m. If m and m/n, Ĥ GP H is consistent. 5 A rigorous theory is established in Robinson (995), Giraitis, Robinson et Samarov (997,) for Gaussian processes, Velasco (999) for linear processes. Unfortunately, GPH (983) suggested the rule of thumb m = n. This is not correct but still often used. 5

14 H H H H H ARFIMA(,d,)+noise. Fundamental examples Spectral density: ARFIMA(,d,). Spectral density: f(x) = σ π sin(x/) H. No bias term: parametric rate of convergence! f(x) = σ π sin(x/) H + σ N π = σ { sin(x/) H + O(x H ) }. π The bias term depends on H, the best possible rate of convergence of the GPH estimator is n (H )/H. Moreover Ĥ GP H is negatively biased. Deo and Hurvich () estimateur GPH : rendements absolus.9 estimateur GPH : log rendements.4 estimateur GPH : ARFIMA(,.4,).5 estimateur GPH : ARFIMA(,.4,) + bruit blanc m m m m GPH estimator: left ARFIMA(,.4,) right ARFIMA(,.4,) + white noise H = m 55 GPH estimator: left returns US$/DM, H = /; middle absolute returns, H =? right log squarred returns, H =? If the LMSV (or FIEGARCH) model is correct for modelling the volatility, is there a way to estimate H more efficiently? 56

15 Estimation of the Hurst coefficient of the LMSV process Hurvich, Moulines and Soulier (-4) R t = e Y t/ ɛ t, {ɛ t } i.i.d. sequence, independent of the long memory process {Y t } with spectral density f Y (x) = c sin(x/) H ( + O(x β )). Log-squared returns: η t = log(ɛ t ) c, c = E[log(ɛ t )]. X t = log(r t ) = Y t + η t + c, The spectral density of {X t } is: f X (x) = f Y (x) + σ η π C sin(x/) H { + τ sin(x/) H + O(x β ) }. Assumption : β > H. The rate of convergence of the GPH estimator is n (H )/4H. Very bad if H is close to /. Is it possible to improve this rate of convergence? Signal + noise model Yes: refine the heuristic approximation. Denote θ = (C, H, τ), f(θ, x) = Cx H { + τx H } and ξ k = I k f(θ, x). Then {ξ k, k m} can be considered as i.i.d. random variables with standard exponential distribution, if m = o(n β/β+ ). Pseudo-maximum likelihood (Whittle) estimator: m { ˆθ = arg min log(f(θ, x k )) + I } k. θ Θ f(θ, x k= k ) If m = o(n β/(β+) ) and n (4H )/4H = o(m), Then m / (Ĥ H) is asymptotically normal with variance H (H ). The rate of convergence is optimal, but the asymptotic variance tends to infinity as H tends to /. Identifiability problem as H tends to /. Modification of the local Whittle estimator (Künsch 986, Robinson 995). 59 6

16 d.5.45 Estimation of H = d + / =.9. ARFIMA(,.4,) + noise. signal/noise ratio =. blue : GPH without correction red : Local Whittle with noise correction m = [n.5 ] m = [n.6 ] m = [n.7 ] m = [n.8 ] Ĥ Ĥ GP H GPH and modified Whittle estimators for US$/DM exchange rate, n = Quoted from Hurvich and Ray (3) Bandwidth 6 6 Asymptotic properties of the Discrete Fourier transforms of duration driven long memory processes. X t = n Z U n [Sn,S n+η n) (t). {S n } n Z : points of a stationary renewal process; {U n }: i.i.d. zero mean, finite variance, independant of {S n } {η n }: i.i.d. non-negative durations; with regularly varying tails: P(η > t) = l(t)t β. Then {X t } is stationary, has long memory with Hurst index H = (3 β)/, its spectral density is regularly varying: Discrete Fourier transform: d k = n X t e itx k πn t= What is the behaviour of the renormalized DFT? f(x) = l(x)x H

17 Conclusion Non linear long memory processes form a rich class of processes, whose properties are not fully known. The statistical procedures to detect long memory seem to be robust, but their theoretical properties are far from well established. Practionneers are often not aware of this and apply the GPH estimator blindedly. 68 log sin(xj/) Ave log Periodogram Taqqu-Levy Process; d=. Ave d^gph=.977 log sin(xj/) Ave log Periodogram Taqqu-Levy Process; d=.4 Ave d^gph=.3994 log sin(xj/) Ave log Periodogram Parke Process; d=. Ave d^gph=.978 log sin(xj/) Ave log Periodogram Parke Process; d=.4 Ave d^gph =.3986 Scatterplots of Log Periodogram vs. log sin(x j/) ; j=,,...,4999; H =.6 (left) and H =.9 (right). 67 j= ( p=. ) Quantiles of Standard Normal normed Aj j= ( p=.) Quantiles of Standard Normal normed Aj j=n^. ( p=.4) Quantiles of Standard Normal normed Aj j=n^.4 ( p=.47 ) Quantiles of Standard Normal normed Aj j=n^.6 ( p=.544 ) Quantiles of Standard Normal normed Aj j=n^.8 ( p=.67 ) Quantiles of Standard Normal normed Aj j=n/- ( p=.4 ) Quantiles of Standard Normal normed Aj j=n/- ( p=.736 ) Quantiles of Standard Normal normed Aj QQ Plots of the Normalized Fourier Cosine Coefficients Re(d j)/ f(xj) for the Error duration process; n=, β =., H=.4 66 Two regimes: Low frequencies: for any fixed integers, u, k,..., k u n / /β (d k,..., d ku ) weakly converges to a β-stable vector. High frequencies: if k is a sequence of integers such that k/n and k/n /β, then d k / f(x k ) converges to a standard complex Gaussian random variable. 65