Long-Range Dependence and Self-Similarity c Vladas Pipiras and Murad S. Taqqu January 24, 2016
Contents Contents 2 Preface 8 List of abbreviations 10 Notation 11 1 A brief overview of times series and stochastic processes 13 1.1 Stochastic processes and time series.......................... 13 1.1.1 Gaussian stochastic processes.......................... 14 1.1.2 Stationarity (of increments)........................... 15 1.1.3 Weak or second-order stationarity (of increments).............. 15 1.2 Time domain perspective................................ 16 1.2.1 Representations in the time domain...................... 16 1.3 Spectral domain perspective............................... 18 1.3.1 Spectral density................................. 18 1.3.2 Linear filtering.................................. 20 1.3.3 Periodogram................................... 20 1.3.4 Spectral representation............................. 21 1.4 Integral representations heuristics............................ 23 1.4.1 Representations of a Gaussian continuous-time process............ 24 2 Basics of long-range dependence and self-similarity 26 2.1 Definitions of long-range dependent series....................... 26 2.2 Relations between the various definitions of long-range dependence......... 29 2.2.1 Some useful properties of slowly and regularly varying functions...... 29 2.2.2 Comparing conditions II and III........................ 31 2.2.3 Comparing conditions II and V......................... 31 2.2.4 Comparing conditions I and II......................... 32 2.2.5 Comparing conditions II and IV........................ 34 2.2.6 Comparing conditions I and IV......................... 36 2.2.7 Comparing conditions IV and III........................ 36 2.2.8 Comparing conditions IV and V........................ 37 2.3 Short-range dependent series and their several examples............... 38 2.4 Examples of long-range dependent series: FARIMA models............. 42 2.4.1 FARIMA(0,d,0) series.............................. 42 2.4.2 FARIMA(p, d, q) series.............................. 48 2
2.5 Definition and basic properties of self-similar processes................ 49 2.6 Examples of self-similar processes............................ 52 2.6.1 Fractional Brownian motion........................... 52 2.6.2 Bifractional Brownian motion.......................... 59 2.6.3 The Rosenblatt process............................. 61 2.6.4 SαS Lévy motion................................ 63 2.6.5 Linear fractional stable motion......................... 64 2.6.6 Log-fractional stable motion........................... 65 2.6.7 The Telecom process............................... 66 2.6.8 Linear fractional Lévy motion.......................... 67 2.7 The Lamperti transformation.............................. 69 2.8 Connections between long-range dependent series and self-similar processes.... 70 2.9 Long- and short-range dependent series with infinite variance............ 79 2.9.1 First definition of LRD under heavy tails: condition A............ 79 2.9.2 Second definition of LRD under heavy tails: condition B........... 84 2.9.3 Third definition of LRD under heavy tails: codifference........... 84 2.10 Heuristic methods of estimation............................. 86 2.10.1 The R/S method................................. 86 2.10.2 Aggregated variance method.......................... 90 2.10.3 Regression in the spectral domain....................... 90 2.10.4 Wavelet-based estimation............................ 95 2.11 Generation of Gaussian long- and short-range dependent series........... 101 2.11.1 Using Cholesky decomposition......................... 101 2.11.2 Using circulant matrix embedding....................... 101 2.12 Exercises......................................... 107 3 Physical models for long-range dependence and self-similarity 109 3.1 Aggregation of short-range dependent series...................... 109 3.2 Mixture of correlated random walks.......................... 113 3.3 Infinite source Poisson model with heavy tails..................... 115 3.3.1 Model formulation................................ 115 3.3.2 Workload process and its basic properties................... 118 3.3.3 Input rate regimes................................ 123 3.3.4 Limiting behavior of the scaled workload process............... 125 3.4 Power-law shot noise model............................... 141 3.5 Hierarchical model.................................... 144 3.6 Regime switching..................................... 147 3.7 Elastic collision of particles............................... 152 3.8 Motion of a tagged particle in a simple symmetric exclusion model......... 157 3.9 Power-law Pólya s urn.................................. 161 3.10 Random walk in random scenery............................ 166 3.11 Two-dimensional Ising model.............................. 169 3.11.1 Model formulation and result.......................... 169 3.11.2 Correlations, dimers and Pfaffians....................... 172 3.11.3 Computation of the inverse........................... 185 3.11.4 The strong Szegö limit theorem......................... 195 3.11.5 Long-range dependence at critical temperature................ 197 3.12 Stochastic heat equation................................. 200 3
3.13 The Weierstrass function connection.......................... 201 3.14 Exercises......................................... 206 4 Hermite processes 208 4.1 Hermite polynomials and multiple stochastic integrals................ 208 4.2 Integral representations of Hermite processes..................... 210 4.2.1 Integral representation in the time domain................... 210 4.2.2 Integral representation in the spectral domain................. 212 4.2.3 Integral representation on an interval...................... 213 4.2.4 Summary..................................... 217 4.3 Moments, cumulants and diagram formulae for multiple integrals.......... 219 4.3.1 Diagram formulae................................ 219 4.3.2 Multigraphs.................................... 222 4.3.3 Relation between diagrams and multigraphs.................. 223 4.3.4 Diagram and multigraph formulae for Hermite polynomials......... 227 4.4 Moments and cumulants of Hermite processes..................... 230 4.5 Multiple integrals of order two............................. 234 4.6 The Rosenblatt process................................. 237 4.7 The Rosenblatt distribution............................... 239 4.8 Generalized Hermite and related processes....................... 246 4.9 Exercises......................................... 251 5 Non-central and central limit theorems 252 5.1 Non-linear functions of Gaussian random variables.................. 252 5.2 Hermite rank....................................... 255 5.3 Non-central limit theorem................................ 256 5.4 Central limit theorem.................................. 266 5.5 Limit theorems in the linear case............................ 273 5.5.1 Direct approach for entire functions...................... 273 5.5.2 Approach based on martingale differences................... 277 5.6 Multivariate limit theorems............................... 281 5.6.1 The SRD case.................................. 281 5.6.2 The LRD case.................................. 284 5.6.3 The mixed case.................................. 286 5.6.4 Multivariate limits of multilinear processes.................. 289 5.7 Generation of non-gaussian long- and short-range dependent series......... 292 5.7.1 Matching a marginal distribution........................ 293 5.7.2 Relationship between autocorrelations..................... 295 5.7.3 Price theorem................................... 297 5.7.4 Matching a targeted autocovariance for series with prescribed marginal.. 299 5.8 Exercises......................................... 303 6 Fractional calculus and fractional Wiener integrals 305 6.1 Fractional integrals and derivatives........................... 305 6.1.1 Fractional integrals on an interval....................... 305 6.1.2 Riemann-Liouville fractional derivatives D on an interval.......... 308 6.1.3 Fractional integrals and derivatives on the real line.............. 311 6.1.4 Marchaud fractional derivatives D on the real line.............. 313 4
6.1.5 The Fourier transform perspective....................... 315 6.2 Representations of fractional Brownian motion.................... 317 6.2.1 Representation of FBM on an interval..................... 317 6.2.2 Representations of FBM on the real line.................... 325 6.3 Fractional Wiener integrals and their deterministic integrands............ 326 6.3.1 The Gaussian space generated by fractional Wiener integrals........ 326 6.3.2 Classes of integrands on an interval....................... 329 6.3.3 Subspaces of classes of integrands........................ 334 6.3.4 The fundamental martingale.......................... 337 6.3.5 The deconvolution formula........................... 338 6.3.6 Classes of integrands on the real line...................... 339 6.3.7 Connection to the reproducing kernel Hilbert space.............. 340 6.4 Applications........................................ 342 6.4.1 Girsanov s formula for FBM........................... 342 6.4.2 The prediction formula for FBM........................ 344 6.4.3 Elementary linear filtering involving FBM................... 347 6.5 Exercises......................................... 349 7 Stochastic integration with respect to fractional Brownian motion 350 7.1 Stochastic integration with random integrands.................... 350 7.1.1 FBM and the semimartingale property..................... 350 7.1.2 Divergence integral for FBM.......................... 352 7.1.3 Self-integration of FBM............................. 354 7.1.4 Itô s formulas................................... 359 7.2 Applications of stochastic integration.......................... 364 7.2.1 Stochastic differential equations driven by FBM................ 364 7.2.2 Regularity of laws related to FBM....................... 365 7.2.3 Numerical solutions of SDEs driven by FBM................. 369 7.2.4 Convergence to normal law using Stein s method............... 376 7.2.5 Local time of FBM................................ 380 7.3 Exercises......................................... 383 8 Series representations of FBM 385 8.1 Karhunen-Loève decomposition and FBM....................... 385 8.1.1 The case of general stochastic processes.................... 385 8.1.2 The cases of BM and FBM........................... 386 8.2 Wavelet expansion of FBM............................... 387 8.2.1 Orthogonal wavelet bases............................ 388 8.2.2 Fractional wavelets................................ 392 8.2.3 Fractional conjugate mirror filters....................... 396 8.2.4 Wavelet-based expansion and simulation of FBM............... 399 8.3 Paley-Wiener representation of FBM.......................... 401 8.3.1 Complex-valued FBM and its representations................. 402 8.3.2 Space L a and its orthonormal basis....................... 403 8.3.3 Expansion of FBM................................ 408 8.4 Exercises......................................... 409 5
9 Multidimensional models 411 9.1 Fundamentals of multidimensional models....................... 412 9.1.1 Basics of matrix analysis............................. 412 9.1.2 Vector setting................................... 413 9.1.3 Spatial setting.................................. 415 9.2 Operator self-similarity.................................. 416 9.3 Operator fractional Brownian motions......................... 418 9.3.1 Integral representations............................. 419 9.3.2 Time reversible OFBMs............................. 425 9.3.3 Vector fractional Brownian motions...................... 427 9.3.4 Identifiability questions............................. 432 9.4 Vector long-range dependence.............................. 434 9.4.1 Definitions and basic properties......................... 435 9.4.2 Vector FARIMA(0,D,0) series......................... 438 9.4.3 Vector FGN series................................ 441 9.4.4 Fractional cointegration............................. 442 9.5 Operator fractional Brownian fields........................... 446 9.5.1 M homogeneous functions........................... 447 9.5.2 Integral representations............................. 449 9.5.3 Special subclasses and examples of OFBFs.................. 458 9.6 Spatial long-range dependence.............................. 460 9.6.1 Definitions and basic properties......................... 461 9.6.2 Examples..................................... 463 9.7 Exercises......................................... 465 10 Maximum likelihood estimation methods 467 10.1 Exact Gaussian MLE in the time domain....................... 467 10.2 Approximate MLE.................................... 470 10.2.1 Whittle estimation in the spectral domain................... 470 10.2.2 Autoregressive approximation.......................... 477 10.3 Model selection and diagnostics............................. 478 10.4 Forecasting........................................ 480 10.5 R packages and case studies............................... 481 10.5.1 The ARFIMA package.............................. 481 10.5.2 The FRACDIFF package............................ 483 10.6 Local Whittle estimation................................. 484 10.6.1 Local Whittle estimator............................. 485 10.6.2 Bandwidth selection............................... 489 10.6.3 Bias reduction and rate optimality....................... 491 10.7 Broadband Whittle approach.............................. 492 10.8 Exercises......................................... 494 11 Historical notes and extensions 496 11.1 Notes to Chapter 1.................................... 496 11.2 Notes to Chapter 2.................................... 497 11.3 Notes to Chapter 3.................................... 499 11.4 Notes to Chapter 4.................................... 502 11.5 Notes to Chapter 5.................................... 503 6
11.6 Notes to Chapter 6.................................... 505 11.7 Notes to Chapter 7.................................... 506 11.8 Notes to Chapter 8.................................... 507 11.9 Notes to Chapter 9.................................... 507 11.10Notes to Chapter 10................................... 509 11.11Other topics........................................ 512 A Auxiliary notions and results 515 A.1 Fourier series and Fourier transforms.......................... 515 A.1.1 Fourier series and Fourier transform for sequences.............. 515 A.1.2 Fourier transform for functions......................... 516 A.2 Fourier series of regularly varying sequences...................... 517 A.3 Weak and vague convergence of measures....................... 521 A.3.1 The case of probability measures........................ 521 A.3.2 The case of locally finite measures....................... 522 A.4 Stable and heavy-tailed random variables and series................. 523 B Integrals with respect to random measures 525 B.1 Single integrals with respect to random measures................... 525 B.1.1 Integrals with respect to random measures with orthogonal increments... 526 B.1.2 Integrals with respect to Gaussian measures.................. 527 B.1.3 Integrals with respect to stable measures................... 529 B.1.4 Integrals with respect to Poisson measures................... 530 B.1.5 Integrals with respect to Lévy measures.................... 531 B.2 Multiple integrals with respect to Gaussian measures................. 532 C Basics of Malliavin calculus 537 C.1 Isonormal Gaussian processes.............................. 537 C.2 Derivative operator.................................... 537 C.3 Divergence integral.................................... 541 C.4 Generator of the Ornstein-Uhlenbeck semigroup.................... 543 Bibliography 544 7