Statistics of Stochastic Processes

Transcription

1 Prof. Dr. J. Franke All of Statistics 4.1 Statistics of Stochastic Processes discrete time: sequence of r.v...., X 1, X 0, X 1, X 2,... X t R d in general. Here: d = 1. continuous time: random function X(t), a t b ( a < b ) spatial processes (e.g. image analysis, d = 2, 3): X z, z Z d (discrete) X(z), z R d (continuous)

2 Prof. Dr. J. Franke All of Statistics 4.2 Stationarity (spatial homogeneity) L(X t1,..., X tn ) = L(X t1 +s,..., X tn +s) Analogously for t j, s R, z j, s Z d, z j, s R d for all n 1, t 1,..., t n, s Z Note: Distribution of X(t), a t b, as a random function, e.g. as a random variable with values in C[a, b], completely determined by the finite-dimensional distributions L(X(t 1 ),..., X(t n )) n 1, t 1,..., t n [a, b] Gaussian process if L(X(t 1 ),..., X(t n )) n-dimensional normal Examples: a) autoregressive process of order 1 or AR(1)-process X t = ax t 1 + Z t, EZ t = 0, var Z t = σ 2 < Z t innovation at time t; uncorrelated or even i.i.d.

3 Prof. Dr. J. Franke All of Statistics 4.3 (X t µ) = a(x t 1 µ) + Z t, EZ t = 0, var Z t = σ 2 < if EX t = µ 0. Gaussian if innovations Z t i.i.d N (0, σ 2 ). Stationary a < 1. For a = 1: random walk X t = X 0 + t j=1 Z j, var X t = var X 0 + tσ 2 distribution of X t depends on t not stationary b) stationary AR(1)-process solves stochastic difference equation: with b > 0. X t = X t X t 1 = (a 1)X t + Z t = bx t + Z t

4 Prof. Dr. J. Franke All of Statistics 4.4 AR(1): X t = 0.9X t 1 + Z t

5 Prof. Dr. J. Franke All of Statistics 4.5 Gaussian random walks with EX t = 0 for all t

6 Prof. Dr. J. Franke All of Statistics 4.6 X t = X t X t 1 = bx t + Z t, b > 0 time discretization: t 1 t t, t 0 stochastic differential equation (Ornstein-Uhlenbeck process) dx(t) = bx(t) + σdw (t), b > 0 W (t) standard Wiener process or Brownian motion, e.g. on [0, ) with W (0) = 0: i) process with independent increments: W (t) W (s) independent of W (s), 0 s < t ii) Gaussian: increment W (t) W (s) is N (0, t s)-distributed Wiener process = continuous-time analogue to and limit of Gaussian random walk (used for simulation)

7 Prof. Dr. J. Franke All of Statistics 4.7 c) nonlinear autoregressive process of order p or NLAR(p)-process: X t = m(x t 1,..., X t p ) + Z t, EZ t = 0, var Z t = σ 2 < innovations Z t usually i.i.d. Key property for statistics: 1 realization X 1,..., X N of part of a stationary process suffices for statistical inference (estimation, testing,...), e.g. (under weak assumptions) ergodic theorem implies: X N = 1 N N t=1 X t µ = EX s (independent of s) Extension of inference to local stationarity or piecewise stationarity possible

8 Prof. Dr. J. Franke All of Statistics 4.8 NLAR(2): X t = ψ(x t 1 + X t 2 ) + Z t, ψ = logistic function

9 Prof. Dr. J. Franke All of Statistics 4.9 Markov property: distributions of future X s, s > t given past and present only depend on current value X t Examples: AR(1), Ornstein-Uhlenbeck, NLAR(1) Markov random fields (spatial): distributions of X z beyond a boundary only depend on data on the boundary Statistical problems: Estimation, e.g. for parametric Gaussian NLAR(p) by leastsquares as in regression: N t=p+1 ( X t m(x t 1,..., X t p, b)) 2 = min b! Forecasting: Fit appropriate autoregressive scheme to the data (e.g. NLAR(p)), e.g. estimate b by ˆb, and predict X N+1 by X N+1 = m(x N,..., X N+1 p,ˆb)

10 Prof. Dr. J. Franke All of Statistics 4.10 AR(1) with changepoint at 151: mean shift of +2

11 Prof. Dr. J. Franke All of Statistics 4.11 AR(1) with changepoint at 151: start of trend with slope 0.05

12 Prof. Dr. J. Franke All of Statistics 4.12 AR(1) with changepoint at 151: change of dependence from a = 0.9 to a = 0.5

13 Prof. Dr. J. Franke All of Statistics 4.13 Spectral Analysis of Stationary Processes Here only stationary time series:..., X 1, X 0, X 1, X 2,... Analogously: spatial processes (stationary spatially homogeneous) Extension to nonstationary data: wavelets Goal: Detect and interpret periodic features in the data structure Use those periodic features to differ, e.g. by hypothesis tests, between or to classify different kinds of objects

14 Prof. Dr. J. Franke All of Statistics 4.14 Building blocks of spectral decomposition: sines and cosines

15 Prof. Dr. J. Franke All of Statistics 4.15 period/wavelength l, frequency ν = 1 l, angular frequency ω = 2πν, phase (shift) φ, amplitude A

16 Prof. Dr. J. Franke All of Statistics 4.16 Deterministic signal: How to reconstruct the cosine-components? Fourier analysis

17 Prof. Dr. J. Franke All of Statistics 4.17 Random signals and spectral representation X 1,..., X N sample from a random time series..., X 1, X 0, X 1, X 2,... Assumption (weak stationarity of signal): var X t < and cov(x t, X s ) = r t s i.e. linear dependence between X t, X s depends on time span t s between the observations only. Strict stationarity requires that the whole data generating mechanism does not change with time: (X t1 +s,..., X tn +s) has same distribution as (X t1,..., X tn ) for all n 1, t 1,..., t n, s

18 Prof. Dr. J. Franke All of Statistics 4.18 Terminology: r t autocovariance at lag t f(ω) power spectral density at (angular) frequency ω r t = cov(x t+s, X s ), f(ω) = t= r t e itω, π ω π if r t <. Remark: For real-valued signals: f(ω) = f( ω) 0. Both are equivalent characterizations of the stationary signal X t Example: White noise X t, X s uncorrelated for t s r t = 0 for all t 0 f(ω) r 0 = constant for all ω

19 Prof. Dr. J. Franke All of Statistics 4.19 Spectral representation theorem : Every stationary signal can be arbitrarily well approximated by a random trignometric polynomial if only m is large enough: X t m j= m Z j e iω jt = A 0 + m j=1 A j cos(ω j t + Φ j ) with random and independent A 0 R, A j 0, 0 Φ j < 2π, j = 1,..., m and Fourier frequencies ω j = 2πj M, M = 2m + 1. Cramér s Theorem: X t = 1 2π π π eiωt dz(ω) Z(ω) complex-valued stochastic processes with orthogonal increments, Z( π) = 0, e.g. Wiener process, E Z(ω + dω) Z(ω) 2 f(ω)dω

20 Prof. Dr. J. Franke All of Statistics 4.20 Two special cases: a) discrete spectrum: X t = A 0 + n l=1 A l cos(ω l t + Φ l ) with fixed, but unknown n and ωl not necessarily Fourier frequencies spectral lines at ωl, l = 1,..., n b) continuous spectrum: X t has a spectral density f(ω) which is large in frequency bands dominating the cyclic behaviour of the signal The examples on the slides , have a mixed spectrum, i.e. the signal is an additive superposition of a periodic signal and another signal (here: white noise) with continuous spectrum.

21 Prof. Dr. J. Franke All of Statistics 4.21 Fourier frequencies ω k = 2πk N. Assume: X t = m j= m Z j e iω jt, t = 1,..., N, N = 2m + 1 Discrete Fourier transform (DFT) of data: D N (ω k ) = 1 N N t=1 X t e itω k, k = m,..., m one-to-one data transformation X 1,..., X N D N (ω m ),..., D N (ω m ) fast calculation: FFT algorithm (fast Fourier transform) and its inverse (MATLAB: fft and ffti) D N (ω k ) complex, difficult to plot periodogram I N (ω k ) = D N (ω k ) 2, k = m,..., m, I N (ω k ) = Z k 2

22 Prof. Dr. J. Franke All of Statistics 4.22 I N (ω k ) = 1 N N t=1 X t e itω k 2, k = m,..., m X t real-valued I N (ω k ) = I N ( ω k ) symmetric. Periodogram for discrete and continuous spectrum, if N : a) I N (ω j ) 1 2 A2 l N if ω j ω l 0, I N(ω j ) 0 else. b) EI N (ω j ) f(ω j ), but var I N (ω j ) constant > 0 In case b), the periodogram will not converge for N, but it even becomes more and more irregular visually. For estimating the power spectral density, the periodogram has to be smoothed: f(ω k ) = L l= L w l I N (ω k+l ), w l = w l, L l= L w l = 1.

23 Prof. Dr. J. Franke All of Statistics 4.23

24 Prof. Dr. J. Franke All of Statistics 4.24

25 Prof. Dr. J. Franke All of Statistics 4.25

26 Prof. Dr. J. Franke All of Statistics 4.26

27 Prof. Dr. J. Franke All of Statistics 4.27

28 Prof. Dr. J. Franke All of Statistics 4.28

29 Prof. Dr. J. Franke All of Statistics 4.29

30 Prof. Dr. J. Franke All of Statistics 4.30

31 Prof. Dr. J. Franke All of Statistics 4.31

32 Prof. Dr. J. Franke All of Statistics 4.32

33 Prof. Dr. J. Franke All of Statistics 4.33

34 Prof. Dr. J. Franke All of Statistics 4.34

35 Prof. Dr. J. Franke All of Statistics 4.35

36 Prof. Dr. J. Franke All of Statistics 4.36

37 Prof. Dr. J. Franke All of Statistics 4.37

38 Prof. Dr. J. Franke All of Statistics 4.38 Data example: Wolfer s sunspot numbers (since 1700) The annual data represent an index for the number of large sunspots and clusters of smaller sunspots related to the activity of the sun. The data here are taken from MATLAB. a.k.a. Wolf number (A. Wolfer has been the student of R. Wolf, both active in Zürich in the 19 th century

39 Prof. Dr. J. Franke All of Statistics 4.39

40 Prof. Dr. J. Franke All of Statistics 4.40 periodogram of raw sunspot numbers X t

41 Prof. Dr. J. Franke All of Statistics 4.41 periodogram I 0 N (ω k) of centered sunspot numbers X 0 t = X t X N

42 Prof. Dr. J. Franke All of Statistics 4.42 averaged periodogram ˆf(ω k ) = 1 3 (I0 N (ω k 1) + I 0 N (ω k) + I 0 N (ω k+1))

43 Prof. Dr. J. Franke All of Statistics 4.43 averaged periodogram with weights 15 1 (1, 2, 3, 3, 3, 2, 1)

44 Prof. Dr. J. Franke All of Statistics 4.44

45 Prof. Dr. J. Franke All of Statistics 4.45

46 Prof. Dr. J. Franke All of Statistics 4.46

47 Prof. Dr. J. Franke All of Statistics 4.47

48 Prof. Dr. J. Franke All of Statistics 4.48 Data example: EEG-like data The following sample has been generated from a time series model which has been fitted to 10 seconds of real EEG data of a young child which has been in an idle state of mind during EEG recording. The simulated data correspond to approximately 1.5 sec of brain activity. For the original data and the fitted model (an autoregression of order 8), compare: H.W. Steinberg, T. Gasser, J. Franke: Fitting autoregressive models to EEG time series: An empirical comparison of estimates of the order. IEEE Trans. Acoustics, Speech and Signal Proc., 33, (1985)

49 Prof. Dr. J. Franke All of Statistics 4.49

50 Prof. Dr. J. Franke All of Statistics 4.50

51 Prof. Dr. J. Franke All of Statistics 4.51