Chapter 3 - Temporal processes

Transcription

1 STK Intro 1 Chapter 3 - Temporal processes Odd Kolbjørnsen and Geir Storvik January

2 STK Intro 2 Temporal processes Data collected over time Past, present, future, change Temporal aspect important? Two (separate) approaches to modelling Statistical Variability through randomness Learning dynamic structuce from data Dynamic system theory Typically deterministic

3 Hierarchical (statistical) models Data model Process model In time series setting - called state space models Example Y 1 N(µ 1, σ 2 1) Y t =αy t 1 + W t, t = 2, 3,... Z t =βy t + η t, t = 1, 2, 3,... W t ind (0, σ 2 W ) η t ind (0, σ 2 η)

4 STK Intro 4 Characterization of temporal processes Denoted Y( ) or {Y(r) : r D t } r time index Y(r) possibly multivariate, deterministic or stochastic D t R 1 D t specify type of process Continuous time process: D t (, ) or [0, ): Y(t) Discrete time process: D t N or N + : Y t Point process: Random times Example: Time of tornado. Mark: Severity of tornado Continuous time: Measured discrete Modelled discrete? Dynamic system theory: Often modelled continuous (PDE s)

5 Temporal processes, Discrete values and time STK Intro 5

6 TK Intro 6 Joint and conditional distributions Time series: {Y t : t = 0,..., T }. Joint distribution [Y 0, Y 1,..., Y T ] = [Y T Y T 1,..., Y 0 ][Y T 1 Y T 2,..., Y 0 ]... [Y 1 Y 0 ][Y 0 ] Markov assumption imply [Y t Y t 1,..., Y 0 ] = [Y t Y t 1 ] [Y 0, Y 1,..., Y T ] = [Y 0 ] Dramatic simplification! Possible extension: T [Y t Y t 1 ] t=1 [Y t Y t 1,..., Y 0 ] = [Y t Y t 1,..., Y t k ]

7 Joint and conditional distributions STK Intro 7

8 Joint and conditional distributions STK Intro 8

9 STK Intro 9 Deterministic and stochastic modelling PDE: dy (t) dt = f (Y (t)), t 0 Completely determined by Y (0). Discrete time: Y t = M(Y t 1 ), t = 1, 2,... Completely determined by Y 0. Extension to stochastic models: Y t = M(Y t 1 ) + η t, t = 1, 2,... No clear distinction Deterministic chaotic processes can look random

10 STK Intro 10 Deterministic dynamical systems Simple difference model of order 1: α t = θ 1 α t 1. Homogeneous system Example: α 0 = 1, θ 1 = 0.7. Equilibrium = 0 Nonhomogeneous system: α t = θ 1 α t 1 + g t. Possible equilibrium point depend on {g t }. Forces external to system can be modelled as a random process. Example: g t N(0, ).

11 Backwards-shift operator B: Backwards-shift operator: Bα t = α t 1 α t =θ 1 α t 1 = θ 1 Bα t Extensions: Give α t =θ 1 α t 1 + θ 2 α t 2 = θ 1 Bα t + θ 2 B 2 α t (1 θ 1 B θ 2 B 2 )α t = 0 (1 θ 1 z θ 2 z 2 ) is the characteristic polynomial Behavior of dynamical system is described by the roots of this polynomial Roots z 1 > 1 and z 2 > 1: Equilibrium value is attracting (stable) Roots z 1 < 1 or z 2 < 1: Equilibrium value is repelling (unstable)

12 STK Intro 12 Examples Dynamical system α t = 0.64α t 2, Roots: z 1 = 1.25i, z 2 = 1.25i t = 2, 3,..., α 0 = 0, α 1 = 5 Dynamical system α t = 1.1α t 2, Roots: z 1 = 0.953, z 2 = t = 2, 3,..., α 0 = 0, α 1 = 5

13 TK Intro 13 Multivariate time series α t = (α t (1),..., α t (p α )) T 1,..., p α might be spatial locations! Generalization of AR(1) model α t =Mα t 1 = M t α 0 Note: AR(3)=Multivariate AR(1): β t β t 1 β t 2 β t =θ 1 β t 1 + θ 2 β t 2 + θ 3 β t 3 θ 1 θ 2 θ 3 β t 1 = β t α t =Mα t 1 β t 3

14 Spectral representation of linear dynamic models Assume α t = Mα t 1 = M t α 0 Linear algebra: M = WΛW 1 Λ: Diagonal matrix with eigenvalues W: Right eigenvectors M 2 =WΛW 1 WΛW 1 =WΛ 2 W 1 M t =WΛ t W 1 α t =WΛ t W 1 α 0 =WΛ t c = p α i=1 c i λ t i w i Dynamic behaviour depend on λ i s

15 Nonlinear discrete dynamical systems Example: Logistic dynamical system Population dynamics: θ 0 is carying capacity θ 1 is a growth parameter Small changes in parameters can lead to large changes in equilibrium

16 TK Intro 16 Other issues Bifurcation: Small changes in parameters give different sets of equilibrium points Chaos behaviour: Sensitivity to initial behaviour State space reconstruction: If α t = M(α t 1 ; θ), sometimes possible to reconstruct α t from lowdimensional observations of α t

17 STK Intro 17 Need for statistical models Uncertainty breeds chaos Sometimes possible to model chaos through deterministic systems, but statistical model usually easier Statistical models specify structure in behaviour through probability models Statistical models can predict, chaotic deterministic systems can not

18 TK Intro 18 Time series Assume {Y t, t = 0, 1,...}, time series (statistical dynamical model) Mean function µ t = E(Y t ), deterministic Autocovariance function C Y (t, r) = cov(y t, Y r ) = E{(Y t µ t )(Y r µ r )}, Variance function σ 2 t = var(y t ) = C Y (t, t) Autocorrelation function C Y (t, r) ρ Y (t, r) = [ 1, 1] CY (t, t)c Y (r, r) t, r D t

19 Covariance functions are nonnegative definite functions Consider V = N i=1 a iy ti = a T Y I for I = {t i } N i=1 any finite subset of D t var(v ) 0 var[v ] = N ai 2 var(y ti ) + 2 i=1 =a T C YI a 1 i<j N a i a j cov(y ti, Y tj ) where C YI is covariance matrix for Y I Need a T C YI a 0 for all a C YI specified through C Y ( ) A function satisfying this requirement is a nonnegative definite function

20 Stationarity Weak/second order stationarity require 1 E(Y t ) = µ, for all t D t 2 cov(y t, Y r ) = C Y (t r) for all t, r D t Strong stationarity For any (t 1,..., t m ) and any τ [Y t1,..., Y tm ] = [Y t1+τ,..., Y tm+τ ]

21 STK Intro 21 Estimation Assume second-order stationarity T ˆµ = 1 T Y t Sample mean t=1 T τ Ĉ(τ) = 1 T (Y t+τ ˆµ)(Y t ˆµ) Empirical autocovariance function t=1 Can divided by T τ for unbiased estimate Dividing by T gives a nonnegative definite function (τ) ˆρ(τ) =ĈY [ 1, 1] Ĉ Y (0)

22 Example: estimation mean STK Intro 22

23 Example: estimation correlation STK Intro 23

24 Basic time series {W t, t N } iid zero-mean random variables with var(w t ) = σ 2 w White noise Y t = W t C(τ) = { σω, 2 τ = 0 0, τ = ±1, ±2, Random-walk Y t = Y t 1 + W t. Not stationary Autoregressive AR(p) Y t = α 1 Y t α p Y t p + W t Moving average MA(q) Y t = W t + β 1 W t β q W t q ARMA(p,q) Y t = α 1 Y t 1 + +α p Y t p +W t +β 1 W t 1 + +β q W t q VAR Y t = M 1 Y t M p Y t p + W t

25 Parameter estimation Method-of-moments Set of linear equations for AR models Maximum likelihood Kalman filtering for calculation of likelihood Numerical optimization for maximizing likelihood

26 Nonlinear time series Typically combination of basic time series models and deterministic dynamical models Y t = M(Y t 1,..., Y t p ) + τ(y t 1,..., Y t p )W t

27 STK Intro 27 Covariance functions for AR/MA processes AR(1): C Y (τ) = α1 τ MA(1): C Y (τ) given by 1, τ = 0 C Y (τ) = σw 2 β 1, τ = 1 0, τ > 1 General AR processes: Decay exponentially MA processes: Zero outside lag

28 Hierearchichal models STK Intro 28