STK4150 - Intro 1 Chapter 3 - Temporal processes Odd Kolbjørnsen and Geir Storvik January 23 2017
STK4150 - Intro 2 Temporal processes Data collected over time Past, present, future, change Temporal aspect important? Two (separate) approaches to modelling Statistical Variability through randomness Learning dynamic structuce from data Dynamic system theory Typically deterministic
Hierarchical (statistical) models Data model Process model In time series setting - called state space models Example Y 1 N(µ 1, σ 2 1) Y t =αy t 1 + W t, t = 2, 3,... Z t =βy t + η t, t = 1, 2, 3,... W t ind (0, σ 2 W ) η t ind (0, σ 2 η)
STK4150 - Intro 4 Characterization of temporal processes Denoted Y( ) or {Y(r) : r D t } r time index Y(r) possibly multivariate, deterministic or stochastic D t R 1 D t specify type of process Continuous time process: D t (, ) or [0, ): Y(t) Discrete time process: D t N or N + : Y t Point process: Random times Example: Time of tornado. Mark: Severity of tornado Continuous time: Measured discrete Modelled discrete? Dynamic system theory: Often modelled continuous (PDE s)
Temporal processes, Discrete values and time STK4150 - Intro 5
TK4150 - Intro 6 Joint and conditional distributions Time series: {Y t : t = 0,..., T }. Joint distribution [Y 0, Y 1,..., Y T ] = [Y T Y T 1,..., Y 0 ][Y T 1 Y T 2,..., Y 0 ]... [Y 1 Y 0 ][Y 0 ] Markov assumption imply [Y t Y t 1,..., Y 0 ] = [Y t Y t 1 ] [Y 0, Y 1,..., Y T ] = [Y 0 ] Dramatic simplification! Possible extension: T [Y t Y t 1 ] t=1 [Y t Y t 1,..., Y 0 ] = [Y t Y t 1,..., Y t k ]
Joint and conditional distributions STK4150 - Intro 7
Joint and conditional distributions STK4150 - Intro 8
STK4150 - Intro 9 Deterministic and stochastic modelling PDE: dy (t) dt = f (Y (t)), t 0 Completely determined by Y (0). Discrete time: Y t = M(Y t 1 ), t = 1, 2,... Completely determined by Y 0. Extension to stochastic models: Y t = M(Y t 1 ) + η t, t = 1, 2,... No clear distinction Deterministic chaotic processes can look random
STK4150 - Intro 10 Deterministic dynamical systems Simple difference model of order 1: α t = θ 1 α t 1. Homogeneous system Example: α 0 = 1, θ 1 = 0.7. Equilibrium = 0 Nonhomogeneous system: α t = θ 1 α t 1 + g t. Possible equilibrium point depend on {g t }. Forces external to system can be modelled as a random process. Example: g t N(0, 0.2 2 ).
Backwards-shift operator B: Backwards-shift operator: Bα t = α t 1 α t =θ 1 α t 1 = θ 1 Bα t Extensions: Give α t =θ 1 α t 1 + θ 2 α t 2 = θ 1 Bα t + θ 2 B 2 α t (1 θ 1 B θ 2 B 2 )α t = 0 (1 θ 1 z θ 2 z 2 ) is the characteristic polynomial Behavior of dynamical system is described by the roots of this polynomial Roots z 1 > 1 and z 2 > 1: Equilibrium value is attracting (stable) Roots z 1 < 1 or z 2 < 1: Equilibrium value is repelling (unstable)
STK4150 - Intro 12 Examples Dynamical system α t = 0.64α t 2, Roots: z 1 = 1.25i, z 2 = 1.25i t = 2, 3,..., α 0 = 0, α 1 = 5 Dynamical system α t = 1.1α t 2, Roots: z 1 = 0.953, z 2 = 0.953 t = 2, 3,..., α 0 = 0, α 1 = 5
TK4150 - Intro 13 Multivariate time series α t = (α t (1),..., α t (p α )) T 1,..., p α might be spatial locations! Generalization of AR(1) model α t =Mα t 1 = M t α 0 Note: AR(3)=Multivariate AR(1): β t β t 1 β t 2 β t =θ 1 β t 1 + θ 2 β t 2 + θ 3 β t 3 θ 1 θ 2 θ 3 β t 1 = 1 0 0 β t 2 0 1 0 α t =Mα t 1 β t 3
Spectral representation of linear dynamic models Assume α t = Mα t 1 = M t α 0 Linear algebra: M = WΛW 1 Λ: Diagonal matrix with eigenvalues W: Right eigenvectors M 2 =WΛW 1 WΛW 1 =WΛ 2 W 1 M t =WΛ t W 1 α t =WΛ t W 1 α 0 =WΛ t c = p α i=1 c i λ t i w i Dynamic behaviour depend on λ i s
Nonlinear discrete dynamical systems Example: Logistic dynamical system Population dynamics: θ 0 is carying capacity θ 1 is a growth parameter Small changes in parameters can lead to large changes in equilibrium
TK4150 - Intro 16 Other issues Bifurcation: Small changes in parameters give different sets of equilibrium points Chaos behaviour: Sensitivity to initial behaviour State space reconstruction: If α t = M(α t 1 ; θ), sometimes possible to reconstruct α t from lowdimensional observations of α t
STK4150 - Intro 17 Need for statistical models Uncertainty breeds chaos Sometimes possible to model chaos through deterministic systems, but statistical model usually easier Statistical models specify structure in behaviour through probability models Statistical models can predict, chaotic deterministic systems can not
TK4150 - Intro 18 Time series Assume {Y t, t = 0, 1,...}, time series (statistical dynamical model) Mean function µ t = E(Y t ), deterministic Autocovariance function C Y (t, r) = cov(y t, Y r ) = E{(Y t µ t )(Y r µ r )}, Variance function σ 2 t = var(y t ) = C Y (t, t) Autocorrelation function C Y (t, r) ρ Y (t, r) = [ 1, 1] CY (t, t)c Y (r, r) t, r D t
Covariance functions are nonnegative definite functions Consider V = N i=1 a iy ti = a T Y I for I = {t i } N i=1 any finite subset of D t var(v ) 0 var[v ] = N ai 2 var(y ti ) + 2 i=1 =a T C YI a 1 i<j N a i a j cov(y ti, Y tj ) where C YI is covariance matrix for Y I Need a T C YI a 0 for all a C YI specified through C Y ( ) A function satisfying this requirement is a nonnegative definite function
Stationarity Weak/second order stationarity require 1 E(Y t ) = µ, for all t D t 2 cov(y t, Y r ) = C Y (t r) for all t, r D t Strong stationarity For any (t 1,..., t m ) and any τ [Y t1,..., Y tm ] = [Y t1+τ,..., Y tm+τ ]
STK4150 - Intro 21 Estimation Assume second-order stationarity T ˆµ = 1 T Y t Sample mean t=1 T τ Ĉ(τ) = 1 T (Y t+τ ˆµ)(Y t ˆµ) Empirical autocovariance function t=1 Can divided by T τ for unbiased estimate Dividing by T gives a nonnegative definite function (τ) ˆρ(τ) =ĈY [ 1, 1] Ĉ Y (0)
Example: estimation mean STK4150 - Intro 22
Example: estimation correlation STK4150 - Intro 23
Basic time series {W t, t N } iid zero-mean random variables with var(w t ) = σ 2 w White noise Y t = W t C(τ) = { σω, 2 τ = 0 0, τ = ±1, ±2, Random-walk Y t = Y t 1 + W t. Not stationary Autoregressive AR(p) Y t = α 1 Y t 1 + + α p Y t p + W t Moving average MA(q) Y t = W t + β 1 W t 1 + + β q W t q ARMA(p,q) Y t = α 1 Y t 1 + +α p Y t p +W t +β 1 W t 1 + +β q W t q VAR Y t = M 1 Y t 1 + + M p Y t p + W t
Parameter estimation Method-of-moments Set of linear equations for AR models Maximum likelihood Kalman filtering for calculation of likelihood Numerical optimization for maximizing likelihood
Nonlinear time series Typically combination of basic time series models and deterministic dynamical models Y t = M(Y t 1,..., Y t p ) + τ(y t 1,..., Y t p )W t
STK4150 - Intro 27 Covariance functions for AR/MA processes AR(1): C Y (τ) = α1 τ MA(1): C Y (τ) given by 1, τ = 0 C Y (τ) = σw 2 β 1, τ = 1 0, τ > 1 General AR processes: Decay exponentially MA processes: Zero outside lag
Hierearchichal models STK4150 - Intro 28