STA 6857 Autocorrelation and Cross-Correlation & Stationary Time Series ( 1.4, 1.5)

Outline 1 Announcements 2 Autocorrelation and Cross-Correlation 3 Stationary Time Series 4 Homework 1c Arthur Berg STA 6857 Autocorrelation and Cross-Correlation & Stationary Time Series ( 1.4, 1.5) 2/ 25

Homework Our TA, Aixin Tan, will have office hours on Thursdays from 1 2pm in 218 Griffin-Floyd. Homework 1c will be assigned today and the last part of homework 1, homework 1d, will be assigned on Friday. Homework 1 will be collected on Friday, September 7. Don t wait till the last minute to do the homework, because more homework will follow next week. Arthur Berg STA 6857 Autocorrelation and Cross-Correlation & Stationary Time Series ( 1.4, 1.5) 4/ 25

Questions from Last Time We ve seen the AR(p) and MA(q) models. Which one do we use? In scientific modeling we wish to choose the model that best describes or explains the data. Later we will develop many techniques to help us choose and fit the best model for our data. Is this white noise process in the models unique? We will see later that any stationary time series can be described as a MA( ) + deterministic part by the Wold decomposition, where the white noise process in the MA( ) part is unique. So in short, the answer is yes. We will also see later how to estimate the white noise process which will aid in forecasting. Arthur Berg STA 6857 Autocorrelation and Cross-Correlation & Stationary Time Series ( 1.4, 1.5) 5/ 25

Notational Disclaimer For convenience, we will follow the textbook s style of not distinguishing the random sequence (typically denoted with capital letters in statistics) with an observation of the random sequence (typically denoted with lower-case letters in statistics). We shall use {x t } in both situations and the distinction between the random and non-random cases will be clear from the context. Arthur Berg STA 6857 Autocorrelation and Cross-Correlation & Stationary Time Series ( 1.4, 1.5) 6/ 25

Joint Distribution The joint distribution contains all of the information about the time series. There is no feasible way to estimate the joint distribution without some strict assumptions. Definition (Joint Distribution Function (Joint CDF)) Given time points t 1, t 2,..., t n, the joint CDF of x t1, x t2,..., x tn, evaluated at constants c 1,..., c n, is given by F(c 1, c 2,..., c n ) = P (x t1 c 1, x t2 c 2,..., x tn c n )) Example (CDF of w t iid N (0, 1)) where F(c 1, c 2,..., c n ) = Φ(x) = 1 2π x n Φ(c t ) t=1 e z2 /2 dz Arthur Berg STA 6857 Autocorrelation and Cross-Correlation & Stationary Time Series ( 1.4, 1.5) 8/ 25

Univariate Distribution and Density We may be also interested in the marginal distribution function at a particular time t. Knowing all marginal distributions cannot give you the full joint distribution. One dimensional distribution function: F t (x) = P (x t x) with corresponding density function (if exists) f t (x) = F t(x) x Arthur Berg STA 6857 Autocorrelation and Cross-Correlation & Stationary Time Series ( 1.4, 1.5) 9/ 25

Mean Function Definition (Mean Function) The mean function of a time series {x t } is given by (if it exists) µ t = E(x t ) = Example (Mean of an MA(q)) Let w t WN(0, σ 2 ) and x f t (x) dx x t = w t + θ 1 w t 1 + θ 2 w t 2 + + θ q w t q then µ t = E(x t ) = E(w t ) + θ 1 E(w t 1 ) + θ 2 E(w t 2 ) + + θ q E(w t q ) 0 (free of the time variable t) Arthur Berg STA 6857 Autocorrelation and Cross-Correlation & Stationary Time Series ( 1.4, 1.5) 10/ 25

Mean Function Examples Example (Mean of a Random Walk with Drift) We saw before that if x t = δ + x t 1 + w t where x 0 = 0, then x t has the representation t x t = δt + j=1 and the mean function of x t is µ t = E(x t ) = δt Example (Mean of Signal + Noise Model) If x t = s t + w t where w t is a mean zero time series, then µ t = E(x t ) = s t w j Arthur Berg STA 6857 Autocorrelation and Cross-Correlation & Stationary Time Series ( 1.4, 1.5) 11/ 25

The Autocovariance Function Definition (Autocovariance Function) The autocovariance function of a general random sequence {x t } is So in particular, we have γ(s, t) = cov(x s, x t ) = E [(x s µ s ) (x t µ t )] var(x t ) = cov(x t, x t ) = γ(t, t) = E [ (x t µ t ) 2] Also note that γ(s, t) = γ(t, s) since cov(x s, x t ) = cov(x t, x s ). Example (Autocovariance of White Noise) Let w t WN(0, σ 2 ). By the definition of white noise, we have { σ 2, s = t γ(s, t) = E(w s, w t ) = 0, s t Arthur Berg STA 6857 Autocorrelation and Cross-Correlation & Stationary Time Series ( 1.4, 1.5) 12/ 25

Autocovariance Function of MA(1) Example (Autocovariance of MA(1)) Let w t WN(0, σ 2 ) and x t = w t + θ 1 w t 1, then γ(s, t) = cov(w t + θ 1 w t 1, w s + θ 1 w s 1 ) = cov(w t, w s ) + θ 1 cov(w t, w s 1 )+ + θ 1 cov(w t 1, w s ) + θ1 2 cov(w t 1, w s 1 ) If s = t, then γ(s, t) = γ(t, t) = σ 2 + θ1σ 2 2 = (θ1 2 + 1)σ 2 If s = t 1 or s = t + 1, then γ(s, t) = γ(t, t + 1) = γ(t, t 1) = θ 1 σ 2 So all together we have (θ1 2 + 1)σ2, ifs = t γ(s, t) = θ 1 σ 2, if s t = 1 0, else Arthur Berg STA 6857 Autocorrelation and Cross-Correlation & Stationary Time Series ( 1.4, 1.5) 13/ 25

Autocovariance Function of a Random Walk Example (Autocovariance of a Random Walk)) If x t = t j=1 w j, then s γ(s, t) = cov(x s, x t ) = cov w j, = s j=1 k=1 j=1 t k=1 w k t cov (w j, w k ) = min(s, t)σ 2 Note the dependence on s and t is not just a function of s t! (The random walk is not stationary.) Arthur Berg STA 6857 Autocorrelation and Cross-Correlation & Stationary Time Series ( 1.4, 1.5) 14/ 25

The Autocorrelation Function Definition (Autocorrelation Function) The autocorrelation function (ACF) of a general random sequence {x t } is ρ(s, t) = γ(s, t) γ(s, s)γ(t, t) As is well know from the Cauchy-Schwartz inequality, the correlation of two random variables is bounded between -1 and 1. Hence ρ(s, t) 1. And ρ(s, t) = 1 implies an exact linear relationship between x s and x t. Arthur Berg STA 6857 Autocorrelation and Cross-Correlation & Stationary Time Series ( 1.4, 1.5) 15/ 25

The Cross-Covariance and Cross-Correlation Functions The predictability of one series on another is me measured with the cross-covariance and cross-correlation functions. Definition (Cross-Covariance Function) The cross-covariance function of two general time series {x t } and {y t } is defined as ρ xy (s, t) = E [(x s µ x s)(y t µ yt )] Definition (Cross-Correlation Function (CCF)) The cross-correlation function (CCF) of two general time series {x t } and {y t } is defined as γ xy (s, t) ρ xy (s, t) = γxy (s, s)γ xy (t, t) Arthur Berg STA 6857 Autocorrelation and Cross-Correlation & Stationary Time Series ( 1.4, 1.5) 16/ 25

Strict Stationarity Definition (Strict Stationarity) A time series {x t } is strictly stationary if any collection {x t1, x t2,..., x tn } has the same joint distribution as the time shifted set Strict stationarity implies the following: {x t1 +h, x t2 +h,..., x tn+h} All marginal distributions are equal, i.e. P(x s c) = P(x t c) for all s, t, and c. The autocovariance function is shift-independent, i.e. γ(s, t) = γ(s + h, t + h). Strict stationarity is typically assumes too much. This leads us to the weaker assumption of weak stationarity. Arthur Berg STA 6857 Autocorrelation and Cross-Correlation & Stationary Time Series ( 1.4, 1.5) 18/ 25

Weak Stationarity Definition (Weak Stationarity) A weakly stationary time series is a finite variance process that (i) has a mean function, µ t, that is constant (so it doesn t depend on t); (ii) has a covariance function, γ(s, t), that dependents on s and t only through the difference s t. From now on when we say stationary, we mean weakly stationary. Since the mean function is free of t, we will simply drop the t and write µ(= µ t ) Since γ(s, t) = γ(s + h, t + h), we have γ(s, t) = γ(s t, 0). So the autocovariance function can be thought of as a function of only one variable (h = s t). Arthur Berg STA 6857 Autocorrelation and Cross-Correlation & Stationary Time Series ( 1.4, 1.5) 19/ 25

Autocovariance and ACF of a Stationary Time Series Definition (Autocovariance Function of a Stationary Time Series) The autocovariance function of a stationary time series is γ(h) = E [(x t+h µ) (x t µ)] (for any value of t). Definition (Autocorrelation Function of a Stationary Time Series) The autocorrelation function of a stationary time series is ρ(h) = γ(t + h, t) = γ(h) γ(t + h, t + h)γ(t, t) γ(0) Arthur Berg STA 6857 Autocorrelation and Cross-Correlation & Stationary Time Series ( 1.4, 1.5) 20/ 25

Jointly Stationary Time Series Definition (Jointly Stationary) Two time series, {x t } and {y t }, are jointly stationary if the are each stationary, and the cross-covariance function is only a function of the lag h, i.e. is the same for all t. γ xy (h) = E [(x t+h µ x )(y t µ y )] Definition (CCF of Jointly Stationary Time Series) The Cross-Correlation function of jointly stationary time series {x t } and {y t } is ρ xy (h) = γ xy (h) γx (0)γ y (0) Arthur Berg STA 6857 Autocorrelation and Cross-Correlation & Stationary Time Series ( 1.4, 1.5) 21/ 25

Example Using Cross-Correlation Example (Prediction Using Cross-Correlation) Consider the model y t = Ax t l + w t where l is a positive integer. Then clearly one can use {x t } to predict y t. To be able to detect such a model, we consider the cross-correlation function. Assuming µ x = µ y = 0, we have γ xy (h) = E (y t+h x t ) = E [(Ax t+h l + w t+h )x t ] = E (Ax t+h l x t ) + E (w t+h x t ) = Aγ x (h l) So the cross-covariance function is a shifted and scaled version of the autocovariance function of the {x t } series. Arthur Berg STA 6857 Autocorrelation and Cross-Correlation & Stationary Time Series ( 1.4, 1.5) 22/ 25

Textbook Reading Read the following sections from the textbook 1.6 (Estimation of Correlation) 1.7 (Vector-Valued and Multidimensional Series) Arthur Berg STA 6857 Autocorrelation and Cross-Correlation & Stationary Time Series ( 1.4, 1.5) 24/ 25

Textbook Problems Do the following exercises from the textbook 1.6 1.7 1.9 (Note: cos(a B) = sin A sin B + cos A cos B) 1.13 1.15 Arthur Berg STA 6857 Autocorrelation and Cross-Correlation & Stationary Time Series ( 1.4, 1.5) 25/ 25