3 Theory of stationary random processes

Transcription

1 3 Theory of stationary random processes 3.1 Linear filters and the General linear process A filter is a transformation of one random sequence {U t } into another, {Y t }. A linear filter is a transformation of the form Y t = a j U t j. j= If {U t } is stationary and a finite number of the a j are nonzero, then {Y t } is also stationary. When infinitely many a j are nonzero, {Y t } may or may not be stationary, depending on the precise form of the a j. c Time Series General linear process: Y t = a j Z t j where Z t is white noise and the a j are constants. Mean: E [Y t ] = 0 Variance: Var [Y t ] = σ 2 a 2 j Autocovariance: Cov [Y s, Y t ] = σ 2 a i a t s+i i=0 Stationary: mean and variance are constant provided i=0 a 2 i <. The autocovariance depends on k = t s, again if the sum is finite. A necessary and sufficient condition for a general linear process to be stationary is a 2 i < i=0 c Time Series

2 In practice, a general linear process is a useful model only when its coefficients a j are expressible in terms of a finite number of parameters, which we can then hope to estimate from a set of data. A very rich class of models which satisfies this requirement is the class of autoregressive moving average (ARMA) processes. c Time Series ARMA processes {Z t } white noise is the basic building block of these processes. Moving Average, Y t MA(q), q Y t = Z t + β j Z t j Autoregressive, Y t AR(p), Y t = α i Y t i + Z t i=1 ARMA, Y t ARMA(p, q) q Y t = α i Y t i + Z t + β j Z t j i=1 The above assume a zero mean, but can consider q Y t µ = α i (Y t i µ) + Z t + β j Z t j i=1 c Time Series

3 3.3 Backward Shift Operator The above notation is relatively cumbersome, and it gets worse! We use the backward shift operator notation to simplify things. Recall: BY t = Y t 1, B j Y t = Y t j, and 1Y t = Y t. AR(p) may be written (1 α 1 B α p B p )Y t = Z t and the MA(q) may be written Y t = (1 + β 1 B + + β q B q )Z t Combining for ARMA(p, q) model: φ(b)y t = θ(b)z t, φ(b) = 1 α i B i q, θ(b) = 1 + β j B j i=1 c Time Series Note the side conditions φ(0) = θ(0) = 1 These eliminate redundancy in the specification of {Y t }. φ(b)y t = θ(b)z t assumes no factors in common. Operations involving B are assumed to follow algebraic lines, so, for example, ( 1 B 2 ) 1 Y t = = = ( ) B j 2 Y t 2 j B j Y t 2 j Y t j c Time Series

4 3.4 Second-order properties of Moving Average process, MA(q) Now, we know where Y t = θ(b)z t q θ(b) = β j B j with β 0 = 1 and θ(0) = 1. Autocovariance function: { γ k = σ 2 q k i=0 β i+kβ i k = 0, 1, 2,..., q 0 k > q Autocorrelations: for k = 0, 1, 2,..., q ρ k = q k i=0 β i+kβ i q i=0 β2 i The cut-off in γ k after k = q is a characteristic of the MA(q) process. Stationarity: for finite β j, always stationary. c Time Series Invertibility: Consider model MA(1): Y t = Z t + βz t 1 The only non-zero autocorrelation is ρ 1 = β 1+β 2 However if we replace β by 1/β, ρ 1 doesn t change. Thus the model Y t = Z t + 1 β Z t 1 has the same second order properties as the model above. To resolve the ambiguity, write and invert to give Y t = (1 + βb)z t Z t = (1 + βb) 1 Y t = ( 1) j β j Y t j, expressing {Z t } as a linear filter of {Y t }. Invertibility: influence of present and past Y t on Z t vanishes as lag increases if and only if 1 < β < 1. In general, invertibility means the roots of θ(u) = 0 must be greater than 1 in absolute value. c Time Series

5 Invertibility ensures a unique MA process for a given acf. Comment: MA processes have been used in many areas, particularly econometrics where, for example, economic indicators are affected by a variety of random effects such as strikes, terrorist attacks, government decisions, shortages of raw materials, etc. Such events will not only have an immediate effect, but may also affect economic indicators to a lesser extent in several subsequent periods. Thus it is at least plausible that an MA process may be an appropriate model. Example: See handout on simulated MA(1) processes: β = 0.9, 0.5, 0.5, 0.9, n = 200. We will discuss the behaviour of these series in the lectures. c Time Series Autoregressive process, AR(p) Recall: Y t is expressed as a linear combination of the past Y s and a white noise term Z t. Note also that it expresses the white noise sequence {Z t } as a linear filter of {Y t }. We write φ(b)y t = Z t where φ(b) = 1 α j B j and φ(0) = 1. Stationary if and only if all the roots of the equation φ(u) = 0 have modulus greater than one (this is a theorem; for the proof, see handout and Diggle, p.76 77). c Time Series

6 Example: AR(1). Thus, Y t = αy t 1 + Z t. Y t αy t 1 = Z t i.e. (1 αb)y t = Z t, where (1 αu) = φ(u). This has root 1/α. Provided α < 1, then 1/α > 1 as required for stationarity. Recall earlier work with this example where we assumed this result. Autocovariance function for AR(p) process: in general, need to solve the Yule-Walker equations ρ k = α j ρ k j, k = 1, 2,... c Time Series To see this, consider Y t = α j Y t j + Z t. Multiply both sides by Y t k to give Y t Y t k = α j Y t j Y t k + Z t Y t k. Taking expectations of both sides leads to γ k = E(Y t Y t k ) = α j γ k j (check this as an exercise). Finally, dividing both sides by γ 0 = var(y t ) gives the autocorrelation coefficients ρ k = α j ρ k j, k = 1, 2,.... This gives a way of calculating the ρ k from the α j : can solve by successive substitution, or solve as a system of linear difference equations. c Time Series

7 Difference Equations: pth order, constant coefficients λ k+p + a 1 λ k+p a p λ k = b k a 1, a 2,..., a p are known constants, b k an arbitrary function of k. Homogeneous equation: b k = 0 auxiliary equation: λ k = λ k is a solution iff λ p + a 1 λ p a p = 0 distinct roots r 1, r 2,..., r p, the solution is λ k = c 1 r k c pr k p some equal roots, for example r 1 = r 2 = = r m = r for some m, λ k = (c 1 + c 2 k + c m k m 1 )r k + c m+1 r k m c pr k p the constants c i are determined by initial conditions. Solution to general equation: general solution to homogeneous equation plus any particular solution to the general equation. c Time Series Thus we look for solutions to the Yule-Walker equations of the form ρ k = λ k The auxiliary equation is λ p α 1 λ p 1 α p = 0 In fact, the general solution for distinct roots λ 1,..., λ p, has the form ρ k = c 1 λ k c pλ k p [ Note that the Yule-Walker equations can be used to infer values of α j corresponding to an observed set of sample autocorrelation coefficients: this suggests a method of parameter estimation for autoregressive processes.] c Time Series

8 Example: AR(2). The general form of this process is Y t = α 1 Y t 1 + α 2 Y t 2 + Z t. This model generates a rich variety of second-order properties depending on the numerical values of the parameters α 1 and α 2. Write model as Find ρ k. Y t α 1 Y t 1 α 2 Y t 1 = Z t ( ) The Yule-Walker equations are ρ k = α 1 ρ k 1 + α 2 ρ k 2, k > 0 We look for solutions of the form ρ k = λ k. c Time Series Substitute into ( ) to give λ k = α 1 λ k 1 + α 2 λ k 2, i.e., λ 2 α 1 λ α 2 = 0, the auxiliary equation. We know that solutions to the auxiliary equation are solutions to ( ) as well. This quadratic equation has two roots: α 1 ± α α 2 2 Call these λ 1 = 1 2 ( ) α 1 + α α 2, λ 2 = 1 2 ( ) α 1 α α 2, c Time Series

9 Now, the nature of the solution depends on whether the roots are real and distinct, i.e., α α 2 > 0 real and coincident, i.e., α α 2 = 0, or complex, i.e., α α 2 < 0. In the lectures, we will obtain the general solution for each case and consider some illustrative examples. c Time Series ARMA(p,q) processes We write φ(b)y t = θ(b)z t where φ(b) = 1 α j B j q, θ(b) = 1 + β j B j, and φ(0) = 1, θ(0) = 1. Autocorrelation function is found in specific cases by expressing Y t as a general linear process, that is Y t = {φ(b)} 1 θ(b)z t = a j B j Z t where the coefficients a j are determined by a formal power series expansion of {φ(b)} 1. c Time Series

10 Stationarity is determined by the AR part, that is, if the roots of φ(u) = 0 are greater than one in absolute value. Invertibility: the ARMA(p,q) process is invertible if the roots of the polynomial equation θ(u) = 0 are all greater than one in absolute value. Example: the ARMA(1,1) process. See also separate handout of simulated ARMA processes. ARMA processes are valuable in time series analysis owing to their ability to approximate a wide range of second-order behaviour using only a small number of parameters. Additional motivation is their central role in Box- Jenkins forecasting, and their extension to ARIMA processes. Further comments are made at the end of Chapter 4. c Time Series