10 Linear Prediction. Statistics 626

Transcription

1 10 Linear Prediction If we have a realization x(1),...,x(n) fromatimeseriesx, we would like a rule for how to take the x s and the probability properties of X to find the function of the data that best predicts future values in some optimal way. We visualize taking all realizations in the ensemble of realizations that have values x(1),...,x(n) for X(1),...,X(n), using our rule to predict what the value at time n + h would be, and choosing the rule that gives the right answer on the average and the smallest average squared distance of the predicted value from the actual value no matter what values we have for x(1),...,x(n). Such a rule is called the best unbiased predictor (BUP) of X(n + h) from X(1),...,X(n) and is calculated by E(X(n + h) X(1),...,X(n)), which is called the conditional expectation of X(n) given X(1),...,X(n). The technical definition of conditional expectation is very complicated (see the text) but for our purposes, it follows the same rules as does the usual expectation. Topic 10: Linear Prediction Copyright c 1999 by H.J. Newton Slide 1

2 If our populations X(1),...,X(n) are jointly normally distributed, then the BUP becomes a linear function of x(1),...,x(n) and can be easily calculated (as we will see below). If they are not normally distributed, then in general finding the BUP is next to impossible, so we will restrict ourselves to finding the best linear unbiased predictor (BLUP). Thus we need to study BLUP s carefully BLUPs for Covariance Stationary Time Series If X is a covariance stationary time series with autocovariance function R, then 1. The BLUP of X(n + h) given X(1),...,X(n) is ˆX nh = λ 1 X(n)+ + λ n X(1), where the vector λ of coefficients satisfies the prediction normal equations Γλ = r, where Γ is the (n n) Toeplitz matrix having R( j k ) as its jkth element, and the vector r of length n is (R(h),...,R(n + h 1)) T. 2. The variance of the h step ahead prediction error is given by ˆσ 2 nh = R(0) r T Γ 1 r. Topic 10: Linear Prediction Copyright c 1999 by H.J. Newton Slide 2

3 Ex: Consider a realization of length two from X MA(1,β =2,σ 2 =1. Then, since R(0) = 5, R(1)=2, and all other R s are zero, we have ˆX 21 = λ 1 X(2) + λ 2 X(1), where [ ][ ] 5 2 λ1 2 5 λ 2 = [ ] 2, 0 which gives λ 1 =10/21 and λ 2 = 4/21, so ˆX 21 = X(2) 4 21 X(1). Further, if we wanted to get a prediction interval for X(3), we would find [ ] ( ) 2 ˆσ 21 2 =5 (2 0) =85/21, and then we could say that 95 of the values of X(3) are in the interval ˆX 21 ± /21. Topic 10: Linear Prediction Copyright c 1999 by H.J. Newton Slide 3

4 10.2 Levison s Algorithm and Partial Autocorrelations From X(1),...,X(n), to get the h step ahead predictor ˆXnh of X(n + h) and its prediction error variance ˆσ 2 nh = Var(X(n + h) ˆX nh ),wemustsolve where Γ n λ nh = r nh, Γ n = Toepl(R(0),R(1),...,R(n 1)) λ nh = (λ nh (1),...,λ nh (n)) T r nh = (R(h),...,R(n + h 1)) T and then ˆX nh = λ nh (1)X(n)+ + λ nh (n)x(1) ˆσ 2 nh = R(0) r T nhγ 1 r nh. This appears to be a massive problem, both in storing the (n n) matrix Γ n as n could easily be in the thousands, and in the number of numerical operations needed to solve the system (it takes proportional to n 3 operations to solve a general system of equations). Fortunately, there exist a variety of remarkably effective computaional tricks to solve these problems, including an algorithm called Levinson s Topic 10: Linear Prediction Copyright c 1999 by H.J. Newton Slide 4

5 recursion that applies when h =1, that is, for doing one step ahead prediction. Levinson s Recursions: If we denote λ nh and ˆσ 2 nh by λ n and ˆσ 2 n when h =1,wehave λ 1 (1) = ρ(1), ˆσ 1 2 = R(0)(1 λ2 1 (1), and then for j =2,...,n: λ j (j) = R(j) j 1 k=1 λ j 1(k)R(j k), ˆσ 2 j 1 λ j (k) = λ j 1 (k) λ j (j)λ j 1 (j k), k =1,...,j 1, ˆσ j 2 = ˆσ j 1(1 2 λ 2 j(j)). Remarks: 1. This algorithm takes only proportional to n 2 numerical operations and one only need store two of the λ j vectors at any given point in the recursion (we only need the one for j 1 to get the one for j). 2. It can be shown that λ j (j) is the correlation between the errors in predicting X(t) from the next j 1 X s and in predicting X(t + j) from the previous j 1 X s and thus θ(j) =λ j (j) is defined to be the partial autocorrelation of lag j. Topic 10: Linear Prediction Copyright c 1999 by H.J. Newton Slide 5

6 3. At the jth step of the recursion, we have λ j (1),...,λ j (j) which are the coefficients needed to find the one step ahead predictor of X(j +1)given X(1),...,X(j). Thus a common procedure is to use the first X to predict the second, the first two to predict the third, and so on. Then we could calculate the set of one step ahead prediction errors e(2) = X(2) ˆX 11,...,e(n) =X(n) ˆX n 1,1, and taking as the best predictor of X(1) given no data to be the mean of the time series, we have e(1) = X(1) if the mean is zero (as is usually assumed). Topic 10: Linear Prediction Copyright c 1999 by H.J. Newton Slide 6