STAT 443 Final Exam Review. 1 Basic Definitions. 2 Statistical Tests. L A TEXer: W. Kong

Transcription

1 STAT 443 Final Exam Review L A TEXer: W Kong 1 Basic Definitions Definition 11 The time series {X t } with E[X 2 t ] < is said to be weakly stationary if: 1 µ X (t) = E[X t ] is independent of t 2 γ X (t, t + h) = Cov(X t, X t+h ) is independent of t for all h; the covariance only depends on the distance h instead of t 3 E[X 2 t ] < is also one of the conditions for weak stationarity Definition 12 Let x 1,, x n be observations of a time series The sample mean of x 1,, x n is x = 1 n The sample autocovariance function is ˆγ(h) = 1 n n h t=1 The sample autocorrelation function is ( xt+ h x ) (x t x), h ( n, n) ˆρ(h) = ˆγ(h), h ( n, n) ˆγ(0) n x i i=1 2 Statistical Tests The Shapiro-Wilk Test is as follows: H 0 : Y 1,, Y n come from a Gaussian distribution Reject H 0 if the p-value of this test is small In R, if the data is stored in the vector y, then use the command shapirotest(y) The Difference Sign Test is as follows: Count the number S of values such that y i y i 1 > 0 For large iid sequences µ S = E[S] = n 1, σs 2 = n For large n, S is approximately N(µ S, σs 2 ), therefore, W = S µ S σ 2 S N(0, 1) A large positive value of S µ S indicates the presence of increasing (decreasing) trend We reject (H 0 : data is random) if W > z 1 α/2 but this may not work for seasonal data 1

2 The Runs Test is as follows: Estimate the median and call it m Let n 1 be the number of observations > m and n 2 be the number of observations < m Let R be the number of consecutive observations which are all smaller (larger) than m For large iid sequences µ R = E[R] = 1 + 2n 1n 2, σr 2 = (µ R 1)(µ R 2) n 1 + n 2 n 1 + n 2 1 For large number of observations, R µ R σ R N(0, 1) 3 Filters and Smoothing 1 (Finite Moving Average Filter) Let q be a non-negative integer and consider the two-sided moving average of the series X t We have m t 1 2q + 1 q j= q X t j = 1 2q + 1 q m t j + 1 q Y t j 2q + 1 j= q 0 j= q 2 (Exponential Smoothing) For fixed α [0, 1] define the recursion ˆm t = αx t + (1 α) ˆm t 1 with initial condition ˆm 1 = X 1 This gives an exponentially decreasing weighted moving average where in the general t 2 case, t 2 ˆm t = α(1 α) j X t j + (1 α) t 1 X 1 Note that a smaller α creates a smoother plot compared to a larger α 3 (Polynomial Regression) This is just developing a parametric polynomial form of m t in the form where k is chosen arbitrarily m t = k β i t i 4 We can also eliminate the trend through differencing where i=0 X t = X t X t 1 = (1 B)X t and, B are known to be the differencing and backshift operators respectively Exponentiating these operators is equivalent to function composition In this case, we are applying differencing to get a stationary process (by eliminating the trend) Holt-Winters (Special Cases) In the case that β = γ = 0 we have no trend or seasonal updates in the H-W algorithm 2

3 Here, we have L t = αx t + (1 α)l t 1 which is exactly (simple) exponential smoothing under α In the case that γ = 0 we have no seasonal component and there are two H-W equations for updating L t and T t We call the above case double exponential smoothing 4 Linear Processes Definition 41 A process {X t } is called a moving average process of order q if X t = Z t + θ 1 Z t θ q Z t q where {Z t } W N(0, σ 2 ) and θ 1,, θ q are constants Sometimes Z t is referred to as the innovation Notice that these innovations are uncorrelated, have constant variance and zero mean Deriving the mean and autocovariance function of M A(q), it is easy to see that this process is stationary Definition 42 We say that a process {X t } is q dependent if X t and are X s are independent if t s > q That is, they are dependent if they are within q steps of each other Similarly, we saay that that stationary time series is q correlated if γ(h) = 0 whenever h > q Example 41 It is easy to show that the MA(q) process is q correlated The inverse of this statement is also true Proposition 41 If {X t } is a stationary q correlated time series with mean 0, then it can be represented as the MA(q) process Definition 43 process {X t } is called a autoregressive process of order p if X t = X t + φ 1 X t φ p X t p + Z t where {Z t } W N(0, σ 2 ) and φ 1,, φ p are constants Definition 44 {X t } is called a Gaussian time series if all its joint distributions are multivariate normal That is for any set i 1,, i m with each n N, the random vector (X i1,, X im ) follows a multivariate normal distribution Example 42 Consider the stationary Gaussian time series {X t } Suppose X n has been observed and we want to forecast X t+h using m(x n ), a function of X n Let us measure the quality of the forecast by ) MSE = E ([X n+h m(x n )] 2 X n It can be shown that m( ) which minimizes MSE in a general case is m(x n ) = E(X n+h X n ) Example 43 We now consider the problem of predicting X n+h, h > 0 for a stationary time series with known mean µ and ACVF γ( ) based on previous values {X n,, X 1 } showing the linear predictor of X n+h by P n X n+h We are interested in P n X n+h = a 0 + a 1 X n + a 2 X n a n X 1 which minimizes S(a 0,, a n ) = E [(X n+h P n X n+h ) 2] To get a 0, a 1,, a n we need to solve the system S a j = 0 for j = 0, 1,, n Doing so, we get ( ) n a 0 = µ 1 a i, Γ n a n = γ n (h) i=1 3

4 where a n = a 1 a 2 a n, Γ n = γ(0) γ(1) γ(n 1) γ(1) γ(0) γ(n 2) γ(n 1) γ(n 2) γ(0), γ n(h) = γ(h) γ(h + 1) γ(n + h 1) Here, a n = Γ 1 n γ n (h) = a n = Γ 1 n γ(0) ρ n(h) where ρ n (h) = γn(h) Note 1 Here are some properties from the above: P n X n+h is defined by µ, γ(h) γ(0) P n X n+1 = µ + n i=1 a i(x n+1 i µ) [ It can be shown that E (X n+h P n X n+h ) 2] = γ(0) a T n γ n (h) E(X n+h P n X n+h ) = 0 E [(X n+h P n X n+h )X j ] = 0 for j = 1, 2,, n Example 44 Derive the one-step prediction for the AR(1) model (Here, h = 1) To find the linear predictor, we need to solve Γ n a n = γ n (h) = Γ na n γ(0) = γ n(h) γ(0) 1 φ φ n 1 φ 1 φ n 2 = φ n 1 φ n 2 1 a 1 a 2 a n = φ φ 2 φ n An obvious solution is a n = a 1 a 2 a n You can use the formula of MSE to get = P nx n+1 = µ + = P n X n+1 = n a i (X n+1 i µ) i=1 n a i X n+1 i = a 1 X n + 0 = φx n i=1 MSE = γ(0) a T n γ n (h) = γ(0) φγ(1) = γ(0) φ 2 γ(0) = γ(0)[1 φ 2 ] = σ 2 5 Causal and Invertible Processes Definition 51 The time series {X t } is a linear process if X t = j= ψ jz t j for all t where {Z t } W N(0, σ 2 ) and ψ j is a sequence of constants such that j= ψ j < 4

5 Example 51 Show that AR(1) with φ < 1 is a linear process We know that X t = φx t 1 + Z t W N(0,σ 2 ) and we showed before that X t = φj Z t j Since φ < 1 then if ψ j = φ j then j= ψ j and therefore all assumptions in the definition above are satisfied So AR(1) is a linear process Definition 52 A linear process j= ψ jz t j is causal or future independent if ψ j = 0 for any j < 0 {X t, t T } is an ARMA(p, q) process if 1) {X t, t T } is stationary 2) X t φ 1 X t 1 φ 2 X t 2 φ p X t p = Z t + θ 1 Z t θ q Z t q where {Z t } W N(0, σ 2 ) 3) Polynomials (1 φ 1 z φ p z p ) and (1 + θ 1 z + + θ q z q ) have no common factors/roots (IMPORTANT FOR THE FINAL!) Definition 53 An ARMA(p, q) process φ(b)x t = θ(b)z t where Z t W N(0, σ 2 ) is causal if there exists constants {ψ j } such that ψ j < and X t = ψ jz t j for any t This condition is equivalent to for any z C such that z 1 Remark 51 f the condition above holds true, then and we have φ(z) = 1 φ 1 z 1 φ 2 z 2 φ p z p 0 θ(z) = ψ(z) φ(z) = θ(z) = φ(z) ψ(z) = 1 + θ 1 z + + θ q z q = (1 φ 1 z φ p z p )(ψ 0 + ψ 1 z + ) 1 = ψ 0 θ 1 = ψ 1 φ 1 ψ 0 Definition 54 An ARMA(p, q) process {X t } is invertible if there exists constants {Π j } such that Π j < and Z t = Π jx t j for all t Invertibility is equivalent to the condition θ(z) = 1 + θ 1 z + + θ q z q 0 for any z C such that z 1 Using the same methods above, one can get that Π 0 = 1 φ 1 = Π 0 θ 1 + Π 1 Example 52 Consider {X t, t T } satisfying X t 05X t 1 = Z t + 04Z t 1 where {Z t } W N(0, σ 2 ) Investigate the causality and invertibility of X t If the series is causal (invertible) then provide the causal (invertible) solutions These are called the M A( ) and AR( ) representations [Causality] We have φ(z) = 1 05z causal We then have = z = 2 = z > 1 Since this is outside the unit circle, X t is z = (1 05z)(ψ 0 + ψ 1 z + ) = ψ 0 = 1, ψ 1 05ψ 0 = 04, ψ 2 05ψ 1 = 0, = ψ 0 = 1, ψ 1 = 09, ψ 2 = 09(05), ψ 3 = 09(05) 2, 5

6 We can kind of see the pattern (and prove using induction) { ψ j = 1 j = 0 ψ j = ψ j = 09(05) j 1 j 0 = X t = Z t + 09 (05) j 1 Z t j [Invertibility] We have θ(z) = z = 0 = z = 10/4 = z > 1 Since this is outside the unit circle, X t is invertible We then have, like above, 1 05z = (1 + 04z)(Π 0 + Π 1 z + ) = Π 0 = 1, Π Π 0 = 05, Π Π 1 = 0, = Π 0 = 1, Π 0 = 09, ψ 2 = 09( 04), ψ 3 = 09( 04) 2, We can kind of see the pattern (and prove using induction) { ψ j = 1 j = 0 ψ j = ψ j = 09( 04) j 1 j 0 = X t = Z t 09 ( 04) j 1 Z t j Remark 52 (ACVF of ARMA processes) Consider a causal, stationary process φ(b)x t = θ(b)z t with Z t W N(0, σ 2 ) The MA( ) representation of X t is X t = ψ jz t j where E[X t ] = 0 We have γ(h) = E[X t X t+h ] E[X t ]E[X t+h ] =0 = E ψ j Z t j ψ j Z t+h j Notice that E[Z t Z s ] = 0 when t s We then have { γ(h) = ψ jψ j+h E[Zj 2 ] h 0 ψ jψ j h E[Zj 2] h < 0 = σ2 ψ j ψ j+ h Example 53 Derive the ACVF for the following ARM A(1, 1) process X t φx t 1 = Z t θz t 1 where Z t W N(0, σ 2 ) and φ < 1 Note that φ(z) is causal because 1 φz = 0 = z = 1/φ > 1 It can be shown, with similar methods above, that { ψ j = φ(φ + θ) j = 0 ψ j = ψ j = φ j 1 (φ + θ) j 0 Now if h = 0 then γ(0) = σ 2 ψj 2 = σ ψ 2 j = σ (θ + φ) 2 = σ 2 [ φ 2(j 1) ] 1 + (θ + φ) 2 φ 2i i=0 [ = σ (θ + ] φ)2 φ 1 φ 2 6

7 If h 0 then γ(0) = σ 2 ψ j ψ j+ h = σ 2 ψ 0 ψ h + ψ j ψ j+ h = σ 2 φ h 1 (θ + φ) + (θ + φ) 2 = σ 2 φ h 1 (θ + φ) + (θ + φ) 2 φ h 1 [ = σ 2 φ h 1 (θ + φ) + (θ + φ)2 φ h +1 ] 1 φ 4 φ j 1 φ j+ h φ 2j Summary 1 For ACF and PACF, we have the following summary: ACF PACF M A(q) Zero after lag q Decays exponentially AR(p) Decays exponentially Zero after lag p In the general case of ARMA processes, the PACF is defined as α(0) = 1 and α(h) = Φ hh for h 1 where Φ hh is the last component of the vector Φ h = Γ 1 h γ h in which γ(0) γ(1) γ(h 1) γ(1) γ(0) γ(h 2) Γ h =, γ h = γ(h 1) γ(h 2) γ(0) Example 54 Calculate α(2) for an M A(1) process X t = Z t + θz t 1, {Z t } W N(0, σ 2 ) γ(1) γ(2) γ(h) We have shown before that (1 + θ 2 )σ 2 h = 0 γ(h) = θσ 2 h = 1 0 h 2 We have Φ = Γ 1 h γ h So α(h) is the last element of Φ h and h = 1 = Φ 11 = (γ(0)) 1 γ(1) = γ(1) γ(0) = θ 1 + θ 2 ( ) (1 + θ h = 2 = 2 )σ 2 θσ 2 1 ( θσ 2 θσ 2 (1 + θ 2 )σ 2 0 ) ( = θ(1+θ 2 )σ 4 (1+θ 2 ) 2 σ 4 θ 2 σ 4 θσ 2 (1+θ 2 ) 2 σ 4 θ 2 σ 4 ) Where the last element of the case of h = 2, in reduced form, is It can be shown, in general, that α(2) = Φ 22 = θ θ 2 + θ 4 α(h) = Φ hh = ( θ)h h i=0 θ2h 7

8 6 ARIMA/SARIMA Models Definition 61 Let d be a non-negative integer {X t, t T } is an ARIMA(p, d, q) process if Y t = (1 B) d X t is a causal ARMA(p, q) process The definition above means that {X t, t T } satisfies an equation of the form φ (B)X t φ(b)(1 B) d X t = θ(b)z t, {Z t } W N(0, σ 2 ) Note that φ (1) = 0 = X t is not stationary unless d = 0 Therefore, {X t } is stationary iff d = 0 in which case it is reduced to an ARMA(p, q) process in the previous case Recall that if {X t } exhibits a polynomial trend of the form m(t) = α 0 + α 1 t + + α d t d then (1 B) d X t will not have that trend any more Therefore, ARIMA models (when d 0) are appropriate when the trend in the data is well approximated by a polynomial degree d Recall the operator B where B k X t = X t k Clearly (1 B k ) and (1 B) k are different filters The latter is performing k times differencing, but the former is differencing once in lag k In R, we will write diff(x,difference=k) (1 B) k X t diff(x,lag=k) (1 B k )X t Definition 62 If d, D are non-negative integers, then {X t t T } is a seasonal ARIMA(p, d, q) (P, D, Q) S process with period S if the differenced series is a causal ARMA process defined by Y t = d D S X t = (1 B) d (1 B S ) D X t φ(b)φ(b S )Y t = θ(b)θ(b S )Z t, Z t W N(0, σ 2 ) Remark 61 Notice that the process {X t, t T } is causal iff φ(z) 0 Φ(z) 0 for all z : z < 1 Example 61 Derive the ACF of SARIMA(0, 0, 1) 12 = SARIMA(0, 0, 0) (0, 0, 1) 12 This gives us the general form X t = Z t + Θ 1 Z t 12, Z t W N(0, σ 2 ) Show, as an exercise, that (1 + Θ 2 1)σ 2 h = 0 γ(h) = Cov(X t, X t+h ) = Θ 1 σ 2 h = 12 0 otherwise ρ(h) = γ(h) γ(0) = 1 h = 0 θ 1+θ h = otherwise Definition 63 Consider a causal AR(p) model (1) X t φ 1 X t 1 φ p X t p = Z t with causal solution X t = ψ jz t j where {Z t } W N(0, σ 2 ) Multiply both sides of (1) by X t j with j = 0, 1, 2,, p and taking expectations will give us E[X t X t j ] φ 1 E[X t 1 X t j ] φ p E[X t p X t j ] = E[Z t X t j ] = γ(j) φ 1 γ(j 1) φ p γ(j p) = E[Z t X t j ] 8

9 We then have { E[Zt X t j ] = E[Z t X t ] = E [Z t ψ jz t j ] = E[Z 2 t ] = σ 2 j = 0 E[Z t X t j ] = 0 j > 0 So the original equation reduces to { γ(0) φ 1 γ(1) φ p γ(p) = σ 2 j = 0 γ(j) φ 1 γ( j 1 ) γ( j p ) = 0 j 0 These are called the Yule-Walker equations This can be easily generalized to a matrix form Γ p φ = γ p Based on a sample {x 1, x 2,, x n } the parameters φ and σ 2 can be estimated by ˆφ = 1 ˆΓ p ˆγ p where the matrices are defined in a similar fashion as the best linear predictor section The system above is called the sample Yule-Walker equations We can write Yule-Walker equations in terms of ACF too Explicitly, if we divide ˆγ p by γ(0) and multiply it in ˆΓ p then ˆφ = 1 ˆR p ˆρ p ˆR p = ˆΓ p ˆγ(0) = 1 1 ˆR p = ˆΓ p ˆγ(0) ˆρ p = ˆγ p /ˆγ(0) where ˆσ 2 = ˆγ(0) [1 ˆφ ] ˆρ p Notice that ˆγ(0) is the sample variance of {x 1,, x n } Based on a sample {x 1,, x n }, the above equations will provide the parameter estimates Using advanced probability theory, it can be shown that φ 1 φ 1 φ = MV N φ =, σ2 n Γ 1 p φ p φ p for large n If we replace σ 2 and Γ p by their sample estimates ˆσ 2 and ˆΓ p we can use this result for large-sample confidence intervals for the parameters φ 1,, φ p Example 62 Based on the following sample ACF and PACF, an AR(2) has been proposed for the data Provide the Yule-Walker estimates of the parameters as well as 95% confidence intervals for the parameters in φ The data was collected over a window of 200 points with sample variance 369 with the following table: We want to estimate φ 1 and φ 2 in h ˆf(h) ˆα(h) X t = φ 1 X t 1 + φ 2 X t 2 + Z t, {Z t } N(0, σ 2 ) The system is Similarly, [ ˆφ = ] 1 [ [ [ ˆσ 2 = ˆγ(0) 1 ˆφ ˆρ(1) ˆρ(2) 369 ] = [ ]] = 1112 ] 9

10 Therefore the estimated model is X t = 0594X t X t 1 + Z t, {Z t } W N(0, 1112) Now φ N ) (φ, σ2 n Γ 1 2 ([ ] 0594 = N, 1112 [ ([ ] [ ]) = N, ]) So the 95% CI s for φ 1, φ 2 are ˆφ 1 ± 196 Vˆar( φ) = 0594 ± = (0455, 0733) ˆφ 2 ± 196 Vˆar( φ) = 0276 ± = (0137, 0415) 7 Forecasting We discuss how forecasting works under our studied processes 71 Forecasting AR(p) Let X t = p φ jx t j + Z t, Z t W N{0, σ 2 } be a causal AR(p) process We have ˆX n+h = E[X n+h X 1,, X n ], h > 0 h 1 p = E φ j X n+h j + φ j X n+h j X 1,, X n + E [Z n+h X 1,, X n ] j=h h 1 p = E φ j X n+h j X 1,, X n + E φ j X n+h j X 1,, X n due to the uncorrelatedness of Z n+h with respect to X k If h = 1, then the above equation becomes If h = 2, 3,, p then remark that and so ˆX n+1 = j=h p φ j X n+1 j j < h = n + h j > n j h = n + h j n =0 ˆX n+h = = p j=h h 1 φ j X n+h j + φ j E (X n+h j X 1,, X n ) h 1 φ j ˆXn+h j + p φ j X n+h j j=h 10

11 If h > p, then n + h j > n and ˆX n+h = p φ j E (X n+h j X 1,, X n ) = p φ j ˆXn+h j In summary, for a causal AR(p), the h step predictor is ˆX n+1 = p φ jx n+1 j h = 1 ˆX n+h = h 1 φ ˆX j n+h j + p j=h φ jx n+h j h = 2, 3,, p p φ ˆX j n+h j h > p In AR(p), the h step prediction is a linear combination of the previous steps We either have the previous p steps in X 1,, X n so we substitute the values (like the h = 1 case), or we don t have all or some of them, in which case we recursively predict Given a dataset, φ j can be estimated and ˆX n+h will be computed Example 71 Based on the annual sales data of a chain store, an AR(2) model with parameters ˆφ 1 = 1 and ˆφ 2 = 021 has bee fitted If the total sales of the last 3 years have been 9, 11 and 10 million dollars Forecast this year s total sales (2013) as well as that of 2015 We have Now X t = X t 1 021X t 2 + Z t, {Z t } W N(0, σ 2 ) ˆX 2013 = X X 2011 = 669 ˆX 2015 = ˆX ˆX 2013 = ˆX (669) and since then ˆX 2014 = ˆX ˆX 2012 = = 48 ˆX 2015 = (669) = Forecasting MA(q) MA processes are linear combinations of white noise To do forecasting in M A(q), we need to estimate θ 1,, θ q as well as approximate the innovations Z t, Z t+1, First, consider the very simple case of MA(1) where X t = Z t + θz t 1, {Z t } W N(0, σ 2 ) We have If h = 1, then the above equation is ˆX n+h = E [X n+h X 1,, X n ] = E [Z n+h X 1,, X n ] + θe [Z n+h 1 X 1,, X n ] ˆX n+1 = E [Z n+1 X 1,, X n ] +θe [Z n X 1,, X n ] =0 = θe [Z n X 1,, X n ] = θz n and if h > 1 then the equation becomes ˆX n+1 = E[Z n+h ] + θe Z n + h 1 X 1,, X n = 0 >n 11

12 Now we need to plug in a value for Z n We approximate the Z i s by U i s as follows Let U 0 = 0 and we estimate Ẑ t = U t = X t θu t 1, U 0 = 0 from the fact that Z t = X t θz t 1 We can then get that U 0 = 0 U 1 = X 1 U 2 = X 2 θx 1 U 3 = X 3 θx 2 + θ 2 X 1 Notice that as i, U i will need a convergence condition where θ < 1 is sufficient This was the invertibility condition for MA(1) We see that the U i s are recursively calculable and for an invertible MA(1) process, we have { θu n h = 1 ˆX n+h = 0 h > 1, U t = X t θu t 1, U 0 = 0 Now consider an MA(q) process X t = Z t + θ 1 Z t θ q Z t q We have ˆX n+h = E [X n+h X 1,, X n ] = E [Z n+h X 1,, X n ] + θ 1 E [Z n+h 1 X 1,, X n ] + + θ q E [Z n+h q X 1,, X n ] If h > q then the above equation s value is zero since we have n + h q > n If 0 < h q then at least some of the terms in the above are non-zero In particular, q ˆX n+h = θ j E [Z n+h 1 X 1,, X n ] = q θ j E [Z n+h 1 X 1,, X n ] and for j = h, h + 1,, q we know E[Z n+h j X 1,, X n ] = Z n+h j and hence j=h ˆX n+h = q θ j Z n+h j j=h Similar to MA(1), we approximate Z i s by U i s, provided the MA(q) process is invertible That is, θ(z) = 1 + θ 1 z + + θ q z q 0 for all z 1 Therefore, assuming that then U t = X t q θ ju t j and U 0 = U 1 = U 2 = = 0 U 0 = 0 U 1 = X 1 U 2 = X 2 θ 1 X 1 U 3 = X 3 θ 2 X 2 + θ 2 θ 1 X 1 In summary, for an invertible MA(q) process, we have { q j=h ˆX n+h = θ ju n+h j 1 h q 0 h > q where U 0 = U i = = 0, i < 0 and U t = X t q θ ju t j for t = 1, 2, 3, 12

13 Example 72 Consider the MA(1) process X t = Z t +05Z t 1 where {Z n } W N(0, σ 2 ) If X 1 = 03, X 2 = 01, X 3 = 01, predict X 4, X 5 Notice that ˆX 5 = ˆX 3+2 which is a 2-step prediction based on the history X 1 = X 2 = X 3 Since this is an MA(1) model, hence 1-correlated, ˆX5 = 0 For X 4 we have where ˆX 4 = U 0 = 0 and hence ˆX 4 = 05(0225) = = θ j U 3+1 j = θ 1 U 3 = 05U 3 U 1 = X 1 05U 0 = X 1 = 03 U 2 = X 2 05U 1 = 01 (05)(03) = 025 U 3 = X 3 05U 2 = 01 (05)( 025) = 0225 Example 73 Consider the MA(1) process X t = Z t + θz t 1 with {Z t } W N(0, σ 2 ) and θ < 1 Show that the one-step predictor ˆX n+1 = θu n is equal to the predictor ˆX n+1 = n ( θ) j X n j+1 This is by definition of U n which we can write the closed form and hence i=1 n 1 U n = X n + ( θ) i X n i, n 2 i=1 n 1 n 1 n ˆX n+1 = θu n = θx n ( θ) i+1 X n i = ( θ) i+1 X n i = ( θ) j X n j+1 = ˆXn+1 i=0 Clearly for n = 0, 1 we have ˆX n+1 = ˆXn+1 as well This shows that even in the MA process, the predictor may be written as a linear function of the history 13