Problem Set 2 Solution Sketches Time Series Analysis Spring 2010

Transcription

1 Problem Set 2 Solution Sketches Time Series Analysis Spring 2010 Forecasting 1. Let X and Y be two random variables such that E(X 2 ) < and E(Y 2 ) <. a) Compute Ê(Y X) (the linear projection of Y on X and a constant). Ê(Y X) = α 0 + α 1 X, where α 1 = cov(x, Y )/var(x) and α 0 = E(Y ) α 1 E(X). This is the population equivalent of a simple linear regression. b) Give a concrete example where E(Y X) Ê(Y X). There was an example in the last homework: If X N(0, 1) and Y = X 2 then E(Y X) = X 2 but Ê(Y X) = 2. (The difference between the conditional mean and linear projection is the reason we distinguish between martingale difference and white noise sequences.) 2. Let Y t be an MA( ) process: Y t = µ + ɛ t + ψ 1 ɛ t 1 + ψ 2 ɛ t , where ɛ t w.n. (0, σ 2 ) and i ψ2 i := S <. a) Let information at time t be given by I t = {ɛ t, ɛ t 1, ɛ t 2...}. Derive the optimal linear forecast of Y t+h, h > 0 based on I t under mean square loss. (Hint: follow the naive procedure and prove that it gives you the linear projection of Y t+h on I t.) Y t+h = µ + ɛ t+h + ψ 1 ɛ t+h 1 + ψ 2 ɛ t+h ψ h ɛ t + ψ h+1 ɛ t Claim: Ŷ t+h t = Ê(Y t+h I t ) = µ + ψ h ɛ t + ψ h+1 ɛ t To show that Ŷt+h t is indeed the linear projection of Y t+h on I t we need to verify E [(ɛ t+h + ψ 1 ɛ t+h ψ h 1 ɛ t+1 ) ɛ j ] = 0 for j t But this is immediate, since ɛ t is white noise. 1

2 b) Show that the forecast error associated with the optimal linear forecast of Y t+h follows an MA(h 1) process. e t,h = ɛ t+h + ψ 1 ɛ t+h ψ h 1 ɛ t+1 For h fixed this is precisely the definition of an MA(h 1) process. c) Derive the variance of the forecast error and show that it approaches the unconditional variance of Y t as h. V (Y t ) = V (ɛ t + ψ 1 ɛ t ) = σ 2 ( 1 + ψ1 2 + ψ ) V (e t,h ) = σ 2 ( 1 + ψ1 2 + ψ ψh 1 2 ) Clearly V (e t,h ) converges to V (Y t ) as h d) Assuming, in addition, that the marginal distribution of ɛ t is normal, construct a 95% interval forecast (i.e. a confidence interval for the value of Y t+h ). Using the normality assumption, the point forecast ± 2 standard errors is an approximate 95% CI. 3. (Forecasting under alternative loss functions.) One way to solve the general minimization problem min f E[L(Y t+h, f(x t ))] (1) is to construct a solution pointwise for each possible value of X t. Thus, to obtain the optimal predictor f : support(x t ) R, suppose that the observed value of X t is x t and find a number ŷ that solves the conditional problem min ŷ E[L(Y t+h, ŷ) X t = x t ] = min ŷ L(y, ŷ)df (y x t ), where F (y x t ) denotes the conditional distribution of Y t+h given X t = x t. Then one can set f (x t ) = ŷ. By the law of iterated expectations, and the monotonicity of the expectations operator, f will then solve (1). Suppose Y t follows an AR(1) process Y t = φy t 1 + ɛ t, 2

3 where φ < 1 and ɛ t i.i.d. unif[ 1, 1]. Furthermore, let L(y, ŷ) = a(y ŷ) if y > ŷ and L(y, ŷ) = b if y ŷ, where a and b are positive constants. a) Let X t = (Y t, Y t 1,...). Show rigorously that E(Y t+1 X t ) = φy t. Then derive the conditional distribution of Y t+1 given X t? One needs to show that E(ɛ t+1 Y t, Y t 1,...) = 0. Note that Y t j = s=0 φs ɛ t j s, j = 0, 1, 2,... In other words, Y t j depends only on shocks dated t j and earlier. Because of the iid assumption, ɛ t+1 is independent of (ɛ t, ɛ t 1,...), and so ɛ t+1 is independent of any function of (ɛ t, ɛ t 1,...), even vector-valued. It follows that ɛ t+1 is independent of (Y t, Y t 1,...). Therefore, E(ɛ t+1 Y t, Y t 1,...) = E(ɛ t+1 ) = 0. Clearly, the conditional distribution of Y t+1 given Y t is uniform[ 1 + φy t, 1 + φy t ]. b) Write down E[L(Y t+1, ŷ) X t ] as an integral. Split the integral into two parts depending on whether Y t+1 ŷ is positive or negative. Do the integration. For ŷ [ 1 + φy t, 1 + φy t ] we can write E[L(Y t+1, ŷ) X t ] = + 1+φYt ŷ ŷ a(y ŷ) [ 1+φY t,1+φy t](y)dy 1+φY t b [ 1+φY t,1+φy t](y)dy c) The expression you got in part b) should be a function of ŷ and X t. Find the minimum w.r.t. ŷ. This is the optimal predictor of Y t+1 under L given X t. d) Compare this predictor with E(Y t+1 X t ) = φy t, the optimal predictor under square loss. In particular, is there bias relative to E(Y t+1 X t )? Spectral analysis 4. Suppose you observe a time series at regular daily intervals (e.g. at noon every day). It should be intuitively clear that there is nothing you can learn about cycles with periods less than 1 day, i.e. frequencies greater than 2π. It is less clear, but also true, that you can t learn about cycles with periods shorter than 2 days, i.e. frequencies greater than π. (This is why the spectral density is defined for frequencies less than 3

4 π only.) To see this formally, let Y t = cos(ωt), ω > π, t Z. Show that there exists ω [0, π] such that Y t = cos(ω t) for all t. There exists an integer k > 0 and ω [ π, π] such that ω = ω + k2π. Then Y t = cos(ωt) = cos(k2πt + ωt) = cos( ωt), where the last equality follows because kt is an integer and the periodicity of the cosine function is 2π. Finally, note that cosine is an even function so that Y t = cos( ω t), where ω [0, π]. Set ω := ω. 5. Let Y t MA(1); in particular, let Y t = θɛ t 1 + ɛ t, where θ > 1 and ɛ t is white noise with variance σ 2. Show that the process has an invertible representation, i.e. there exists a white noise process η t such that Y t = 1 θ η t 1 + η t. (Hints: Given the form of the proposed representation, use the filtering theorem to derive the spectral density of η t. Show that it s constant, and argue that the fact that it s constant implies that the process is white noise.) Define η t := (1 + 1 θ L) 1 Y t. We need to show that η t is white noise. Now, Y t = (1 + θl)ɛ t so that η t = (1 + 1 θ L) 1 (1 + θl)ɛ t = h(l)ɛ t, where h(l) = (1 + 1 θ L) 1 (1 + θl). The spectral density of η t is then given by s η (ω) = h(e iω )h(e iω ) σ2 2π = θ2 σ 2 2π. Hence, η t has a constant spectral density. We know that white noise sequences have constant spectral densities, but is the reverse implication true? The answer is yes, because there is a one-to-one relationship between the autocorrelation sequence of a covariance stationary time series and its spectral density. This is the uniqueness part in the Herglotz theorem. 6.) Volatile time series are often smoothed out by some sort of averaging. In particular, let X t be a covariance stationary time series and let X t denote its smoothed version defined by an m-period centered moving average X t = (X t m X t 1 + X t + X t X t+m )/(2m + 1). a) Let m = 1. Using lag operator notation write down the linear filter that transforms X t into X t. X t = X t 1 + X t + X t+1 3 = ( 1 3 L ) 3 L 1 X t ; therefore, the linear filter that transforms X t into X t is h(l) = 1 3 L L 1. 4

5 b) Find the transfer function of the filter (i.e. the function by which the spectral density of X t has to be multiplied to obtain the spectral density of Xt ). ( 1 h(e iω )h(e iω ) = 3 eiω ) ( 1 3 e iω 3 e iω ) 3 eiω ( 1 = 3 eiω ) 2 3 e iω = 1 (1 + 2cos(ω))2 9 c) Compare the spectral density of X t with the spectral density of Xt. Calculate the zeros of the transfer function analytically. Then use Matlab or any other program to graph the rest of the filter function. Which frequencies are missing from the spectrum of Xt? Which frequencies are dampened? Which frequencies are amplified? Assuming you have monthly observations calculate the period of the cycles corresponding to these frequencies. (1 + 2cos(ω)) = 0 cos(ω) = 1 2 ω = 2π 3 (It s enough to find the root that lies in [0, π].) This means that cycles corresponding to the frequency ω = 2π 3 will be missing from the spectral density of Xt. For, say, monthly data, this frequency corresponds to a period of 2π ω = 3 months m= Frequency d) Explain why it is appropriate to describe the effect of averaging as smoothing. As seen in the figure, the transfer function is less than or equal to 1 for all frequencies in [0, π]. This means that the contribution of all cycles to the variance is reduced (V ( X t ) V (X t )). This is especially true for high frequency cycles. The variance contribution of cycles with frequencies greater than or equal to π 2 (periods less than or equal to 4 months) gets multiplied by a factor of 0.1 or less. Therefore, the time series plot of Xt should look smoother than X t. 7.) This is a real data exercise. The ascii text file clothing.txt contains 105 observations on the monthly inflation rate of the price index of clothing articles in Hungary from 1995:04 to 2003:12. 5

6 a) Write a code that calculates the Bartlett-estimator of the spectral density evaluated at the frequencies j2π/t, j = 1,..., (T 1)/2 for T odd. (See equations [6.3.14] and [6.3.15] in Hamilton.) b) Plot the estimated spectral density using (i) q = 20 and (ii) q = 40 autocorrelations. Describe briefly the effect of this choice on the estimator. Identify the cycles that seem to contribute the most to the variance of inflation. You should find a very pronounced peak at about frequency 1 to 1.1, which corresponds to a 6 month cycle. It s pretty clear that this cycle should be related to the change of seasons (summer vs. winter). There is a smaller but still obvious peak at frequency approx. 2.1, which gives a period of 3 months. This gives a shorter seasonal cycle. (Hungary has four pretty distinct seasons: winter, spring, summer and fall.) On q = 20 vs. q = 40. I thought this choice would make more of a difference, but the two graphs appear quite similar visually. The choice of q in the estimator governs the biasvariance tradeoff present in any nonparametric estimation method. Large q means using more estimated autocorrelations in estimating the spectrum this will give less bias but more variance (the estimated spectrum will appear very spikey or noisy ). A small q will give a smoother picture (less variance but more bias). No free lunch: oversmoothing might obscure details, while undersmoothing might introduce spurious details. Try q = 5 vs. q = 50 to really see what I am talking about. c) Apply the seasonal differencing filter (1 L 12 ) to the inflation rate. Estimate and plot the spectral density of the transformed sequence for a single choice of q. Interpret the result. The seasonal differencing filter will eliminate the 6-month and 3-month cycles (see the calculations on p of Hamilton). ARMA modeling 8.) This is a real data exercise. The ascii text file arma.txt contains 500 observations from an ARMA(p, q) process ordered as a column vector (the first entry is the earliest observation). Your job is to identify the underlying model, i.e. find p 0 and q 0 that adequately describes the data. You can work with any software package. 6

7 a) Estimate and graph the autocorrelation function along with the approximate 95% confidence bands. Estimate and graph the partial autocorrelation function along with the approximate 95% confidence bands. Make an educated guess about the values of p and q based on these graphs. Since both graphs show exponential-type decay, the process is likely to be mixed (i.e. ARMA as opposed to pure AR or MA). Though it s hard to be specific about the orders, note that the ACF shows a pretty big drop after the first lag and then the decay is much more gradual afterwards. To me this suggests just one MA term. b) Implement the Hannan-Rissanen procedure for various (p, q) combinations to see if your initial guess holds up. (Hint: You can use, say, K = 30 in the initial autoregression. I am also willing to let you know that p and q are both less than or equal to 6.) You ll continue to work with the model you identified here on the next problem set. The true process was ARMA(3,1). If your code is correct, you will indeed find that the minimum BIC occurs at p = 3 and q = 1. Notes on the ARMA identification problem Matlab notes If you are working with Matlab, here are a few hints. An easy way to get the data into Matlab is to put the text file in the Matlab work directory and use the command y=load( arma.txt ). This creates the vector y consisting of the sample observations. Also, there is an autocorr and parcorr function in Matlab, although I don t think it s part of the basic package. (It s pretty easy to write a routine that does the job anyway.) On the Hannan-Rissanen procedure Looking at my notes, I noticed that I gave you the form of the Bayesian Information Criterion (BIC) incorrectly (embarrassing once again). By way of compensation, I am giving you step-by-step instructions on the Hannan-Rissanen procedure. It has the correct formula for BIC. 7

8 Suppose Y t is to be modeled as an ARMA(p, q) process, with 0 p p and 0 q q. In this case one can choose p and q according to the Hannan-Rissanen procedure (Granger and Newbold 1986, p ). Let K > p be an integer and suppose there are q + K + T observations available on Y t, indexed as t = q K + 1,..., 0, 1,..., T. For a large K use OLS to estimate the autoregression Y t = a 0 + a 1 Y t a K Y t K + ɛ t, t = q + 1,..., T. Record the residuals ˆɛ t = Y t â 0 â 1 Y t 1... â K Y t K, t = q + 1,..., T. Use the residual sequence ˆɛ t to estimate (by OLS) an ARMA(p, q) model: Y t = c + φ 1 Y t φ p Y t p + θ 1ˆɛ t θ qˆɛ t q + e t for t = 1,..., T. Calculate the BIC log(ˆσ 2 p,q) + (p + q + 1) log T T, where ˆσ 2 p,q is the estimated error variance from the ARMA model above. Repeat the previous step for different values of (p, q) with 0 p p and 0 q q until you find the combination that minimizes this criterion. 8